brainy/docs/architecture
David Snelling 2cdf70ee0f feat: subtype top-level field + trackField + migrateField
Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).

Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
  BaseStorage, mirrored after nounCountsByType with the same self-heal
  rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path

Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
  cardinality + per-NounType breakdown stats. Backed by the aggregation
  engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update

Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
  streams every entity, copies the value from one path to another, and
  (unless readBoth) clears the source. Supports top-level standard fields,
  metadata.X, and data.X paths. Idempotent — safe to re-run.

Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
  quick-start all treat subtype as a core primitive with anonymous example
  vocabularies (employee/customer/invoice/milestone).

Tests
- 26 new integration tests covering write/read/update/delete round-trips,
  counts rollup decrement + re-route on mutation, trackField + byField
  with and without perType, vocabulary whitelist enforcement, and
  migrateField for metadata.X → subtype and data.X → subtype paths
  including readBoth deprecation-window semantics.

Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:25:28 -07:00
..
aggregation.md feat: add stddev/variance aggregation ops with Welford's online algorithm 2026-02-17 17:04:11 -08:00
augmentation-system-audit.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00
augmentations-actual.md feat: migrate embeddings to Candle WASM + remove semantic type inference 2026-01-06 12:52:34 -08:00
augmentations.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00
CLUSTERING_ALGORITHMS_ANALYSIS.md docs: add deprecation warnings for addNoun and addVerb methods 2025-09-17 10:48:41 -07:00
data-storage-architecture.md fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
distributed-storage.md docs: add comprehensive scaling and storage architecture documentation 2025-09-08 14:49:25 -07:00
finite-type-system.md feat: subtype top-level field + trackField + migrateField 2026-06-04 17:25:28 -07:00
index-architecture.md feat: add aggregation engine with incremental SUM/COUNT/AVG/MIN/MAX, GROUP BY, and time windows 2026-02-16 16:57:53 -08:00
initialization-and-rebuild.md fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
multiprocess-storage-mixin.md docs: storage-adapter inheritance contract + correct the hasStorageMethod story 2026-05-15 13:20:18 -07:00
noun-verb-taxonomy.md docs: add public frontmatter to docs for soulcraft.com/docs pipeline 2026-02-19 17:04:05 -08:00
overview.md docs: add comprehensive index architecture documentation 2025-10-13 13:42:04 -07:00
PERFORMANCE_ANALYSIS.md feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
storage-architecture.md fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
triple-intelligence.md docs: add public frontmatter to docs for soulcraft.com/docs pipeline 2026-02-19 17:04:05 -08:00
zero-config.md docs: add public frontmatter to docs for soulcraft.com/docs pipeline 2026-02-19 17:04:05 -08:00