open-brainy/src
David Snelling 2cdf70ee0f feat: subtype top-level field + trackField + migrateField
Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).

Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
  BaseStorage, mirrored after nounCountsByType with the same self-heal
  rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path

Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
  cardinality + per-NounType breakdown stats. Backed by the aggregation
  engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update

Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
  streams every entity, copies the value from one path to another, and
  (unless readBoth) clears the source. Supports top-level standard fields,
  metadata.X, and data.X paths. Idempotent — safe to re-run.

Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
  quick-start all treat subtype as a core primitive with anonymous example
  vocabularies (employee/customer/invoice/milestone).

Tests
- 26 new integration tests covering write/read/update/delete round-trips,
  counts rollup decrement + re-route on mutation, trackField + byField
  with and without perType, vocabulary whitelist enforcement, and
  migrateField for metadata.X → subtype and data.X → subtype paths
  including readBoth deprecation-window semantics.

Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:25:28 -07:00
..
aggregation fix: deterministic code-point string collation for column store + aggregation 2026-05-26 17:35:18 -07:00
api refactor: remove src/cortex/ directory and fix README claims 2026-02-01 11:07:26 -08:00
cli feat: multi-process safety + read-only inspector mode 2026-05-15 11:25:05 -07:00
config fix: remove dead model config code for true zero-config 2025-12-18 10:31:02 -08:00
critical feat: replace transformers.js with direct ONNX WASM for Bun compatibility 2025-12-17 17:42:37 -08:00
data feat: Phase 3 - Unified Semantic Type Inference (Nouns + Verbs) 2025-10-16 10:59:26 -07:00
distributed feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
embeddings fix: suppress misleading 'Using Q8 WASM' log when Cortex native is active 2026-03-22 14:53:48 -07:00
errors fix: find()/stats() correctness + Cortex compat (BR-FIND-WHERE-ZERO, BR-DEFENSIVE-INTERFACE) 2026-05-15 12:31:28 -07:00
graph feat: export provider contracts for the plugin surface brainy consumes 2026-05-27 14:45:40 -07:00
hnsw feat: graph link compression — delta-varint connections (2.4.0 #3) 2026-05-28 10:37:47 -07:00
import fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
importers refactor: remove augmentation system and semantic type matching 2026-02-01 10:48:56 -08:00
indexes/columnStore feat: column-store JS↔native interchange — raw-blob unify (2.4.0 #4) 2026-05-28 10:37:47 -07:00
integrations refactor: remove augmentation system and semantic type matching 2026-02-01 10:48:56 -08:00
interfaces fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
mcp refactor: remove augmentation system and semantic type matching 2026-02-01 10:48:56 -08:00
migration feat: add migration system with error handling, validation, and enterprise hardening 2026-02-09 16:13:14 -08:00
neural feat: array-unnest groupBy for aggregates + batch-embed entity extraction 2026-05-26 14:20:40 -07:00
patterns 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
scripts feat: add node: protocol to all Node.js built-in imports for bundler compatibility 2025-09-17 14:20:21 -07:00
storage feat: subtype top-level field + trackField + migrateField 2026-06-04 17:25:28 -07:00
streaming feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
transaction fix: flush all native providers on shutdown to prevent data loss 2026-02-01 16:23:49 -08:00
triple fix: flush all native providers on shutdown to prevent data loss 2026-02-01 16:23:49 -08:00
types feat: subtype top-level field + trackField + migrateField 2026-06-04 17:25:28 -07:00
universal fix: prevent fs adapter from throwing on import in browser 2025-09-17 16:22:02 -07:00
utils feat: subtype top-level field + trackField + migrateField 2026-06-04 17:25:28 -07:00
versioning fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
vfs fix: set verb.source/target to entity UUID instead of NounType 2026-02-02 09:20:38 -08:00
brainy.ts feat: subtype top-level field + trackField + migrateField 2026-06-04 17:25:28 -07:00
coreTypes.ts feat: subtype top-level field + trackField + migrateField 2026-06-04 17:25:28 -07:00
index.ts feat: add stddev/variance aggregation ops with Welford's online algorithm 2026-02-17 17:04:11 -08:00
internals.ts feat(8.0): EntityIdMapper U32 ceiling + EntityIdSpaceExceeded error 2026-06-02 10:20:01 -07:00
pipeline.ts refactor: remove augmentation system and semantic type matching 2026-02-01 10:48:56 -08:00
plugin.ts feat(plugin): DiskAnnProvider contract + HNSWConfig.type/diskann knobs 2026-05-28 14:29:12 -07:00
setup.ts fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
unified.ts chore: enforce consistent coding style and semicolon removal 2025-09-29 09:50:59 -07:00
worker.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00