diff --git a/README.md b/README.md index 40f82a37..a3ec8378 100644 --- a/README.md +++ b/README.md @@ -247,26 +247,49 @@ Plugins are opt-in. Brainy never auto-imports packages unless listed in `plugins Model any domain — healthcare (`Patient → diagnoses → Condition`), finance (`Account → transfers → Transaction`), education (`Student → completes → Course`), or your own. -### Subtypes — sub-classification within a NounType +### Subtypes — sub-classification within a NounType *or* VerbType -The 42 NounTypes are intentionally coarse. Use the top-level `subtype` field to sub-classify entities within a type — flat string, no hierarchy, your choice of vocabulary: +Both noun types and verb types are intentionally coarse. Use the top-level `subtype` field to sub-classify entities AND relationships within a type — flat string, no hierarchy, your choice of vocabulary: ```javascript +// Nouns: sub-classify entities await brain.add({ data: 'Avery Brooks — runs the AI lab', type: NounType.Person, subtype: 'employee' // 'customer', 'vendor', 'contractor', … }) +// Verbs: sub-classify relationships +await brain.relate({ + from: ceoId, + to: vpId, + type: VerbType.ReportsTo, + subtype: 'direct' // 'dotted-line', 'matrix', … +}) + // Filter on the fast path — column-store hit, not metadata fallback: const employees = await brain.find({ type: NounType.Person, subtype: 'employee' }) +const directReports = await brain.getRelations({ from: ceoId, subtype: 'direct' }) -// O(1) counts via the persisted rollup: +// O(1) counts via the persisted rollups: brain.counts.bySubtype(NounType.Person) // → { employee: 12, customer: 847, vendor: 34 } + +brain.counts.byRelationshipSubtype(VerbType.ReportsTo) +// → { direct: 12, 'dotted-line': 3 } ``` -For other facets you want counted (`status`, `source`, `role`), register them with `brain.trackField(name)`. Renaming an existing convention to `subtype`? Use `brain.migrateField({from, to})`. Full guide: **[Subtypes & Facets](docs/guides/subtypes-and-facets.md)**. +**Enforce the pairing.** Register a vocabulary per type or turn on brain-wide strict mode to ensure every entity AND relationship has both `type` AND `subtype`: + +```javascript +// Per-type rule with vocabulary +brain.requireSubtype(NounType.Person, { values: ['employee', 'customer'], required: true }) + +// Or brain-wide strict mode +const brain = new Brainy({ requireSubtype: true }) +``` + +For other facets you want counted (`status`, `source`, `role`), register them with `brain.trackField(name)`. Renaming an existing convention to `subtype`? Use `brain.migrateField({from, to, entityKind: 'both'})` to walk nouns AND verbs in one pass. Full guide: **[Subtypes & Facets](docs/guides/subtypes-and-facets.md)**. **[Noun-Verb Taxonomy](docs/architecture/noun-verb-taxonomy.md)** | **[Stage 3 Canonical Reference](docs/STAGE3-CANONICAL-TAXONOMY.md)** diff --git a/RELEASES.md b/RELEASES.md index 3913618e..d82f0c84 100644 --- a/RELEASES.md +++ b/RELEASES.md @@ -11,6 +11,190 @@ Collective. The SDK wraps it — most products never call Brainy directly. Read --- +## v7.30.0 — 2026-06-05 + +**Affected products:** consumers modeling typed relationships with sub-classification +(direct vs dotted-line management; spouse / sibling / colleague; collaborator vs competitor; +etc.), and anyone wanting to enforce the pairing of `type` + `subtype` on every write. +Additive; drop-in from 7.29.x. No deprecations. + +### Symmetric — `subtype` on relationships (parity with 7.29.0 nouns) + +`subtype?: string` is now a first-class standard field on every relationship, mirroring the +noun-side work shipped in 7.29.0. Verbs and nouns are now first-class peers — every API +available on the noun side has a verb-side mirror. + +```typescript +const ceoId = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'Avery' }) +const vpId = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'Jordan' }) + +await brain.relate({ + from: ceoId, + to: vpId, + type: VerbType.ReportsTo, + subtype: 'direct' // sub-classification on the edge +}) +``` + +**Read & filter:** + +```typescript +// Fast-path filter — column-store hit, not metadata fallback +const direct = await brain.getRelations({ + from: ceoId, + type: VerbType.ReportsTo, + subtype: 'direct' +}) + +// Set membership +const all = await brain.getRelations({ + from: ceoId, + type: VerbType.ReportsTo, + subtype: ['direct', 'dotted-line'] +}) + +// Traversal filter (depth-1 in JS; multi-hop lands on Cortex native) +const reports = await brain.find({ + connected: { from: ceoId, via: VerbType.ReportsTo, subtype: 'direct', depth: 1 } +}) +``` + +### New — `updateRelation()` closes a long-standing gap + +Verbs previously had no update method — the only way to change a relationship was +delete-then-recreate, which lost the relation id. 7.30 ships `brain.updateRelation()`: + +```typescript +await brain.updateRelation({ id: relId, subtype: 'dotted-line' }) +await brain.updateRelation({ id: relId, weight: 0.5, confidence: 0.9 }) + +// Change verb type — re-indexes in graph adjacency, id preserved +await brain.updateRelation({ id: relId, type: VerbType.WorksWith }) +``` + +### New — O(1) verb subtype counts via the persisted rollup + +`_system/verb-subtype-statistics.json` mirrors the noun-side rollup shipped in 7.29.0. +Per-VerbType-per-subtype counts are maintained incrementally and persisted; reads are O(1): + +```typescript +brain.counts.byRelationshipSubtype(VerbType.ReportsTo) +// → { direct: 12, 'dotted-line': 3 } + +brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'direct') // O(1) point +// → 12 + +brain.counts.topRelationshipSubtypes(VerbType.ReportsTo, 3) +// → [['direct', 12], ['dotted-line', 3]] + +brain.relationshipSubtypesOf(VerbType.ReportsTo) +// → ['direct', 'dotted-line'] +``` + +### New — `brain.requireSubtype(type, options)` per-type enforcement + +Unified API for noun OR verb types — register specific types as requiring a subtype, +optionally with a fixed vocabulary: + +```typescript +brain.requireSubtype(NounType.Person, { + values: ['employee', 'customer', 'vendor'], + required: true +}) + +brain.requireSubtype(VerbType.ReportsTo, { + values: ['direct', 'dotted-line'], + required: true +}) + +// Now this throws — Person requires subtype: +await brain.add({ type: NounType.Person, data: 'no subtype' }) + +// And this throws — 'matrix' isn't in the registered vocabulary: +await brain.relate({ from: a, to: b, type: VerbType.ReportsTo, subtype: 'matrix' }) +``` + +### New — Brain-wide strict mode + +`new Brainy({ requireSubtype: true })` enforces subtype on every public write across the +whole brain. Composes with per-type rules; per-type rules win when both apply. + +```typescript +// Every write must include subtype +const brain = new Brainy({ requireSubtype: true }) + +// Exempt specific types (e.g. catch-all Thing) +const brain2 = new Brainy({ + requireSubtype: { except: [NounType.Thing, NounType.Custom] } +}) +``` + +When strict mode is on: +- Every `add()` / `addMany()` / `update()` / `relate()` / `relateMany()` / `updateRelation()` + rejects writes missing a subtype on a non-exempt type. +- `addMany()` and `relateMany()` validate every item BEFORE any storage write — + atomic-fail semantics, no partial writes. +- Brainy's own infrastructure writes (VFS root, directories, files) bypass via the + `metadata.isVFSEntity: true` marker so existing consumers' VFS usage continues to work. + +The brain-wide flag becomes the default in 8.0.0; the type-level `subtype` field becomes +required at the type system level. The full 8.0 contract upgrade is coordinated through the +internal platform handoff (`CTX-SUBTYPE-8.0-CONTRACT`). + +### `migrateField()` extended to verbs + +The migration helper shipped in 7.29.0 now walks verbs too via the new `entityKind` option: + +```typescript +// Migrate verb-side metadata.kind → top-level subtype +await brain.migrateField({ + from: 'metadata.kind', + to: 'subtype', + entityKind: 'verb' +}) + +// Or walk nouns and verbs in one pass +await brain.migrateField({ + from: 'metadata.kind', + to: 'subtype', + entityKind: 'both' +}) +``` + +Default is `entityKind: 'noun'` (backward-compatible). + +### Symmetry — noun + verb capability matrix + +7.30 closes every gap between nouns and verbs. Every capability available on the noun side +has a verb-side mirror: + +| Capability | Nouns | Verbs | +|---|---|---| +| `subtype` top-level field | ✓ (7.29) | ✓ (7.30 new) | +| Standard-field set | `STANDARD_ENTITY_FIELDS` | `STANDARD_VERB_FIELDS` (new) | +| Field resolver helper | `resolveEntityField` | `resolveVerbField` (new) | +| Statistics rollup | `_system/subtype-statistics.json` | `_system/verb-subtype-statistics.json` (new) | +| Counts breakdown | `counts.bySubtype` / `topSubtypes` / `subtypesOf` | `counts.byRelationshipSubtype` / `topRelationshipSubtypes` / `relationshipSubtypesOf` (new) | +| Fast-path filter | `find({type, subtype})` | `getRelations({type, subtype})` + `find({connected, subtype})` (new) | +| Update method | `update()` | `updateRelation()` (new — closed pre-7.30 gap) | +| Migration helper | `migrateField()` | `migrateField({entityKind: 'verb'\|'both'})` (new) | +| Enforcement | `requireSubtype(NounType, ...)` | `requireSubtype(VerbType, ...)` — one unified API | +| Brain-wide strict mode | `new Brainy({ requireSubtype })` covers both | same | + +### Cortex compatibility + +Verb subtype works under Cortex out of the box via auto-field-indexing. Cortex parity items +for the verb side (native query planner recognition, `verbSubtypeCountsByType` native rollup, +per-edge subtype on native graph adjacency for multi-hop traversal filtering) ship in the +next Cortex release. Not a Brainy blocker. + +### Docs + +Full guide: `docs/guides/subtypes-and-facets.md` (extended with Layer V + Enforcement +sections). The verb subtype + enforcement APIs are documented in `docs/api/README.md`. + +--- + ## v7.29.0 — 2026-06-04 **Affected products:** anyone modeling entities with per-product sub-classification — every diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md index ec28ecb8..e683238f 100644 --- a/docs/DATA_MODEL.md +++ b/docs/DATA_MODEL.md @@ -221,6 +221,47 @@ await brain.add({ `subtype` lives at the **top level** — NOT inside `metadata`, NOT inside `data`. That's how `find({ type, subtype })` routes through the standard-field fast path (column-store hit) instead of the metadata fallback. See **[Subtypes & Facets](./guides/subtypes-and-facets.md)** for the full guide including `trackField()` and `migrateField()`. +### Subtype — sub-classification within a VerbType (7.30+) + +Relationships are first-class citizens too. Every verb (`VerbType`) gets the same `subtype` primitive — a `ReportsTo` relationship might carry `subtype: 'direct'` vs `'dotted-line'`; a `RelatedTo` edge might carry `'spouse'` / `'sibling'` / `'colleague'`. Same shape as the noun side: flat string, no hierarchy, top-level standard field on `HNSWVerbWithMetadata` and on the public `Relation`: + +```typescript +await brain.relate({ + from: ceoId, + to: vpId, + type: VerbType.ReportsTo, + subtype: 'direct', // top-level standard field + metadata: { since: '2025-Q1' } // user-custom fields stay in metadata +}) +``` + +Fast-path filter on the verb side: + +```typescript +const direct = await brain.getRelations({ + from: ceoId, + type: VerbType.ReportsTo, + subtype: 'direct' +}) +``` + +The verb-side rollup at `_system/verb-subtype-statistics.json` mirrors the noun-side `_system/subtype-statistics.json` — same shape, same self-heal machinery. Per-VerbType-per-subtype counts are O(1) via `brain.counts.byRelationshipSubtype()`. + +Verbs and nouns now have full capability parity — every API on the noun side has a verb-side mirror, including the new `brain.updateRelation()` (which closed a pre-7.30 gap where relationships had no update path). + +### Standard verb fields + +The verb-side equivalent of `STANDARD_ENTITY_FIELDS` is `STANDARD_VERB_FIELDS`, exported from `src/coreTypes.ts`. Verb-specific standard fields: + +| Field | Description | +|---|---| +| `verb` | The VerbType enum value | +| `sourceId` / `targetId` | The two endpoints of the relationship | +| `subtype` | Sub-classification within the VerbType (7.30+) | +| `confidence`, `weight`, `createdAt`, `updatedAt`, `service`, `createdBy`, `data` | Same semantics as the noun-side standard fields | + +The companion `resolveVerbField(verb, field)` helper resolves field paths the same way `resolveEntityField` does for nouns: standard fields first, metadata fallback for everything else. + --- ## See Also diff --git a/docs/QUERY_OPERATORS.md b/docs/QUERY_OPERATORS.md index 5cfc5410..a13b0f62 100644 --- a/docs/QUERY_OPERATORS.md +++ b/docs/QUERY_OPERATORS.md @@ -214,6 +214,37 @@ brain.find({ See the **[Subtypes & Facets guide](./guides/subtypes-and-facets.md)** for the full surface. +### Filter relationships by subtype (7.30+) + +Verbs are first-class peers — `getRelations()` and graph traversal both honor subtype filters on the fast path: + +```typescript +// Filter relationships by VerbType subtype +const direct = await brain.getRelations({ + from: ceoId, + type: VerbType.ReportsTo, + subtype: 'direct' +}) + +// Set membership on verb subtype +const all = await brain.getRelations({ + from: ceoId, + type: VerbType.ReportsTo, + subtype: ['direct', 'dotted-line'] +}) + +// Graph traversal — subtype filters traversal edges (depth-1 in 7.30 JS path; +// multi-hop subtype filtering lands on Cortex native) +const reports = await brain.find({ + connected: { + from: ceoId, + via: VerbType.ReportsTo, + subtype: 'direct', + depth: 1 + } +}) +``` + ### Combine semantic search with filters ```typescript diff --git a/docs/api/README.md b/docs/api/README.md index 22516a83..91fedd9b 100644 --- a/docs/api/README.md +++ b/docs/api/README.md @@ -608,9 +608,10 @@ Create a typed relationship between entities. const relId = await brain.relate({ from: sourceId, to: targetId, - type: VerbType.RelatedTo, - data: 'Collaborated on the research paper', // Optional: content for this edge - metadata: { // Optional: structured edge fields + type: VerbType.ReportsTo, + subtype: 'direct', // Optional: sub-classification + data: 'Collaborated on the research paper', // Optional: content for this edge + metadata: { // Optional: structured edge fields strength: 0.9, role: 'primary author' } @@ -621,6 +622,7 @@ const relId = await brain.relate({ - `from`: `string` - Source entity ID (must exist) - `to`: `string` - Target entity ID (must exist) - `type`: `VerbType` - Relationship type +- `subtype?`: `string` - Per-product sub-classification within the VerbType (top-level standard field, fast-path indexed). See [Subtypes & Facets](../guides/subtypes-and-facets.md). - `data?`: `any` - Content for the relationship (overrides auto-computed vector) - `metadata?`: `object` - Structured edge fields - `weight?`: `number` - Connection strength (0-1, default: 1.0) @@ -631,6 +633,35 @@ const relId = await brain.relate({ --- +### `updateRelation(params)` → `Promise` + +Update an existing relationship. Mirror of `update()` for verbs — closed a long-standing gap (verbs had no update path before 7.30). + +```typescript +// Change the subtype on an existing relationship +await brain.updateRelation({ id: relId, subtype: 'dotted-line' }) + +// Update weight + confidence +await brain.updateRelation({ id: relId, weight: 0.7, confidence: 0.9 }) + +// Change verb type (re-indexes in graph adjacency, id preserved) +await brain.updateRelation({ id: relId, type: VerbType.WorksWith }) +``` + +**Parameters:** +- `id`: `string` - Relationship ID (required) +- `type?`: `VerbType` - Change verb type (re-indexes in graph adjacency) +- `subtype?`: `string` - Change sub-classification (omit to preserve existing) +- `weight?`: `number` - New weight (0-1) +- `confidence?`: `number` - New confidence (0-1) +- `data?`: `any` - New content +- `metadata?`: `object` - Metadata to merge (or replace with `merge: false`) +- `merge?`: `boolean` - Merge or replace metadata (default: true) + +**Returns:** `Promise` + +--- + ### `getRelations(params)` → `Promise` Get relationships for an entity. @@ -647,14 +678,32 @@ const related = await brain.getRelations({ from: entityId, type: VerbType.Contains }) + +// Filter by subtype (fast path, column-store hit) +const direct = await brain.getRelations({ + from: entityId, + type: VerbType.ReportsTo, + subtype: 'direct' +}) + +// Set membership on subtype +const all = await brain.getRelations({ + from: entityId, + type: VerbType.ReportsTo, + subtype: ['direct', 'dotted-line'] +}) ``` **Parameters:** - `from?`: `string` - Source entity ID - `to?`: `string` - Target entity ID -- `type?`: `VerbType` - Filter by relationship type +- `type?`: `VerbType | VerbType[]` - Filter by relationship type +- `subtype?`: `string | string[]` - Filter by VerbType subtype (top-level standard field, fast path) +- `service?`: `string` - Multi-tenancy filter +- `limit?`: `number` - Pagination limit (default: 100) +- `offset?`: `number` - Pagination offset -**Returns:** `Promise` - Matching relationships +**Returns:** `Promise` - Matching relationships (each with `subtype` at top level when set) --- @@ -1970,6 +2019,81 @@ brain.subtypesOf(NounType.Person) // → ['customer', 'employee', 'vendor'] ``` +#### `counts.byRelationshipSubtype(verb, subtype?)` → `Record | number` + +Verb-side mirror of `counts.bySubtype`. O(1) per-VerbType-per-subtype counts. + +```typescript +brain.counts.byRelationshipSubtype(VerbType.ReportsTo) +// → { direct: 12, 'dotted-line': 3 } + +brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'direct') +// → 12 +``` + +#### `counts.topRelationshipSubtypes(verb, n=10)` → `Array<[subtype, count]>` + +Top N subtypes for a `VerbType` ranked by count. + +```typescript +brain.counts.topRelationshipSubtypes(VerbType.ReportsTo, 3) +// → [['direct', 12], ['dotted-line', 3]] +``` + +#### `relationshipSubtypesOf(verb)` → `string[]` + +Sorted distinct subtypes seen for a `VerbType`. + +```typescript +brain.relationshipSubtypesOf(VerbType.ReportsTo) +// → ['direct', 'dotted-line'] +``` + +#### `requireSubtype(type, options?)` → `void` + +Register subtype enforcement for a specific `NounType` or `VerbType`. Unified API for nouns and verbs. Composes with the brain-wide `requireSubtype` constructor flag. + +```typescript +// Lock down Person sub-classification +brain.requireSubtype(NounType.Person, { + values: ['employee', 'customer', 'vendor'], + required: true +}) + +// Lock down management edges +brain.requireSubtype(VerbType.ReportsTo, { + values: ['direct', 'dotted-line'], + required: true +}) +``` + +**Parameters:** +- `type`: `NounType | VerbType` - The type to register +- `options.values?`: `string[]` - Vocabulary whitelist (rejects off-vocab values) +- `options.required?`: `boolean` - Whether subtype is required (default: `true`) + +#### Brain-wide strict mode — `new Brainy({ requireSubtype })` + +Constructor option that enforces subtype on every `add()` / `addMany()` / `update()` / `relate()` / `relateMany()` / `updateRelation()` for every type: + +```typescript +// Every write must include subtype +const brain = new Brainy({ requireSubtype: true }) + +// Exempt specific types (e.g. catch-all Thing) +const brain2 = new Brainy({ + requireSubtype: { except: [NounType.Thing, NounType.Custom] } +}) +``` + +When strict mode is on: +- Every public write path checks the pairing guarantee. +- `addMany()` / `relateMany()` validate all items BEFORE any storage write — atomic-fail, no partial writes. +- Brainy's own VFS infrastructure writes bypass via the `metadata.isVFSEntity: true` marker. +- Per-type registrations always apply regardless of the brain-wide flag. + +Becomes the default in 8.0.0. + #### `trackField(name, options?)` → `void` Register a metadata field for cardinality + per-NounType breakdown stats. With `values: [...]`, validates against the whitelist on `add()`/`update()`. diff --git a/docs/architecture/finite-type-system.md b/docs/architecture/finite-type-system.md index c8f70c75..76492ee5 100644 --- a/docs/architecture/finite-type-system.md +++ b/docs/architecture/finite-type-system.md @@ -467,7 +467,11 @@ brain.counts.bySubtype(NounType.Person) // → { employee: 12, customer: 847 } ``` -Subtype has its own statistics rollup (`_system/subtype-statistics.json`) maintained alongside `nounCountsByType`, so per-subtype counts stay O(1) at billion scale. Full guide: **[Subtypes & Facets](../guides/subtypes-and-facets.md)**. +Subtype has its own statistics rollup (`_system/subtype-statistics.json`) maintained alongside `nounCountsByType`, so per-subtype counts stay O(1) at billion scale. + +The same principle applies to **VerbTypes** (7.30+): the 127-verb taxonomy is intentionally coarse, and `subtype` is the per-product axis for relationships too. A `ReportsTo` relationship might carry `subtype: 'direct'` vs `'dotted-line'`; a `RelatedTo` edge might carry `'spouse'` / `'colleague'`. Verb-side rollup lives at `_system/verb-subtype-statistics.json` with identical shape to the noun-side rollup. Per-VerbType-per-subtype counts are O(1) via `brain.counts.byRelationshipSubtype()`. Brainy's design is fully symmetric — nouns and verbs are first-class peers with identical capability surfaces. + +Full guide: **[Subtypes & Facets](../guides/subtypes-and-facets.md)**. ### 2. Semantic not Structural diff --git a/docs/guides/subtypes-and-facets.md b/docs/guides/subtypes-and-facets.md index 0ba1fcde..c0dcb9b2 100644 --- a/docs/guides/subtypes-and-facets.md +++ b/docs/guides/subtypes-and-facets.md @@ -271,8 +271,199 @@ await brain.counts.byField('status', { type: NounType.Event }) await brain.migrateField({ from: 'metadata.kind', to: 'subtype' }) ``` +## Layer V — `subtype` on relationships (verbs) + +Relationships are first-class citizens too. Every verb (`VerbType`) gets the same `subtype` primitive: a `ReportsTo` relationship might carry `subtype: 'direct'` vs `'dotted-line'`; a `RelatedTo` edge might carry `'spouse'` / `'sibling'` / `'colleague'`. Same shape as the noun side: flat string, no hierarchy, top-level standard field, indexed on the fast path. + +### Write + +```typescript +const ceoId = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'Avery' }) +const vpId = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'Jordan' }) +const matrixId = await brain.add({ type: NounType.Person, subtype: 'contractor', data: 'Sam' }) + +await brain.relate({ + from: ceoId, + to: vpId, + type: VerbType.ReportsTo, + subtype: 'direct' +}) + +await brain.relate({ + from: ceoId, + to: matrixId, + type: VerbType.ReportsTo, + subtype: 'dotted-line' +}) +``` + +### Read & filter + +```typescript +// Direct reports — fast path filter (column-store hit, not metadata fallback) +const direct = await brain.getRelations({ + from: ceoId, + type: VerbType.ReportsTo, + subtype: 'direct' +}) + +// Set membership +const allReports = await brain.getRelations({ + from: ceoId, + type: VerbType.ReportsTo, + subtype: ['direct', 'dotted-line'] +}) +``` + +### Update (new — `updateRelation()`) + +Verbs previously had no update method — the only way to change a relationship was delete-then-recreate. 7.30 closes that gap: + +```typescript +// Promote a dotted-line report to direct without losing the edge id +await brain.updateRelation({ id: relationId, subtype: 'direct' }) + +// Or change weight/confidence +await brain.updateRelation({ id: relationId, weight: 0.5, confidence: 0.9 }) +``` + +### Traversal filter + +`find({ connected, subtype })` filters traversal edges by their subtype. Composes with `via` (verb-type filter): + +```typescript +// All direct reports two hops deep (will be supported in Cortex; for now depth-1) +const directChain = await brain.find({ + connected: { + from: ceoId, + via: VerbType.ReportsTo, + subtype: 'direct', + depth: 1 + } +}) +``` + +Multi-hop subtype filtering (`depth > 1`) lights up on the Cortex native path; the JS path throws today rather than return incorrect partial results. + +### Counts + +Same shape as the noun-side counts API: + +```typescript +brain.counts.byRelationshipSubtype(VerbType.ReportsTo) +// → { direct: 12, 'dotted-line': 3 } + +brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'direct') // O(1) point +// → 12 + +brain.counts.topRelationshipSubtypes(VerbType.ReportsTo, 3) +// → [['direct', 12], ['dotted-line', 3]] + +brain.relationshipSubtypesOf(VerbType.ReportsTo) +// → ['direct', 'dotted-line'] +``` + +These are O(1) lookups backed by the persisted `_system/verb-subtype-statistics.json` rollup — same self-heal machinery as the noun-side rollup. + +### Migrate verb fields + +`migrateField()` now walks verbs too: + +```typescript +// Migrate verb-side metadata.kind → top-level subtype +await brain.migrateField({ + from: 'metadata.kind', + to: 'subtype', + entityKind: 'verb' +}) + +// Or migrate both nouns and verbs in one pass +await brain.migrateField({ + from: 'metadata.kind', + to: 'subtype', + entityKind: 'both' +}) +``` + +Default is `entityKind: 'noun'` (backward-compatible with 7.29). + +## Enforcement — `requireSubtype()` + brain-wide strict mode + +By default (7.30) subtype is optional. Two complementary opt-in mechanisms let you enforce the pairing of type + subtype on every write: + +### Per-type registration + +Mark a specific `NounType` or `VerbType` as requiring a subtype, optionally with a fixed vocabulary: + +```typescript +brain.requireSubtype(NounType.Person, { + values: ['employee', 'customer', 'vendor'], + required: true +}) + +brain.requireSubtype(VerbType.ReportsTo, { + values: ['direct', 'dotted-line'], + required: true +}) + +// Now this throws — Person requires a subtype: +await brain.add({ type: NounType.Person, data: 'no subtype' }) + +// And this throws — 'matrix' isn't in the registered vocabulary: +await brain.relate({ + from: a, + to: b, + type: VerbType.ReportsTo, + subtype: 'matrix' +}) +``` + +### Brain-wide strict mode + +Enforce on every write across the whole brain: + +```typescript +const brain = new Brainy({ requireSubtype: true }) + +// Allow specific types to omit subtype (e.g. catch-all `Thing`) +const brain2 = new Brainy({ + requireSubtype: { except: [NounType.Thing, NounType.Custom] } +}) +``` + +When strict mode is on: + +- Every `add()` / `addMany()` / `update()` / `relate()` / `relateMany()` / `updateRelation()` rejects writes missing a subtype on a non-exempt type. +- `addMany()` and `relateMany()` validate every item BEFORE any storage write — atomic-fail semantics, no partial writes. +- Per-type rules registered via `requireSubtype()` compose with the brain-wide flag; specific rules win when both apply. +- Brainy's own internal writes (VFS root, VFS directories, VFS file entities) bypass enforcement via the `metadata.isVFSEntity: true` infrastructure marker. + +### Migrating to strict mode on an existing brain + +`brain.migrateField()` is your friend — populate `subtype` from an existing convention before flipping strict mode on: + +```typescript +// Step 1: backfill subtype from your existing metadata.kind convention +await brain.migrateField({ + from: 'metadata.kind', + to: 'subtype', + entityKind: 'both' +}) + +// Step 2: register the vocabulary +brain.requireSubtype(NounType.Person, { + values: ['employee', 'customer', 'vendor'], + required: true +}) + +// Step 3: future writes must have a subtype matching the vocabulary +await brain.add({ type: NounType.Person, subtype: 'employee', data: '...' }) +``` + ## Reference +### Layer 1 — `subtype` (nouns) + - `brain.add({ ..., subtype: 'value' })` — write the field - `brain.update({ id, subtype: 'value' })` — change the field - `brain.find({ type, subtype })` — filter (fast path) @@ -280,6 +471,29 @@ await brain.migrateField({ from: 'metadata.kind', to: 'subtype' }) - `brain.counts.bySubtype(type, subtype?)` — O(1) counts - `brain.counts.topSubtypes(type, n?)` — top N by count - `brain.subtypesOf(type)` — distinct subtype list + +### Layer V — `subtype` (verbs / relationships) + +- `brain.relate({ ..., subtype: 'value' })` — write the field +- `brain.updateRelation({ id, subtype, type?, weight?, ... })` — change a relationship +- `brain.getRelations({ subtype })` — filter (fast path) +- `brain.getRelations({ subtype: ['a', 'b'] })` — set membership +- `brain.find({ connected: { via, subtype, depth } })` — traversal filter +- `brain.counts.byRelationshipSubtype(verb, subtype?)` — O(1) counts +- `brain.counts.topRelationshipSubtypes(verb, n?)` — top N by count +- `brain.relationshipSubtypesOf(verb)` — distinct subtype list + +### Layer 2 — generic facets + - `brain.trackField(name, { perType?, values? })` — register a facet - `brain.counts.byField(name, { type? })` — facet counts -- `brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })` — rewrite a field + +### Layer 3 — migration + +- `brain.migrateField({ from, to, readBoth?, batchSize?, onProgress?, entityKind? })` — rewrite a field (nouns, verbs, or both) + +### Enforcement + +- `brain.requireSubtype(type, { values?, required })` — per-`NounType` / `VerbType` rule +- `new Brainy({ requireSubtype: true })` — brain-wide strict mode +- `new Brainy({ requireSubtype: { except: [type, ...] } })` — strict with exemptions diff --git a/src/brainy.ts b/src/brainy.ts index 2ecbc28a..dfb0cb99 100644 --- a/src/brainy.ts +++ b/src/brainy.ts @@ -51,6 +51,7 @@ import { validateAddParams, validateUpdateParams, validateRelateParams, + validateUpdateRelationParams, validateFindParams, recordQueryPerformance } from './utils/paramValidation.js' @@ -66,6 +67,7 @@ import { RemoveFromMetadataIndexOperation, RemoveFromGraphIndexOperation, UpdateNounMetadataOperation, + UpdateVerbMetadataOperation, DeleteNounMetadataOperation, DeleteVerbMetadataOperation } from './transaction/operations/index.js' @@ -85,6 +87,7 @@ import { AddParams, UpdateParams, RelateParams, + UpdateRelationParams, FindParams, SimilarParams, GetRelationsParams, @@ -198,6 +201,15 @@ export class Brainy implements BrainyInterface { */ private _trackedFields: Map }> = new Map() + /** + * Per-NounType / per-VerbType subtype enforcement rules registered via + * `brain.requireSubtype(type, options)`. When `required` is true the matching + * write path throws if subtype is missing; when `values` is set the matching + * write path throws if the value is off-vocabulary. Composes with the + * brain-wide `requireSubtype` constructor flag. + */ + private _requiredSubtypes: Map }> = new Map() + // State private initialized = false private dimensions?: number @@ -1033,6 +1045,12 @@ export class Brainy implements BrainyInterface { 'top-level' ) + // Subtype pairing enforcement (Layer 3 — 7.30.0). Per-type rules registered + // via brain.requireSubtype() compose with the brain-wide strict-mode flag. + // Metadata is passed so infrastructure writes (VFS) can bypass the + // missing-subtype check via the `isVFSEntity` marker. + this.enforceSubtypeOnAdd('add', params.type, params.subtype, params.metadata) + // Generate ID if not provided const id = params.id || uuidv4() @@ -1511,6 +1529,16 @@ export class Brainy implements BrainyInterface { ) } + // Subtype pairing enforcement on update (7.30.0). The effective NounType after + // the update is `params.type ?? existing.type`; we look it up if needed. + if (params.subtype !== undefined || params.type !== undefined) { + const existing = await this.get(params.id) + const effectiveType = params.type ?? existing?.type + const effectiveSubtype = params.subtype !== undefined ? params.subtype : existing?.subtype + const effectiveMetadata = params.metadata ?? existing?.metadata + this.enforceSubtypeOnAdd('update', effectiveType, effectiveSubtype, effectiveMetadata) + } + // Get existing entity with vectors (fix for regression) // We need includeVectors: true because: // 1. SaveNounOperation requires the vector @@ -1853,6 +1881,12 @@ export class Brainy implements BrainyInterface { // Zero-config validation (static import for performance) validateRelateParams(params) + // Subtype pairing enforcement (Layer 3 — 7.30.0). Per-type rules registered + // via brain.requireSubtype() compose with the brain-wide strict-mode flag. + // Metadata is passed so infrastructure edges (VFS containment) can bypass + // the missing-subtype check via the `isVFSEntity` / `isVFS` marker. + this.enforceSubtypeOnRelate('relate', params.type, params.subtype, params.metadata) + // Verify entities exist const fromEntity = await this.get(params.from) const toEntity = await this.get(params.to) @@ -1898,6 +1932,7 @@ export class Brainy implements BrainyInterface { const verbMetadata = { ...(params.metadata || {}), verb: params.type, + ...(params.subtype !== undefined && { subtype: params.subtype }), weight: params.weight ?? 1.0, createdAt: Date.now(), ...((params as any).data !== undefined && { data: (params as any).data }) @@ -1913,6 +1948,7 @@ export class Brainy implements BrainyInterface { target: params.to, verb: params.type, type: params.type, + ...(params.subtype !== undefined && { subtype: params.subtype }), weight: params.weight ?? 1.0, metadata: params.metadata, data: (params as any).data, @@ -2021,6 +2057,115 @@ export class Brainy implements BrainyInterface { }) } + /** + * Update an existing relationship. + * + * Mirror of `update()` for relationships. Supports changing the verb type, the + * sub-classification (`subtype`), weight/confidence, the opaque `data` payload, and + * structured metadata (merge by default; set `merge: false` to replace). + * + * If `type` changes, the relationship is re-indexed in the graph adjacency + * (`RemoveFromGraphIndex` + `AddToGraphIndex`) so traversal by verb type stays + * consistent. The relationship ID is preserved across the type change. + * + * @param params - Update parameters (`id` required; at least one field to change) + * @throws If the relationship doesn't exist + * + * @example Change subtype + * await brain.updateRelation({ id: relId, subtype: 'dotted-line' }) + * + * @example Merge metadata + * await brain.updateRelation({ id: relId, metadata: { startDate: '2026-Q3' } }) + * + * @example Replace metadata entirely + * await brain.updateRelation({ id: relId, metadata: { only: 'this' }, merge: false }) + */ + async updateRelation(params: UpdateRelationParams): Promise { + this.assertWritable('updateRelation') + await this.ensureInitialized() + + validateUpdateRelationParams(params) + + const existing = await this.storage.getVerb(params.id) + if (!existing) { + throw new Error(`Relation ${params.id} not found`) + } + + const existingAny = existing as any + const newVerbType = params.type ?? existingAny.verb ?? existingAny.type + + // Subtype pairing enforcement on update (7.30.0). The effective verb type after + // the update may have changed; we check against the new type and the resulting + // subtype value (explicit param or preserved existing). + if (params.subtype !== undefined || params.type !== undefined) { + const effectiveSubtype = params.subtype !== undefined + ? params.subtype + : (existingAny.subtype as string | undefined) + const effectiveMetadata = params.metadata ?? (existingAny.metadata as unknown) + this.enforceSubtypeOnRelate('updateRelation', newVerbType as VerbType, effectiveSubtype, effectiveMetadata) + } + const typeChanged = params.type !== undefined && params.type !== (existingAny.verb ?? existingAny.type) + + // Merge metadata (mirror of update() for nouns) + const newMetadata = params.merge !== false + ? { ...(existingAny.metadata || {}), ...(params.metadata || {}) } + : (params.metadata as any) || existingAny.metadata + + // Build updated stored metadata. System fields ALWAYS win — same shape as relate(). + const updatedMetadata = { + ...newMetadata, + verb: newVerbType, + ...(params.subtype !== undefined + ? { subtype: params.subtype } + : existingAny.subtype !== undefined && { subtype: existingAny.subtype }), + weight: params.weight ?? existingAny.weight ?? 1.0, + ...(params.confidence !== undefined + ? { confidence: params.confidence } + : existingAny.confidence !== undefined && { confidence: existingAny.confidence }), + createdAt: existingAny.createdAt, + updatedAt: Date.now(), + ...(params.data !== undefined + ? { data: params.data } + : existingAny.data !== undefined && { data: existingAny.data }) + } + + // Build the verb view used by the graph index — top-level fields mirror relate()'s. + const verbForIndex: GraphVerb = { + id: params.id, + vector: existingAny.vector, + sourceId: existingAny.sourceId, + targetId: existingAny.targetId, + source: existingAny.sourceId, + target: existingAny.targetId, + verb: newVerbType, + type: newVerbType, + ...(params.subtype !== undefined + ? { subtype: params.subtype } + : existingAny.subtype !== undefined && { subtype: existingAny.subtype }), + weight: updatedMetadata.weight, + metadata: newMetadata, + data: updatedMetadata.data, + createdAt: existingAny.createdAt + } + + await this.transactionManager.executeTransaction(async (tx) => { + tx.addOperation( + new UpdateVerbMetadataOperation(this.storage, params.id, updatedMetadata) + ) + + // If the verb type changed, re-index in graph adjacency so traversal-by-type + // stays consistent. The id is preserved across the swap. + if (typeChanged) { + tx.addOperation( + new RemoveFromGraphIndexOperation(this.graphIndex, existing as any) + ) + tx.addOperation( + new AddToGraphIndexOperation(this.graphIndex, verbForIndex) + ) + } + }) + } + /** * Get relationships between entities * @@ -2087,6 +2232,10 @@ export class Brainy implements BrainyInterface { filter.verbType = Array.isArray(params.type) ? params.type : [params.type] } + if (params.subtype !== undefined) { + filter.subtype = Array.isArray(params.subtype) ? params.subtype : [params.subtype] + } + if (params.service) { filter.service = params.service } @@ -2399,6 +2548,180 @@ export class Brainy implements BrainyInterface { return `__fieldCounts__${name}` } + /** + * Register subtype enforcement for a specific NounType or VerbType. + * + * Two complementary mechanisms shipped together in 7.30.0: + * + * 1. **Per-type enforcement** (this method): mark a specific type as requiring + * a subtype, optionally with a fixed vocabulary. Writes targeting that + * type without a matching subtype throw at the boundary. + * + * 2. **Brain-wide strict mode**: enable via `new Brainy({ requireSubtype: true })`. + * Every public write path checks the strict-mode rule. See + * `BrainyConfig.requireSubtype`. + * + * Both compose: a per-type registration always applies regardless of the + * brain-wide flag. + * + * @param type - The `NounType` or `VerbType` to register + * @param options.values - Optional vocabulary whitelist (rejects off-vocab values) + * @param options.required - When `true`, subtype is required on `add()`/`relate()`/`update()`/`updateRelation()` for this type (default: `true`) + * + * @example Lock down Person sub-classification + * brain.requireSubtype(NounType.Person, { + * values: ['employee', 'customer', 'vendor'], + * required: true + * }) + * + * @example Lock down management relationships + * brain.requireSubtype(VerbType.Manages, { + * values: ['direct', 'dotted-line'], + * required: true + * }) + */ + requireSubtype( + type: NounType | VerbType, + options: { values?: string[]; required?: boolean } = {} + ): void { + if (type === undefined || type === null) { + throw new Error('requireSubtype: type must be a valid NounType or VerbType') + } + const required = options.required !== false + const valuesSet = options.values && options.values.length > 0 + ? new Set(options.values) + : undefined + this._requiredSubtypes.set(String(type), { required, values: valuesSet }) + } + + /** + * Resolve the per-type subtype rule for a given NounType or VerbType. + * Returns `null` when no rule is registered for the type. Used by the + * write-path enforcement hooks (`enforceSubtypeOnAdd`, `enforceSubtypeOnRelate`). + */ + private getSubtypeRule(type: NounType | VerbType | undefined): { required: boolean; values?: Set } | null { + if (type === undefined || type === null) return null + return this._requiredSubtypes.get(String(type)) ?? null + } + + /** + * Check whether brain-wide strict mode is on for a given type. Brain-wide + * `requireSubtype: true` enforces on every type; the `{ except: [...] }` + * form allows the listed types through. Used by the write-path enforcement + * hooks for both nouns and verbs. + */ + private brainWideStrictRequiresSubtype(type: NounType | VerbType | undefined): boolean { + const flag = this.config.requireSubtype + if (!flag) return false + if (flag === true) return true + // { except: [...] } form — strict except for listed types + if (typeof flag === 'object' && Array.isArray((flag as any).except)) { + if (type === undefined || type === null) return true + return !(flag as any).except.includes(type) + } + return false + } + + /** + * Whether a write should bypass subtype enforcement because it represents + * internal Brainy infrastructure rather than user data. Currently triggers on: + * + * - `metadata.isVFSEntity === true` — Virtual File System root + directories + * + file entities. These are platform plumbing and follow their own + * subtype conventions (`'vfs-root'`, `'vfs-directory'`, `'vfs-file'`). + * - `metadata.isVFS === true` — same intent; older marker. + * + * If user code sets these flags, they opt out of enforcement and accept the + * responsibility. Documented as such on `AddParams.metadata` JSDoc. + */ + private isInfrastructureWrite(metadata: unknown): boolean { + if (!metadata || typeof metadata !== 'object') return false + const m = metadata as Record + return m.isVFSEntity === true || m.isVFS === true + } + + /** + * Enforce subtype rules for a noun write (`add` / `update`). + * + * Walks the per-type registration AND the brain-wide strict-mode flag, and + * throws with a descriptive message on missing or off-vocabulary values. + * Called from `add()` / `addMany()` / `update()` before the storage write. + * + * Skips infrastructure writes (see `isInfrastructureWrite`) so Brainy's own + * VFS root + directories + file entities don't get rejected when strict mode + * is on. Vocabulary rules still apply to user-supplied subtypes — the bypass + * only covers the missing-subtype case for internal plumbing. + * + * @param op - 'add' or 'update' (for error messages) + * @param type - The NounType being written + * @param subtype - The subtype value (or undefined) + * @param metadata - The metadata bag (checked for infrastructure markers) + */ + private enforceSubtypeOnAdd( + op: 'add' | 'update', + type: NounType | undefined, + subtype: string | undefined, + metadata?: unknown + ): void { + const rule = this.getSubtypeRule(type) + const strict = this.brainWideStrictRequiresSubtype(type) + const isInfra = this.isInfrastructureWrite(metadata) + + if (!rule && !strict) return + + if (subtype === undefined || subtype === null || subtype === '') { + if ((rule?.required || strict) && !isInfra) { + throw new Error( + `${op}(): NounType.${type} requires subtype but got ${subtype === undefined ? 'undefined' : 'empty'}. ` + + (rule?.values ? `Allowed values: [${Array.from(rule.values).join(', ')}].` : 'Register vocabulary via brain.requireSubtype().') + ) + } + return + } + + if (rule?.values && !rule.values.has(subtype)) { + throw new Error( + `${op}(): NounType.${type} subtype '${subtype}' is not in registered vocabulary ` + + `[${Array.from(rule.values).join(', ')}].` + ) + } + } + + /** + * Enforce subtype rules for a verb write (`relate` / `updateRelation`). + * Mirror of `enforceSubtypeOnAdd` for relationships. Skips infrastructure + * edges (VFS containment, etc.) — same `isVFSEntity` / `isVFS` markers. + */ + private enforceSubtypeOnRelate( + op: 'relate' | 'updateRelation', + verb: VerbType | undefined, + subtype: string | undefined, + metadata?: unknown + ): void { + const rule = this.getSubtypeRule(verb) + const strict = this.brainWideStrictRequiresSubtype(verb) + const isInfra = this.isInfrastructureWrite(metadata) + + if (!rule && !strict) return + + if (subtype === undefined || subtype === null || subtype === '') { + if ((rule?.required || strict) && !isInfra) { + throw new Error( + `${op}(): VerbType.${verb} requires subtype but got ${subtype === undefined ? 'undefined' : 'empty'}. ` + + (rule?.values ? `Allowed values: [${Array.from(rule.values).join(', ')}].` : 'Register vocabulary via brain.requireSubtype().') + ) + } + return + } + + if (rule?.values && !rule.values.has(subtype)) { + throw new Error( + `${op}(): VerbType.${verb} subtype '${subtype}' is not in registered vocabulary ` + + `[${Array.from(rule.values).join(', ')}].` + ) + } + } + /** * Validate a metadata bag (or top-level field assignment) against any registered * value whitelists. Called from `add()`/`update()` after the standard zero-config @@ -3115,6 +3438,19 @@ export class Brainy implements BrainyInterface { this.assertWritable('addMany') await this.ensureInitialized() + // Pre-validate every item against per-type + brain-wide subtype rules BEFORE + // any storage write — atomic-fail semantics: a missing-subtype anywhere in + // the batch fails the whole call, no partial writes. (7.30.0) + for (let i = 0; i < params.items.length; i++) { + const item = params.items[i] + try { + this.enforceSubtypeOnAdd('add', item.type, item.subtype, item.metadata) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + throw new Error(`addMany(): item[${i}] failed subtype enforcement: ${msg}`) + } + } + // Get optimal batch configuration from storage adapter // This automatically adapts to storage characteristics: // - GCS: 50 batch size, 100ms delay, sequential @@ -3465,6 +3801,19 @@ export class Brainy implements BrainyInterface { this.assertWritable('relateMany') await this.ensureInitialized() + // Pre-validate every item against per-type + brain-wide subtype rules BEFORE + // any storage write — atomic-fail semantics: a missing-subtype anywhere in + // the batch fails the whole call, no partial writes. (7.30.0) + for (let i = 0; i < params.items.length; i++) { + const item = params.items[i] + try { + this.enforceSubtypeOnRelate('relate', item.type, item.subtype, item.metadata) + } catch (err) { + const msg = err instanceof Error ? err.message : String(err) + throw new Error(`relateMany(): item[${i}] failed subtype enforcement: ${msg}`) + } + } + // Get optimal batch configuration from storage adapter // Automatically adapts to storage characteristics const storageConfig = this.storage.getBatchConfig() @@ -6291,6 +6640,63 @@ export class Brainy implements BrainyInterface { return Object.fromEntries(this.graphIndex.getAllRelationshipCounts()) }, + /** + * O(1) subtype counts for a given VerbType. Verb-side mirror of + * `bySubtype`. Returns the count for a single (verb, subtype) pair when + * `subtype` is passed; returns the full subtype → count map when omitted. + * Backed by the persisted `_system/verb-subtype-statistics.json` rollup. + * + * @param verb - The VerbType to count subtypes within + * @param subtype - Optional specific subtype string for O(1) point count + * + * @example + * brain.counts.byRelationshipSubtype(VerbType.Manages) + * // → { direct: 12, 'dotted-line': 3 } + * + * brain.counts.byRelationshipSubtype(VerbType.Manages, 'direct') + * // → 12 + */ + byRelationshipSubtype: (verb: VerbType, subtype?: string): number | Record => { + const verbSubtypeMap = typeof (this.storage as any).getVerbSubtypeCountsByType === 'function' + ? (this.storage as any).getVerbSubtypeCountsByType() as Map> + : null + if (!verbSubtypeMap) { + return subtype !== undefined ? 0 : {} + } + const verbIdx = TypeUtils.getVerbIndex(verb) + const inner = verbSubtypeMap.get(verbIdx) + if (!inner) { + return subtype !== undefined ? 0 : {} + } + if (subtype !== undefined) { + return inner.get(subtype) || 0 + } + const result: Record = {} + for (const [k, v] of inner.entries()) result[k] = v + return result + }, + + /** + * Top N subtypes for a VerbType, sorted by count (descending). Mirror of + * `topSubtypes` for verbs. + * + * @example + * brain.counts.topRelationshipSubtypes(VerbType.Manages, 5) + * // → [['direct', 12], ['dotted-line', 3]] + */ + topRelationshipSubtypes: (verb: VerbType, n: number = 10): Array<[string, number]> => { + const verbSubtypeMap = typeof (this.storage as any).getVerbSubtypeCountsByType === 'function' + ? (this.storage as any).getVerbSubtypeCountsByType() as Map> + : null + if (!verbSubtypeMap) return [] + const verbIdx = TypeUtils.getVerbIndex(verb) + const inner = verbSubtypeMap.get(verbIdx) + if (!inner) return [] + return Array.from(inner.entries()) + .sort((a, b) => b[1] - a[1]) + .slice(0, n) + }, + // O(1) count by field-value criteria byCriteria: async (field: string, value: any) => { return this.metadataIndex.getCountForCriteria(field, value) @@ -6437,6 +6843,30 @@ export class Brainy implements BrainyInterface { return Array.from(inner.keys()).sort() } + /** + * Distinct subtypes seen for a given VerbType. Mirror of `subtypesOf` for + * relationships. Reads from the verb subtype-statistics rollup — no scan, no + * storage round-trip. Vocabulary is observed (what's actually in the data), + * not registered. + * + * @param verb - The VerbType to enumerate subtypes for + * @returns Sorted list of distinct subtype strings (empty if none) + * + * @example + * const variants = brain.relationshipSubtypesOf(VerbType.Manages) + * // → ['direct', 'dotted-line'] + */ + relationshipSubtypesOf(verb: VerbType): string[] { + const verbSubtypeMap = typeof (this.storage as any).getVerbSubtypeCountsByType === 'function' + ? (this.storage as any).getVerbSubtypeCountsByType() as Map> + : null + if (!verbSubtypeMap) return [] + const verbIdx = TypeUtils.getVerbIndex(verb) + const inner = verbSubtypeMap.get(verbIdx) + if (!inner) return [] + return Array.from(inner.keys()).sort() + } + /** * Stream-and-rewrite a field across every entity in the brain. * @@ -6484,6 +6914,13 @@ export class Brainy implements BrainyInterface { readBoth?: boolean batchSize?: number onProgress?: (progress: { scanned: number; migrated: number }) => void + /** + * Which entity kind to walk. Defaults to `'noun'` for backward compat with + * 7.29.0. Use `'verb'` to migrate relationship fields, or `'both'` to walk + * nouns then verbs in one pass. Path forms (`'subtype'`, `'metadata.X'`, + * `'data.X'`) are identical on both sides. + */ + entityKind?: 'noun' | 'verb' | 'both' }): Promise<{ scanned: number migrated: number @@ -6507,58 +6944,177 @@ export class Brainy implements BrainyInterface { const toPath = this.parseMigrationPath(options.to) const batchSize = Math.max(1, options.batchSize ?? 100) const readBoth = options.readBoth === true + const entityKind = options.entityKind ?? 'noun' let scanned = 0 let migrated = 0 let skipped = 0 const errors: Array<{ id: string; error: string }> = [] - let offset = 0 - - // Stream via storage pagination — same shape used by streaming.entities() - // when no filter is set. Avoids a full in-memory load on large brains. - while (true) { - const page = await this.storage.getNouns({ pagination: { offset, limit: batchSize } }) - if (page.items.length === 0) break - - for (const noun of page.items) { - scanned++ - try { - const entity = await this.convertNounToEntity(noun as any) - const sourceValue = this.readPath(entity, fromPath) - if (sourceValue === undefined || sourceValue === null) { - skipped++ - continue - } - const targetValue = this.readPath(entity, toPath) - const sourceCleared = readBoth || fromPath.kind === toPath.kind && fromPath.field === toPath.field - if (targetValue === sourceValue && (readBoth || !this.pathExists(entity, fromPath))) { - // Already migrated and (readBoth || source already gone) → no-op - skipped++ - continue - } - - const update = this.buildMigrationUpdate(entity, fromPath, toPath, sourceValue, readBoth) - await this.update(update) - migrated++ - void sourceCleared - } catch (err) { - errors.push({ - id: (noun as any).id ?? '', - error: err instanceof Error ? err.message : String(err) - }) - } - } + const reportProgress = (): void => { if (options.onProgress) { options.onProgress({ scanned, migrated }) } - if (!page.hasMore) break - offset += page.items.length + } + + // Walk nouns when entityKind is 'noun' or 'both'. + if (entityKind === 'noun' || entityKind === 'both') { + let offset = 0 + while (true) { + const page = await this.storage.getNouns({ pagination: { offset, limit: batchSize } }) + if (page.items.length === 0) break + for (const noun of page.items) { + scanned++ + try { + const entity = await this.convertNounToEntity(noun as any) + const sourceValue = this.readPath(entity, fromPath) + if (sourceValue === undefined || sourceValue === null) { + skipped++ + continue + } + const targetValue = this.readPath(entity, toPath) + if (targetValue === sourceValue && (readBoth || !this.pathExists(entity, fromPath))) { + skipped++ + continue + } + const update = this.buildMigrationUpdate(entity, fromPath, toPath, sourceValue, readBoth) + await this.update(update) + migrated++ + } catch (err) { + errors.push({ + id: (noun as any).id ?? '', + error: err instanceof Error ? err.message : String(err) + }) + } + } + reportProgress() + if (!page.hasMore) break + offset += page.items.length + } + } + + // Walk verbs when entityKind is 'verb' or 'both'. Mirror of the noun loop — + // the path forms (`'subtype'`, `'metadata.X'`, `'data.X'`) and the + // readPath / buildMigrationUpdate helpers all work for verbs because + // `Relation` carries the same shape (top-level standard fields + + // metadata bag + optional data object). updateRelation() is the verb-side + // mutator. + if (entityKind === 'verb' || entityKind === 'both') { + let offset = 0 + while (true) { + const page = await this.storage.getVerbs({ pagination: { offset, limit: batchSize } }) + if (page.items.length === 0) break + for (const verb of page.items) { + scanned++ + try { + const edge = this.verbToRelationLike(verb as any) + const sourceValue = this.readPath(edge, fromPath) + if (sourceValue === undefined || sourceValue === null) { + skipped++ + continue + } + const targetValue = this.readPath(edge, toPath) + if (targetValue === sourceValue && (readBoth || !this.pathExists(edge, fromPath))) { + skipped++ + continue + } + const update = this.buildRelationMigrationUpdate(edge, fromPath, toPath, sourceValue, readBoth) + await this.updateRelation(update) + migrated++ + } catch (err) { + errors.push({ + id: (verb as any).id ?? '', + error: err instanceof Error ? err.message : String(err) + }) + } + } + reportProgress() + if (!page.hasMore) break + offset += page.items.length + } } return { scanned, migrated, skipped, errors } } + /** + * Project a storage verb shape onto the Entity-shaped surface that + * `readPath` / `pathExists` already understand. The migration helpers were + * written for nouns; making them work for verbs is just a shape projection — + * `subtype` / `metadata` / `data` live at the same paths on both sides. + */ + private verbToRelationLike(verb: any): Entity { + return { + id: verb.id, + vector: verb.vector, + type: (verb.verb ?? verb.type) as any, + subtype: verb.subtype, + data: verb.data, + metadata: verb.metadata as T, + service: verb.service, + createdAt: verb.createdAt, + updatedAt: verb.updatedAt, + createdBy: verb.createdBy, + confidence: verb.confidence, + weight: verb.weight + } + } + + /** + * Build an `UpdateRelationParams` payload that mirrors `buildMigrationUpdate` + * for verbs. Top-level path → top-level on update; metadata path → metadata + * bag with merge:false; data path → data bag. + */ + private buildRelationMigrationUpdate( + edge: Entity, + from: { kind: 'top' | 'metadata' | 'data'; field: string }, + to: { kind: 'top' | 'metadata' | 'data'; field: string }, + value: unknown, + readBoth: boolean + ): UpdateRelationParams { + const id = edge.id + const update: UpdateRelationParams = { id } + + if (to.kind === 'top') { + ;(update as any)[to.field] = value + } else if (to.kind === 'metadata') { + const base = (edge.metadata as unknown as Record) ?? {} + const nextMeta: Record = { ...base, [to.field]: value } + if (!readBoth && from.kind === 'metadata') { + delete nextMeta[from.field] + } + update.metadata = nextMeta as any + update.merge = false + } else { + const base = (edge.data && typeof edge.data === 'object') ? { ...(edge.data as Record) } : {} + base[to.field] = value + if (!readBoth && from.kind === 'data') { + delete base[from.field] + } + update.data = base + } + + if (!readBoth) { + if (from.kind === 'metadata' && to.kind !== 'metadata') { + const base = (edge.metadata as unknown as Record) ?? {} + const nextMeta: Record = { ...base } + delete nextMeta[from.field] + update.metadata = nextMeta as any + update.merge = false + } else if (from.kind === 'data' && to.kind !== 'data') { + const base = (edge.data && typeof edge.data === 'object') ? { ...(edge.data as Record) } : null + if (base) { + delete base[from.field] + update.data = base + } + } else if (from.kind === 'top' && from.field === 'subtype') { + ;(update as any).subtype = undefined + } + } + + return update + } + /** * Parse a dotted path used by `migrateField` into its routing kind + field name. * `'subtype'` → top-level standard; `'metadata.X'` → metadata; `'data.X'` → data; @@ -7609,6 +8165,18 @@ export class Brainy implements BrainyInterface { const { from, to, depth, direction = 'both' } = params.connected const via = params.connected.via ?? params.connected.type + const subtypeFilter = params.connected.subtype + + // Multi-hop subtype filtering would require per-hop edge enumeration in JS, which doesn't + // scale. The native fast path lands in the next Cortex release (see CTX-SUBTYPE-PARITY-V2). + // For 7.30.0 JS, restrict subtype filtering to depth-1 — explicit error beats silent + // incorrect results. + if (subtypeFilter !== undefined && depth !== undefined && depth > 1) { + throw new Error( + 'find({ connected: { subtype, depth > 1 } }) is not yet supported on the JS path. ' + + 'Use depth: 1, or wait for Cortex native traversal (CTX-SUBTYPE-PARITY-V2).' + ) + } // GraphConstraints speaks 'in' | 'out' | 'both'; neighbors() speaks 'incoming' | 'outgoing' | 'both'. const toNeighborDir = (d: 'in' | 'out' | 'both'): 'incoming' | 'outgoing' | 'both' => @@ -7631,6 +8199,30 @@ export class Brainy implements BrainyInterface { for (const id of await collect(to, toNeighborDir(reverse))) connectedIds.add(id) } + // Subtype filter (depth-1 only — see guard above). Intersect connectedIds with the set of + // {to} ids whose edge from the anchor carries the matching subtype. + if (subtypeFilter !== undefined) { + const subtypeArr = Array.isArray(subtypeFilter) ? subtypeFilter : [subtypeFilter] + const validIds = new Set() + const matchAnchor = async (anchor: string, reverseLookup: boolean): Promise => { + // Pull all edges from anchor that match via + subtype, then intersect. + const edges = await this.getRelations({ + ...(reverseLookup ? { to: anchor } : { from: anchor }), + ...(via && { type: via as VerbType | VerbType[] }), + subtype: subtypeArr, + limit: 10000 + }) + for (const e of edges) { + validIds.add(reverseLookup ? e.from : e.to) + } + } + if (from) await matchAnchor(from, false) + if (to) await matchAnchor(to, true) + for (const id of [...connectedIds]) { + if (!validIds.has(id)) connectedIds.delete(id) + } + } + // Filter existing results to only connected entities if (existingResults.length > 0) { return existingResults.filter(r => connectedIds.has(r.id)) @@ -7921,17 +8513,21 @@ export class Brainy implements BrainyInterface { * Convert verbs to relations (read from top-level) */ private verbsToRelations(verbs: GraphVerb[]): Relation[] { - return verbs.map((v) => ({ - id: v.id, - from: v.sourceId, - to: v.targetId, - type: (v.verb || v.type) as VerbType, - weight: v.weight ?? 1.0, - data: v.data, - metadata: v.metadata, - service: v.service as string, - createdAt: typeof v.createdAt === 'number' ? v.createdAt : Date.now() - })) + return verbs.map((v) => { + const va = v as any + return { + id: v.id, + from: v.sourceId, + to: v.targetId, + type: (v.verb || v.type) as VerbType, + ...(va.subtype !== undefined && { subtype: va.subtype as string }), + weight: v.weight ?? 1.0, + data: v.data, + metadata: v.metadata, + service: v.service as string, + createdAt: typeof v.createdAt === 'number' ? v.createdAt : Date.now() + } + }) } /** @@ -8463,6 +9059,12 @@ export class Brainy implements BrainyInterface { integrations: config?.integrations ?? undefined as any, // Migration — disabled by default, opt-in for automatic migration autoMigrate: config?.autoMigrate ?? false, + // Subtype pairing enforcement (7.30.0) — opt-in. + // false: only per-type rules registered via brain.requireSubtype() apply. + // true: every public write path requires subtype on every type. + // { except: [...] }: strict, but listed types may omit subtype. + // Becomes the default in 8.0.0. + requireSubtype: config?.requireSubtype ?? false, // Multi-process safety mode: config?.mode ?? 'writer', force: config?.force ?? false diff --git a/src/coreTypes.ts b/src/coreTypes.ts index a4203107..8f4b1532 100644 --- a/src/coreTypes.ts +++ b/src/coreTypes.ts @@ -302,6 +302,63 @@ export function resolveEntityField( return entity.metadata?.[field] } +/** + * Standard top-level fields on `HNSWVerbWithMetadata`. Parallel to + * `STANDARD_ENTITY_FIELDS` — the source of truth that drives `resolveVerbField`'s + * fast-path branch and the storage destructure-and-resurface pattern in + * `baseStorage.getVerb()`. Verb-specific entries: `verb` (the VerbType), + * `sourceId`, `targetId`. `subtype` is the 7.30 addition. + */ +export const STANDARD_VERB_FIELDS: ReadonlySet = new Set([ + 'id', + 'vector', + 'connections', + 'verb', + 'sourceId', + 'targetId', + 'subtype', + 'confidence', + 'weight', + 'createdAt', + 'updatedAt', + 'service', + 'createdBy', + 'data' +]) + +/** + * Resolve a field value off a verb by name. Mirror of `resolveEntityField` for + * `HNSWVerbWithMetadata`: standard fields at the top level, custom user fields + * in `verb.metadata`. + * + * Includes the same `type` / `verb` alias treatment as the noun helper + * (`resolveEntityField`'s `noun`/`type` alias) — relation shapes that carry `type` + * instead of `verb` (e.g. the public `Relation` API surface) resolve correctly + * for sorts/groupBys/filters that ask for `verb`. + * + * @param verb - The verb to read from + * @param field - The field name to resolve + * @returns The field value, or undefined if not present + * + * @example + * resolveVerbField(edge, 'createdAt') // reads edge.createdAt (top-level) + * resolveVerbField(edge, 'subtype') // reads edge.subtype (top-level — fast path) + * resolveVerbField(edge, 'customTag') // reads edge.metadata?.customTag + */ +export function resolveVerbField( + verb: HNSWVerbWithMetadata, + field: string +): unknown { + if (field === 'verb' || field === 'type') { + const v = verb as unknown as Record + return v.verb ?? v.type + } + if (STANDARD_VERB_FIELDS.has(field)) { + return (verb as unknown as Record)[field] + } + return verb.metadata?.[field] +} + /** * Combined verb structure for transport/API boundaries * @@ -322,6 +379,15 @@ export interface HNSWVerbWithMetadata { sourceId: string targetId: string + // SUBTYPE — optional per-product sub-classification within a VerbType (e.g. a + // `Manages` relationship might have subtype 'direct' vs 'dotted-line'; a `RelatedTo` + // edge might carry 'spouse' / 'sibling' / 'colleague'). Flat string (no hierarchy) — + // consumers decide the vocabulary. Indexed and rolled up into per-VerbType statistics + // so it's queryable (`getRelations({ verb, subtype })`) and aggregable + // (`groupBy:['subtype']`) on the standard-field fast path, never falling through to + // the metadata fallback. + subtype?: string + // QUALITY METRICS (top-level, explicit) weight?: number confidence?: number @@ -354,6 +420,7 @@ export interface GraphVerb { vector: Vector // Vector representation of the relationship connections?: Map> // Optional connections from HNSW index type?: string // Optional type of the relationship + subtype?: string // Optional sub-classification within the VerbType (7.30+) weight?: number // Optional weight of the relationship confidence?: number // Optional confidence score (0-1) metadata?: any // Optional metadata for the verb diff --git a/src/storage/baseStorage.ts b/src/storage/baseStorage.ts index 60846572..e78a3cbe 100644 --- a/src/storage/baseStorage.ts +++ b/src/storage/baseStorage.ts @@ -236,6 +236,16 @@ export abstract class BaseStorage extends BaseStorageAdapter { */ protected subtypeCountsByType = new Map>() + /** + * Per-VerbType-per-subtype counts. Verb-side mirror of `subtypeCountsByType`. + * Outer key is the VerbType index (matching `verbCountsByType` indexing); + * inner key is the subtype string. Populated incrementally as relationships + * are saved and decremented on delete; persisted to + * `_system/verb-subtype-statistics.json` (same shape as the noun-side rollup + * — `{ counts: { [verbTypeIdx]: { [subtype]: count } }, updatedAt }`). + */ + protected verbSubtypeCountsByType = new Map>() + /** * In-memory map from noun id → its `noun` (NounType) value, populated when * `saveNounMetadata_internal()` runs. `saveNoun_internal()` consults this @@ -258,6 +268,14 @@ export abstract class BaseStorage extends BaseStorageAdapter { * non-empty subtype get an entry. */ protected nounSubtypeByIdCache = new Map() + + /** + * In-memory map from verb id → `{ verb, subtype }` pair. Verb-side mirror of + * `nounTypeByIdCache` + `nounSubtypeByIdCache`. Lets `deleteVerbMetadata` and + * `updateRelation` decrement the right bucket without a re-read of metadata. + * Sparse — only ids with a non-empty subtype get an entry. + */ + protected verbSubtypeByIdCache = new Map() // Total: 676 bytes (99.2% reduction vs Map-based tracking) // Type caches REMOVED - ID-first paths eliminate need for type lookups! @@ -385,6 +403,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { // Load type statistics from storage (if they exist) await this.loadTypeStatistics() await this.loadSubtypeStatistics() + await this.loadVerbSubtypeStatistics() // GraphAdjacencyIndex is now SINGLETON via getGraphIndex() // - Removed direct creation here to fix dual-ownership bug @@ -1002,7 +1021,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { } // Combine into HNSWVerbWithMetadata - Extract standard fields to top-level - const { createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadata + const { subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadata return { id: verb.id, @@ -1012,6 +1031,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { sourceId: verb.sourceId, targetId: verb.targetId, // Standard fields at top-level + subtype: subtype as string | undefined, createdAt: (createdAt as number) || Date.now(), updatedAt: (updatedAt as number) || Date.now(), confidence: confidence as number | undefined, @@ -1076,7 +1096,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { const verb = this.deserializeVerb(vectorData) // Extract standard fields to top-level - const { createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadataData + const { subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadataData results.set(id, { id: verb.id, @@ -1086,6 +1106,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { sourceId: verb.sourceId, targetId: verb.targetId, // Standard fields at top-level + subtype: subtype as string | undefined, createdAt: (createdAt as number) || Date.now(), updatedAt: (updatedAt as number) || Date.now(), confidence: confidence as number | undefined, @@ -1548,6 +1569,13 @@ export abstract class BaseStorage extends BaseStorageAdapter { const filterTargetIds = filter?.targetId ? new Set(Array.isArray(filter.targetId) ? filter.targetId : [filter.targetId]) : null + const filterSubtypes = (filter as any)?.subtype + ? new Set( + Array.isArray((filter as any).subtype) + ? (filter as any).subtype + : [(filter as any).subtype] + ) + : null // Iterate by shards (0x00-0xFF) instead of types - single pass! for (let shard = 0; shard < 256 && collectedVerbs.length < targetCount; shard++) { @@ -1586,9 +1614,18 @@ export abstract class BaseStorage extends BaseStorageAdapter { // Load metadata const metadata = await this.getVerbMetadata(verb.id) + // Apply subtype filter (requires metadata — checked AFTER load) + if (filterSubtypes) { + const subtype = (metadata as any)?.subtype as string | undefined + if (!subtype || !filterSubtypes.has(subtype)) { + continue + } + } + // Combine verb + metadata collectedVerbs.push({ ...verb, + subtype: (metadata as any)?.subtype as string | undefined, weight: metadata?.weight, confidence: metadata?.confidence, createdAt: metadata?.createdAt @@ -1656,8 +1693,13 @@ export abstract class BaseStorage extends BaseStorageAdapter { const offset = pagination.offset || 0 const cursor = pagination.cursor - // Optimize for common filter cases to avoid loading all verbs - if (options?.filter) { + // Optimize for common filter cases to avoid loading all verbs. + // NOTE: subtype filter intentionally disqualifies the fast paths and forces + // fallthrough to the full shard scan (which loads metadata and applies the + // subtype check after the load). Subtype is not stored on the raw HNSWVerb; + // it's a metadata field. The graph-index fast paths return verbs without + // metadata, so they can't apply subtype filtering correctly. + if (options?.filter && !(options.filter as any).subtype) { // CRITICAL VFS FIX: If filtering by sourceId + verbType (most common VFS pattern!) // This is the query PathResolver.getChildren() uses: getRelations({ from: dirId, type: VerbType.Contains }) if ( @@ -2703,6 +2745,34 @@ export abstract class BaseStorage extends BaseStorageAdapter { // Cache verb type for faster lookups + // Track verb subtype changes: on type or subtype change via updateRelation(), + // decrement the prior bucket before incrementing the new one. Symmetric with + // the delete-path decrement in `deleteVerbMetadata()`. + const priorEntry = this.verbSubtypeByIdCache.get(id) + const priorVerbForSubtype = isNew ? undefined : (existingMetadata?.verb as VerbType | undefined) + const newSubtype = typeof metadata.subtype === 'string' && metadata.subtype.length > 0 + ? metadata.subtype as string + : undefined + + if (priorEntry && (priorEntry.subtype !== newSubtype || priorEntry.verb !== verbType)) { + this.decrementVerbSubtypeCount(priorEntry.verb, priorEntry.subtype) + } else if (!priorEntry && priorVerbForSubtype && !isNew) { + // Edge case: cache miss but metadata existed with a subtype (e.g. reader process startup). + const priorSubFromMeta = (existingMetadata as any)?.subtype as string | undefined + if (priorSubFromMeta && (priorSubFromMeta !== newSubtype || priorVerbForSubtype !== verbType)) { + this.decrementVerbSubtypeCount(priorVerbForSubtype, priorSubFromMeta) + } + } + if (newSubtype) { + if (!priorEntry || priorEntry.subtype !== newSubtype || priorEntry.verb !== verbType) { + this.incrementVerbSubtypeCount(verbType, newSubtype) + } + this.verbSubtypeByIdCache.set(id, { verb: verbType, subtype: newSubtype }) + } else if (priorEntry) { + // Subtype cleared by an update — drop the cache entry (decrement done above) + this.verbSubtypeByIdCache.delete(id) + } + // CRITICAL FIX: Increment verb count for new relationships // This runs AFTER metadata is saved // Uses synchronous increment since storage operations are already serialized @@ -2744,6 +2814,13 @@ export abstract class BaseStorage extends BaseStorageAdapter { // Direct O(1) delete with ID-first path const path = getVerbMetadataPath(id) await this.deleteObjectFromBranch(path) + + // Symmetric verb subtype decrement + const priorEntry = this.verbSubtypeByIdCache.get(id) + if (priorEntry) { + this.verbSubtypeByIdCache.delete(id) + this.decrementVerbSubtypeCount(priorEntry.verb, priorEntry.subtype) + } } // ============================================================================ @@ -2965,6 +3042,124 @@ export abstract class BaseStorage extends BaseStorageAdapter { prodLog.info(`[BaseStorage] Rebuilt subtype counts: ${totals} entities across ${this.subtypeCountsByType.size} NounTypes`) } + /** + * Increment the (verb, subtype) count, creating the inner map on first use. + * Verb-side mirror of `incrementSubtypeCount`. Indexed by VerbType index so it + * lines up with `verbCountsByType` and can be reduced into per-verb totals + * without re-keying. + */ + protected incrementVerbSubtypeCount(verb: VerbType, subtype: string): void { + const verbIdx = TypeUtils.getVerbIndex(verb) + if (verbIdx < 0) return + let inner = this.verbSubtypeCountsByType.get(verbIdx) + if (!inner) { + inner = new Map() + this.verbSubtypeCountsByType.set(verbIdx, inner) + } + inner.set(subtype, (inner.get(subtype) || 0) + 1) + } + + /** + * Decrement the (verb, subtype) count. Mirror of `decrementSubtypeCount`. + * Deletes the inner key when it reaches 0 and the outer entry when its inner + * map is empty, so the persisted shape stays compact across heavy churn. + */ + protected decrementVerbSubtypeCount(verb: VerbType, subtype: string): void { + const verbIdx = TypeUtils.getVerbIndex(verb) + if (verbIdx < 0) return + const inner = this.verbSubtypeCountsByType.get(verbIdx) + if (!inner) return + const next = (inner.get(subtype) || 0) - 1 + if (next <= 0) { + inner.delete(subtype) + if (inner.size === 0) this.verbSubtypeCountsByType.delete(verbIdx) + } else { + inner.set(subtype, next) + } + } + + /** + * Load `_system/verb-subtype-statistics.json` into `verbSubtypeCountsByType`. + * Persisted shape mirrors the noun-side rollup: + * `{ counts: { [verbIdx]: { [subtype]: count } }, updatedAt }`. Missing file + * or parse error → start from empty. + */ + protected async loadVerbSubtypeStatistics(): Promise { + try { + const stats = await this.readObjectFromPath(`${SYSTEM_DIR}/verb-subtype-statistics.json`) + if (stats && stats.counts && typeof stats.counts === 'object') { + this.verbSubtypeCountsByType.clear() + for (const [verbKey, subtypeMap] of Object.entries(stats.counts as Record>)) { + const verbIdx = Number(verbKey) + if (!Number.isInteger(verbIdx) || verbIdx < 0 || verbIdx >= VERB_TYPE_COUNT) continue + const inner = new Map() + for (const [subtype, count] of Object.entries(subtypeMap)) { + if (typeof count === 'number' && count > 0) inner.set(subtype, count) + } + if (inner.size > 0) this.verbSubtypeCountsByType.set(verbIdx, inner) + } + } + } catch { + // No existing verb subtype statistics, starting fresh. + } + } + + /** + * Save verb subtype statistics to storage. Mirrors `saveSubtypeStatistics`. + * Same persistence cadence (flushCounts + periodic saves alongside other + * type-statistics). + */ + protected async saveVerbSubtypeStatistics(): Promise { + const counts: Record> = {} + for (const [verbIdx, inner] of this.verbSubtypeCountsByType.entries()) { + const innerObj: Record = {} + for (const [subtype, count] of inner.entries()) innerObj[subtype] = count + counts[String(verbIdx)] = innerObj + } + await this.writeObjectToPath(`${SYSTEM_DIR}/verb-subtype-statistics.json`, { + counts, + updatedAt: Date.now() + }) + } + + /** + * Rebuild verb subtype counts from on-disk metadata. Companion to + * `rebuildSubtypeCounts()`. O(N) over all verbs. Used for poison recovery + * and explicit repair via `brainy inspect repair`. + */ + public async rebuildVerbSubtypeCounts(): Promise { + prodLog.info('[BaseStorage] Rebuilding verb subtype counts from storage...') + this.verbSubtypeCountsByType.clear() + this.verbSubtypeByIdCache.clear() + + for (let shard = 0; shard < 256; shard++) { + const shardHex = shard.toString(16).padStart(2, '0') + const shardDir = `entities/verbs/${shardHex}` + try { + const paths = await this.listObjectsInBranch(shardDir) + for (const path of paths) { + if (!path.includes('/metadata.json')) continue + try { + const metadata = await this.readWithInheritance(path) + if (metadata && metadata.verb && typeof metadata.subtype === 'string' && metadata.subtype.length > 0) { + const verb = metadata.verb as VerbType + const subtype = metadata.subtype as string + this.incrementVerbSubtypeCount(verb, subtype) + const segments = path.split('/') + const idSeg = segments[segments.length - 2] + if (idSeg) this.verbSubtypeByIdCache.set(idSeg, { verb, subtype }) + } + } catch { /* skip unreadable verbs */ } + } + } catch { /* skip missing shards */ } + } + + await this.saveVerbSubtypeStatistics() + const totals = Array.from(this.verbSubtypeCountsByType.values()) + .reduce((sum, inner) => sum + Array.from(inner.values()).reduce((s, n) => s + n, 0), 0) + prodLog.info(`[BaseStorage] Rebuilt verb subtype counts: ${totals} relationships across ${this.verbSubtypeCountsByType.size} VerbTypes`) + } + /** * Persist both counter systems atomically when an explicit flush is * requested. `super.flushCounts()` writes `entityCounts` (the Map in @@ -2982,6 +3177,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { await super.flushCounts() await this.saveTypeStatistics() await this.saveSubtypeStatistics() + await this.saveVerbSubtypeStatistics() } /** @@ -3004,6 +3200,15 @@ export abstract class BaseStorage extends BaseStorageAdapter { return this.subtypeCountsByType } + /** + * Get verb subtype counts (O(1) access). Verb-side mirror of + * `getSubtypeCountsByType`. Outer key: VerbType index. Inner map: subtype → count. + * Returned map is the live in-memory view — callers must treat it as read-only. + */ + public getVerbSubtypeCountsByType(): Map> { + return this.verbSubtypeCountsByType + } + /** * Get verb counts by type (O(1) access to type statistics) * Exposed for MetadataIndexManager to use as single source of truth @@ -3447,6 +3652,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { results.push({ ...verb, + subtype: (metadata as any).subtype as string | undefined, weight: metadata.weight, confidence: metadata.confidence, createdAt: metadata.createdAt @@ -3501,6 +3707,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { results.push({ ...verb, + subtype: (metadata as any)?.subtype as string | undefined, weight: metadata?.weight, confidence: metadata?.confidence, createdAt: metadata?.createdAt @@ -3675,6 +3882,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { if (verb && metadata) { results.push({ ...verb, + subtype: (metadata as any).subtype as string | undefined, weight: metadata.weight, confidence: metadata.confidence, createdAt: metadata.createdAt @@ -3722,6 +3930,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { results.push({ ...verb, + subtype: (metadata as any)?.subtype as string | undefined, weight: metadata?.weight, confidence: metadata?.confidence, createdAt: metadata?.createdAt @@ -3779,7 +3988,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { // Extract standard fields from metadata to top-level const metadataObj = (metadata || {}) as VerbMetadata - const { createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadataObj + const { subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadataObj const verbWithMetadata: HNSWVerbWithMetadata = { id: hnswVerb.id, @@ -3788,6 +3997,7 @@ export abstract class BaseStorage extends BaseStorageAdapter { verb: hnswVerb.verb, sourceId: hnswVerb.sourceId, targetId: hnswVerb.targetId, + subtype: subtype as string | undefined, createdAt: (createdAt as number) || Date.now(), updatedAt: (updatedAt as number) || Date.now(), confidence: confidence as number | undefined, diff --git a/src/types/brainy.types.ts b/src/types/brainy.types.ts index 5005f792..218116c5 100644 --- a/src/types/brainy.types.ts +++ b/src/types/brainy.types.ts @@ -68,6 +68,15 @@ export interface Relation { to: string /** Relationship type classification (VerbType enum) */ type: VerbType + /** + * Per-product sub-classification within the VerbType (e.g. a `Manages` + * relationship might have `subtype: 'direct'` vs `'dotted-line'`; a `RelatedTo` + * edge might carry `'spouse'` / `'sibling'` / `'colleague'`). Flat string, no + * hierarchy. Top-level standard field — indexed on the fast path and rolled into + * per-VerbType statistics so queries like `getRelations({ verb, subtype })` hit + * the column-store directly. + */ + subtype?: string /** Connection strength (0-1, default: 1.0) */ weight?: number /** Opaque content for the relationship (overrides auto-computed vector if provided) */ @@ -220,6 +229,14 @@ export interface RelateParams { to: string /** Relationship type classification (required) */ type: VerbType + /** + * Per-product sub-classification within the VerbType (e.g. a `Manages` + * relationship might have `subtype: 'direct'` or `'dotted-line'`; a `RelatedTo` + * edge might carry `'spouse'` or `'colleague'`). Flat string, no hierarchy. + * Indexed and rolled up into per-VerbType statistics for fast filtering + * (`getRelations({ verb, subtype })`) and aggregation (`groupBy:['subtype']`). + */ + subtype?: string /** Connection strength (0-1, default: 1.0) */ weight?: number /** Content for the relationship (optional — overrides auto-computed vector) */ @@ -241,7 +258,10 @@ export interface RelateParams { */ export interface UpdateRelationParams { id: string // Relation to update + type?: VerbType // Change verb type + subtype?: string // Change sub-classification (omit to preserve existing) weight?: number // New weight + confidence?: number // New confidence (0-1) data?: any // New content metadata?: Partial // Metadata to update merge?: boolean // Merge or replace metadata @@ -334,6 +354,13 @@ export interface GraphConstraints { from?: string // Connected from this entity via?: VerbType | VerbType[] // Via these relationship types type?: VerbType | VerbType[] // Alias for via + /** + * Filter traversal edges by VerbType subtype. Single string for equality, + * array for set membership. Composes with `via` — `{ via: 'manages', subtype: + * 'direct' }` traverses only direct-management edges, not dotted-line. Edges + * without a subtype value are excluded when this filter is set. + */ + subtype?: string | string[] depth?: number // Max traversal depth (default: 1) direction?: 'in' | 'out' | 'both' // Direction of traversal (default: 'both') bidirectional?: boolean // Consider both directions @@ -423,6 +450,15 @@ export interface GetRelationsParams { */ type?: VerbType | VerbType[] + /** + * Filter by VerbType subtype + * + * Top-level standard field — uses the fast path, not the metadata fallback. + * Pass a single string for equality (e.g. `subtype: 'direct'`), or an array + * for set membership (e.g. `subtype: ['direct', 'dotted-line']`). + */ + subtype?: string | string[] + /** * Maximum number of results to return * @@ -1099,6 +1135,26 @@ export interface BrainyConfig { // - true: Automatically run pending migrations during init() for small datasets (<10K entities) autoMigrate?: boolean + /** + * Brain-wide subtype enforcement mode. + * + * Opt-in in 7.30.0 (default: `false`); becomes the default in 8.0.0. + * + * - `false` / `undefined` (default): no brain-wide check. Per-type rules + * registered via `brain.requireSubtype(type, options)` still apply. + * - `true`: every `add()` / `addMany()` / `update()` / `relate()` / + * `relateMany()` / `updateRelation()` rejects writes where the entity's + * NounType (or relationship's VerbType) has no non-empty `subtype` value. + * - `{ except: [NounType.Thing, NounType.Custom] }`: same as `true`, but + * the listed types are allowed through without a subtype. Use for genuine + * catch-all types where no subtype makes sense. + * + * Per-type registrations always compose with the brain-wide flag — a type + * registered with `requireSubtype(type, { required: true })` is always + * enforced regardless of this flag. + */ + requireSubtype?: boolean | { except: Array } + /** * Process role for multi-process safety on filesystem storage. * diff --git a/src/utils/paramValidation.ts b/src/utils/paramValidation.ts index ffc2bd91..830922a6 100644 --- a/src/utils/paramValidation.ts +++ b/src/utils/paramValidation.ts @@ -5,7 +5,7 @@ * Only enforces universal truths, learns everything else */ -import { FindParams, AddParams, UpdateParams, RelateParams } from '../types/brainy.types.js' +import { FindParams, AddParams, UpdateParams, RelateParams, UpdateRelationParams } from '../types/brainy.types.js' import { NounType, VerbType } from '../types/graphTypes.js' // Dynamic import for Node.js os and fs modules @@ -407,7 +407,7 @@ export function validateRelateParams(params: RelateParams): void { if (!params.from) { throw new Error('from entity ID is required') } - + if (!params.to) { throw new Error('to entity ID is required') } @@ -421,7 +421,7 @@ export function validateRelateParams(params: RelateParams): void { } else if (!Object.values(VerbType).includes(params.type)) { throw new Error(`invalid VerbType: ${params.type}`) } - + // Universal truth: weight must be 0-1 if (params.weight !== undefined) { if (params.weight < 0 || params.weight > 1) { @@ -430,6 +430,40 @@ export function validateRelateParams(params: RelateParams): void { } } +/** + * Validate UpdateRelationParams. Mirror of validateUpdateParams for verbs — + * requires id + at least one field to change; bounds-checks weight/confidence; + * accepts type/subtype/weight/confidence/data/metadata changes. + */ +export function validateUpdateRelationParams(params: UpdateRelationParams): void { + if (!params.id) { + throw new Error('id is required for updateRelation') + } + + if ( + !params.data && + !params.metadata && + !params.type && + params.subtype === undefined && + params.weight === undefined && + params.confidence === undefined + ) { + throw new Error('updateRelation: must specify at least one field to update') + } + + if (params.type !== undefined && !Object.values(VerbType).includes(params.type)) { + throw new Error(`invalid VerbType: ${params.type}`) + } + + if (params.weight !== undefined && (params.weight < 0 || params.weight > 1)) { + throw new Error('weight must be between 0 and 1') + } + + if (params.confidence !== undefined && (params.confidence < 0 || params.confidence > 1)) { + throw new Error('confidence must be between 0 and 1') + } +} + /** * Get current validation configuration * Useful for debugging and monitoring diff --git a/src/vfs/VirtualFileSystem.ts b/src/vfs/VirtualFileSystem.ts index a7b220e2..cdf2d752 100644 --- a/src/vfs/VirtualFileSystem.ts +++ b/src/vfs/VirtualFileSystem.ts @@ -245,6 +245,7 @@ export class VirtualFileSystem implements IVirtualFileSystem { id: rootId, // Fixed ID - storage ensures uniqueness data: '/', type: NounType.Collection, + subtype: 'vfs-root', // Standard subtype for the VFS root collection (7.30+) metadata: this.getRootMetadata() }) @@ -511,6 +512,7 @@ export class VirtualFileSystem implements IVirtualFileSystem { const entity = await this.brain.add({ data: embeddingData, // Always provide string for embeddings type: this.getFileNounType(mimeType), + subtype: 'vfs-file', // Standard subtype for VFS file entities (7.30+) metadata }) @@ -519,6 +521,7 @@ export class VirtualFileSystem implements IVirtualFileSystem { from: parentId, to: entity, type: VerbType.Contains, + subtype: 'vfs-contains', // Standard subtype for VFS containment edges (7.30+) metadata: { isVFS: true } // Mark as VFS relationship }) @@ -890,6 +893,7 @@ export class VirtualFileSystem implements IVirtualFileSystem { const entity = await this.brain.add({ data: path, // Directory path as string content type: NounType.Collection, + subtype: 'vfs-directory', // Standard subtype for VFS directory entities (7.30+) metadata }) diff --git a/tests/integration/verb-subtype-and-enforcement.test.ts b/tests/integration/verb-subtype-and-enforcement.test.ts new file mode 100644 index 00000000..15b4c7cc --- /dev/null +++ b/tests/integration/verb-subtype-and-enforcement.test.ts @@ -0,0 +1,518 @@ +/** + * @module tests/integration/verb-subtype-and-enforcement + * @description Integration coverage for the 7.30.0 verb subtype mirror + + * opt-in enforcement primitives: + * + * - Layer V1: `subtype` on `relate()` / `Relation` / `RelateParams` round-trips + * through `getRelations({ subtype })`, the new `updateRelation()` method, and + * the persisted verb-subtype rollup. + * - Layer V2: `brain.counts.byRelationshipSubtype` / `topRelationshipSubtypes` + * / `brain.relationshipSubtypesOf` mirror the noun-side counts API. + * - Layer V3: `find({ connected: { via, subtype } })` filters traversal edges. + * - Layer V4: `migrateField({ entityKind: 'verb' \| 'both' })` rewrites verb + * fields. + * - Layer V5: `brain.requireSubtype(type, opts)` per-type rules + brain-wide + * strict mode (`new Brainy({ requireSubtype: true })`) reject every write + * path missing or off-vocabulary. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest' +import { Brainy } from '../../src/brainy' +import { NounType, VerbType } from '../../src/types/graphTypes' + +describe('Verb subtype + enforcement (7.30.0)', () => { + let brain: Brainy + + beforeEach(async () => { + brain = new Brainy({ storage: { type: 'memory' }, silent: true }) + await brain.init() + }) + + afterEach(async () => { + await brain.close() + }) + + describe('Layer V1: subtype on relationships', () => { + it('persists subtype on relate() and round-trips through getRelations()', async () => { + const fromId = await brain.add({ data: 'Avery', type: NounType.Person }) + const toId = await brain.add({ data: 'Jordan', type: NounType.Person }) + + const relId = await brain.relate({ + from: fromId, + to: toId, + type: VerbType.ReportsTo, + subtype: 'direct' + }) + + const all = await brain.getRelations({ from: fromId }) + const found = all.find(r => r.id === relId) + expect(found).toBeDefined() + expect(found!.subtype).toBe('direct') + expect(found!.type).toBe(VerbType.ReportsTo) + }) + + it('getRelations({ subtype }) filters by single value', async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + const c = await brain.add({ data: 'C', type: NounType.Person }) + + await brain.relate({ from: a, to: b, type: VerbType.ReportsTo, subtype: 'direct' }) + await brain.relate({ from: a, to: c, type: VerbType.ReportsTo, subtype: 'dotted-line' }) + + const direct = await brain.getRelations({ from: a, subtype: 'direct' }) + expect(direct).toHaveLength(1) + expect(direct[0].to).toBe(b) + expect(direct[0].subtype).toBe('direct') + }) + + it('getRelations({ subtype: [...] }) filters by set membership', async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + const c = await brain.add({ data: 'C', type: NounType.Person }) + const d = await brain.add({ data: 'D', type: NounType.Person }) + + await brain.relate({ from: a, to: b, type: VerbType.ReportsTo, subtype: 'direct' }) + await brain.relate({ from: a, to: c, type: VerbType.ReportsTo, subtype: 'dotted-line' }) + await brain.relate({ from: a, to: d, type: VerbType.ReportsTo, subtype: 'matrix' }) + + const both = await brain.getRelations({ + from: a, + type: VerbType.ReportsTo, + subtype: ['direct', 'dotted-line'] + }) + expect(both).toHaveLength(2) + expect(both.map(r => r.subtype).sort()).toEqual(['direct', 'dotted-line']) + }) + + it('updateRelation() changes the subtype in place', async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + const relId = await brain.relate({ + from: a, + to: b, + type: VerbType.ReportsTo, + subtype: 'direct' + }) + + await brain.updateRelation({ id: relId, subtype: 'dotted-line' }) + + const after = await brain.getRelations({ from: a, type: VerbType.ReportsTo }) + const found = after.find(r => r.id === relId) + expect(found!.subtype).toBe('dotted-line') + }) + + it('updateRelation() preserves subtype when omitted', async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + const relId = await brain.relate({ + from: a, + to: b, + type: VerbType.ReportsTo, + subtype: 'direct' + }) + + await brain.updateRelation({ id: relId, weight: 0.5 }) + + const after = await brain.getRelations({ from: a, type: VerbType.ReportsTo }) + const found = after.find(r => r.id === relId) + expect(found!.subtype).toBe('direct') + expect(found!.weight).toBe(0.5) + }) + + it('updateRelation() rejects empty updates', async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + const relId = await brain.relate({ from: a, to: b, type: VerbType.ReportsTo }) + + await expect(brain.updateRelation({ id: relId })).rejects.toThrow(/at least one field/) + }) + }) + + describe('Layer V2: verb counts API', () => { + beforeEach(async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + const c = await brain.add({ data: 'C', type: NounType.Person }) + const d = await brain.add({ data: 'D', type: NounType.Person }) + await brain.relate({ from: a, to: b, type: VerbType.ReportsTo, subtype: 'direct' }) + await brain.relate({ from: a, to: c, type: VerbType.ReportsTo, subtype: 'direct' }) + await brain.relate({ from: a, to: d, type: VerbType.ReportsTo, subtype: 'dotted-line' }) + await brain.relate({ from: b, to: c, type: VerbType.WorksWith, subtype: 'collaborator' }) + }) + + it('counts.byRelationshipSubtype(verb) returns the breakdown', async () => { + const manages = brain.counts.byRelationshipSubtype(VerbType.ReportsTo) + expect(manages).toEqual({ direct: 2, 'dotted-line': 1 }) + }) + + it('counts.byRelationshipSubtype(verb, subtype) returns the O(1) point count', async () => { + expect(brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'direct')).toBe(2) + expect(brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'dotted-line')).toBe(1) + expect(brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'never-existed')).toBe(0) + }) + + it('counts.topRelationshipSubtypes ranks by count', async () => { + const top = brain.counts.topRelationshipSubtypes(VerbType.ReportsTo, 10) + expect(top).toEqual([ + ['direct', 2], + ['dotted-line', 1] + ]) + }) + + it('brain.relationshipSubtypesOf returns sorted distinct subtypes', async () => { + expect(brain.relationshipSubtypesOf(VerbType.ReportsTo)).toEqual([ + 'direct', + 'dotted-line' + ]) + expect(brain.relationshipSubtypesOf(VerbType.WorksWith)).toEqual(['collaborator']) + }) + + it('rollup decrements on unrelate()', async () => { + const a = await brain.add({ data: 'X', type: NounType.Person }) + const b = await brain.add({ data: 'Y', type: NounType.Person }) + const relId = await brain.relate({ + from: a, + to: b, + type: VerbType.ReportsTo, + subtype: 'direct' + }) + expect(brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'direct')).toBe(3) + + await brain.unrelate(relId) + expect(brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'direct')).toBe(2) + }) + + it('rollup re-routes on updateRelation() subtype change', async () => { + const a = await brain.add({ data: 'X', type: NounType.Person }) + const b = await brain.add({ data: 'Y', type: NounType.Person }) + const relId = await brain.relate({ + from: a, + to: b, + type: VerbType.ReportsTo, + subtype: 'direct' + }) + expect(brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'direct')).toBe(3) + expect(brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'dotted-line')).toBe(1) + + await brain.updateRelation({ id: relId, subtype: 'dotted-line' }) + expect(brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'direct')).toBe(2) + expect(brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'dotted-line')).toBe(2) + }) + }) + + describe('Layer V3: find({ connected, subtype }) traversal filter', () => { + it('filters depth-1 traversal by edge subtype', async () => { + const ceo = await brain.add({ data: 'Avery', type: NounType.Person }) + const vp1 = await brain.add({ data: 'Jordan', type: NounType.Person }) + const vp2 = await brain.add({ data: 'Sam', type: NounType.Person }) + const matrix = await brain.add({ data: 'Lee', type: NounType.Person }) + + await brain.relate({ from: ceo, to: vp1, type: VerbType.ReportsTo, subtype: 'direct' }) + await brain.relate({ from: ceo, to: vp2, type: VerbType.ReportsTo, subtype: 'direct' }) + await brain.relate({ from: ceo, to: matrix, type: VerbType.ReportsTo, subtype: 'dotted-line' }) + + const directReports = await brain.find({ + connected: { + from: ceo, + via: VerbType.ReportsTo, + subtype: 'direct', + depth: 1 + } + }) + const ids = directReports.map(r => r.id).sort() + expect(ids).toEqual([vp1, vp2].sort()) + }) + + it('rejects depth > 1 with subtype filter (Cortex native path will land it)', async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + await expect( + brain.find({ + connected: { from: a, via: VerbType.ReportsTo, subtype: 'direct', depth: 2 } + }) + ).rejects.toThrow(/depth > 1/) + }) + }) + + describe('Layer V4: migrateField({ entityKind })', () => { + it("migrates verb metadata.kind → top-level subtype with entityKind: 'verb'", async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + + const relId = await brain.relate({ + from: a, + to: b, + type: VerbType.RelatedTo, + metadata: { kind: 'spouse' } + }) + + const result = await brain.migrateField({ + from: 'metadata.kind', + to: 'subtype', + entityKind: 'verb' + }) + expect(result.errors).toEqual([]) + expect(result.migrated).toBe(1) + + const after = await brain.getRelations({ from: a }) + const found = after.find(r => r.id === relId) + expect(found!.subtype).toBe('spouse') + expect((found!.metadata as any).kind).toBeUndefined() + }) + + it("migrates both noun and verb fields with entityKind: 'both'", async () => { + const a = await brain.add({ + data: 'A', + type: NounType.Person, + metadata: { kind: 'employee' } + }) + const b = await brain.add({ + data: 'B', + type: NounType.Person, + metadata: { kind: 'employee' } + }) + await brain.relate({ + from: a, + to: b, + type: VerbType.RelatedTo, + metadata: { kind: 'colleague' } + }) + + const result = await brain.migrateField({ + from: 'metadata.kind', + to: 'subtype', + entityKind: 'both' + }) + expect(result.errors).toEqual([]) + expect(result.migrated).toBe(3) // 2 nouns + 1 verb + + const ea = await brain.get(a) + expect(ea!.subtype).toBe('employee') + const eb = await brain.get(b) + expect(eb!.subtype).toBe('employee') + const rels = await brain.getRelations({ from: a }) + expect(rels[0].subtype).toBe('colleague') + }) + + it("readBoth: true on verb migration preserves the source field", async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + const relId = await brain.relate({ + from: a, + to: b, + type: VerbType.RelatedTo, + metadata: { kind: 'sibling' } + }) + + const result = await brain.migrateField({ + from: 'metadata.kind', + to: 'subtype', + entityKind: 'verb', + readBoth: true + }) + expect(result.migrated).toBe(1) + + const rels = await brain.getRelations({ from: a }) + const found = rels.find(r => r.id === relId) + expect(found!.subtype).toBe('sibling') + expect((found!.metadata as any).kind).toBe('sibling') + }) + }) + + describe('Layer V5: enforcement', () => { + describe('per-type registration (brain.requireSubtype)', () => { + it("requireSubtype(NounType, { required }) rejects add() without subtype", async () => { + brain.requireSubtype(NounType.Person, { required: true }) + + await expect( + brain.add({ data: 'no subtype', type: NounType.Person }) + ).rejects.toThrow(/requires subtype/) + }) + + it("requireSubtype with values rejects off-vocabulary on add()", async () => { + brain.requireSubtype(NounType.Person, { + values: ['employee', 'customer'], + required: true + }) + + await expect( + brain.add({ data: 'bad', type: NounType.Person, subtype: 'vendor' }) + ).rejects.toThrow(/not in registered vocabulary/) + }) + + it("requireSubtype with values accepts on-vocabulary", async () => { + brain.requireSubtype(NounType.Person, { + values: ['employee', 'customer'], + required: true + }) + + const id = await brain.add({ + data: 'ok', + type: NounType.Person, + subtype: 'employee' + }) + expect(id).toBeDefined() + }) + + it("requireSubtype(VerbType, { required }) rejects relate() without subtype", async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + brain.requireSubtype(VerbType.ReportsTo, { required: true }) + + await expect( + brain.relate({ from: a, to: b, type: VerbType.ReportsTo }) + ).rejects.toThrow(/requires subtype/) + }) + + it("requireSubtype(VerbType, values) rejects off-vocabulary on relate()", async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + brain.requireSubtype(VerbType.ReportsTo, { + values: ['direct', 'dotted-line'], + required: true + }) + + await expect( + brain.relate({ from: a, to: b, type: VerbType.ReportsTo, subtype: 'matrix' }) + ).rejects.toThrow(/not in registered vocabulary/) + }) + + it('enforcement applies to addMany() — atomic-fail, no partial writes', async () => { + brain.requireSubtype(NounType.Person, { required: true }) + + await expect( + brain.addMany({ + items: [ + { data: 'ok', type: NounType.Person, subtype: 'employee' }, + { data: 'missing', type: NounType.Person } + ] + }) + ).rejects.toThrow(/item\[1\] failed subtype enforcement/) + + // Verify no partial writes — first item should NOT be persisted + const persons = await brain.find({ type: NounType.Person, subtype: 'employee' }) + expect(persons).toHaveLength(0) + }) + + it('enforcement applies to relateMany() — atomic-fail', async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + const c = await brain.add({ data: 'C', type: NounType.Person }) + brain.requireSubtype(VerbType.ReportsTo, { required: true }) + + await expect( + brain.relateMany({ + items: [ + { from: a, to: b, type: VerbType.ReportsTo, subtype: 'direct' }, + { from: a, to: c, type: VerbType.ReportsTo } + ] + }) + ).rejects.toThrow(/item\[1\] failed subtype enforcement/) + }) + + it('enforcement applies to update() when changing subtype', async () => { + brain.requireSubtype(NounType.Person, { + values: ['employee', 'customer'], + required: true + }) + const id = await brain.add({ + data: 'ok', + type: NounType.Person, + subtype: 'employee' + }) + + await expect( + brain.update({ id, subtype: 'vendor' }) + ).rejects.toThrow(/not in registered vocabulary/) + }) + + it('enforcement applies to updateRelation() when changing subtype', async () => { + const a = await brain.add({ data: 'A', type: NounType.Person }) + const b = await brain.add({ data: 'B', type: NounType.Person }) + brain.requireSubtype(VerbType.ReportsTo, { + values: ['direct', 'dotted-line'], + required: true + }) + + const relId = await brain.relate({ + from: a, + to: b, + type: VerbType.ReportsTo, + subtype: 'direct' + }) + + await expect( + brain.updateRelation({ id: relId, subtype: 'matrix' }) + ).rejects.toThrow(/not in registered vocabulary/) + }) + }) + + describe('brain-wide strict mode', () => { + it('new Brainy({ requireSubtype: true }) rejects every add() without subtype', async () => { + await brain.close() + brain = new Brainy({ + storage: { type: 'memory' }, + silent: true, + requireSubtype: true + }) + await brain.init() + + await expect( + brain.add({ data: 'no subtype', type: NounType.Person }) + ).rejects.toThrow(/requires subtype/) + }) + + it('strict mode accepts writes with subtype', async () => { + await brain.close() + brain = new Brainy({ + storage: { type: 'memory' }, + silent: true, + requireSubtype: true + }) + await brain.init() + + const id = await brain.add({ + data: 'ok', + type: NounType.Person, + subtype: 'employee' + }) + expect(id).toBeDefined() + }) + + it('strict mode rejects relate() without subtype', async () => { + await brain.close() + brain = new Brainy({ + storage: { type: 'memory' }, + silent: true, + requireSubtype: true + }) + await brain.init() + + const a = await brain.add({ data: 'A', type: NounType.Person, subtype: 'employee' }) + const b = await brain.add({ data: 'B', type: NounType.Person, subtype: 'employee' }) + + await expect( + brain.relate({ from: a, to: b, type: VerbType.ReportsTo }) + ).rejects.toThrow(/requires subtype/) + }) + + it('except clause allows listed types through without subtype', async () => { + await brain.close() + brain = new Brainy({ + storage: { type: 'memory' }, + silent: true, + requireSubtype: { except: [NounType.Thing] } + }) + await brain.init() + + // Thing is exempt — should succeed without subtype + const thing = await brain.add({ data: 'a generic thing', type: NounType.Thing }) + expect(thing).toBeDefined() + + // Person is not exempt — should require subtype + await expect( + brain.add({ data: 'no subtype', type: NounType.Person }) + ).rejects.toThrow(/requires subtype/) + }) + }) + }) +})