soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

feat: verb subtype + updateRelation + requireSubtype enforcement

Browse source 
Brings verbs to first-class parity with nouns. The 7.29.0 subtype primitive
shipped for entities only; this release ships the symmetric verb mirror plus
the enforcement layer for ensuring every entity AND every relationship has
both type AND subtype.

Layer V1 — verb subtype mirror
- HNSWVerbWithMetadata.subtype + STANDARD_VERB_FIELDS set + resolveVerbField()
- Relation<T>.subtype, RelateParams<T>.subtype, UpdateRelationParams<T> extended,
  GetRelationsParams.subtype, GraphConstraints.subtype (for find connected)
- relate() persists subtype on verbMetadata + GraphVerb + transaction ops
- getRelations({ type, subtype }) fast-path filter with set membership
- find({ connected: { via, subtype, depth } }) traversal filter (depth-1 on
  the JS path; explicit error on depth > 1 pointing at Cortex native)
- verbsToRelations + storage destructure sites surface subtype to top-level
- All three graph-index fast-path queries (getVerbsBySource/ByTarget) enrich
  with subtype from metadata

Layer V2 — updateRelation() closes a pre-7.30 gap
- New first-class verb update method (parallel to update() for nouns)
- Changes subtype/type/weight/confidence/data/metadata in place
- Re-indexes in graph adjacency when verb type changes; id preserved
- validateUpdateRelationParams enforces id + at-least-one-field-to-update

Layer V3 — verb subtype storage rollup
- verbSubtypeCountsByType: Map<number, Map<string, number>> on BaseStorage
- verbSubtypeByIdCache for self-heal during update/delete
- incrementVerbSubtypeCount + decrementVerbSubtypeCount maintain state
- loadVerbSubtypeStatistics + saveVerbSubtypeStatistics persist to
  _system/verb-subtype-statistics.json (mirrors noun-side shape)
- rebuildVerbSubtypeCounts for poison recovery / explicit repair
- getVerbSubtypeCountsByType accessor for the public counts API
- Wired into init() / flushCounts() / saveVerbMetadata / deleteVerbMetadata

Layer V4 — verb counts API + relationshipSubtypesOf
- brain.counts.byRelationshipSubtype(verb, subtype?) — O(1) breakdown or point
- brain.counts.topRelationshipSubtypes(verb, n) — top N by count
- brain.relationshipSubtypesOf(verb) — sorted distinct subtypes

Layer V5 — migrateField extended to verbs
- New entityKind?: 'noun' | 'verb' | 'both' option (default 'noun')
- Mirror verb iteration via storage.getVerbs() with same path semantics
- verbToRelationLike + buildRelationMigrationUpdate helpers project the
  storage verb shape onto the Entity<T>-shaped surface readPath understands
- Routes through new updateRelation() for the verb-side rewrite

Enforcement (opt-in in 7.30, default in 8.0)
- brain.requireSubtype(type, options) — unified API for NounType OR VerbType.
  Registers per-type rules with optional values whitelist; composes with the
  brain-wide flag.
- new Brainy({ requireSubtype: true }) — brain-wide strict mode. Every public
  write path validates the pairing guarantee.
- { except: [NounType.Thing, ...] } form for catch-all type exemptions
- Atomic-fail semantics on addMany / relateMany — pre-validate every item
  before any storage write, throw on first failure with item index
- Per-type rules + brain-wide flag both throw with descriptive messages
- VFS infrastructure bypass via metadata.isVFSEntity / isVFS markers so
  brain's own VFS writes don't get rejected when strict mode is on

VFS labeling — concrete subtypes for infrastructure entities
- VFS root: NounType.Collection + subtype: 'vfs-root' (was bare Collection)
- VFS directories: subtype: 'vfs-directory'
- VFS files: subtype: 'vfs-file' (NounType still mime-based)
- VFS containment edges: VerbType.Contains + subtype: 'vfs-contains'
- Lets consumers cleanly enumerate VFS state via find({ subtype: 'vfs-file' })
  and distinguish Brainy's VFS Collections from user-created Collections

Docs
- docs/guides/subtypes-and-facets.md extended with Layer V (Verbs) section +
  Enforcement section. New full reference at the bottom split into Layer 1
  (nouns), Layer V (verbs), Layer 2 (facets), Layer 3 (migration), Enforcement.
- docs/api/README.md adds updateRelation(), getRelations({ subtype }), the
  three verb-side counts methods, requireSubtype(), and the brain-wide
  constructor option. relate() params include subtype.
- docs/DATA_MODEL.md adds a Subtype-for-VerbType section + STANDARD_VERB_FIELDS
- docs/architecture/finite-type-system.md extends Principle 1a to verbs
- docs/QUERY_OPERATORS.md adds a verb-subtype filter section covering
  getRelations and find({connected, subtype}) traversal
- README.md "Subtypes" section now shows both noun + verb in one example +
  the enforcement APIs
- RELEASES.md v7.30.0 entry with the full noun/verb capability parity matrix

Tests
- tests/integration/verb-subtype-and-enforcement.test.ts — 30 new tests
  covering V1 round-trips, V1 set membership, updateRelation in place,
  updateRelation preservation, V2 counts breakdown + point + topN + distinct,
  V2 decrements on unrelate, V2 re-routes on updateRelation, V3 depth-1
  traversal filter, V3 depth>1 explicit error, V4 verb migration, V4 both
  entity kinds, V4 readBoth preservation, V5 per-type required rejection,
  V5 vocabulary rejection, V5 on-vocab acceptance, V5 verb-side enforcement,
  V5 addMany atomic-fail, V5 relateMany atomic-fail, V5 update enforcement,
  V5 updateRelation enforcement, V5 brain-wide strict mode, V5 except clause.

Verification
- Unit suite: 1468/1468 passing
- Noun subtype integration (7.29 carryover): 26/26 passing
- Verb subtype + enforcement integration: 30/30 passing
- Type-check: clean
- Build: clean
- Public closed-source reference audit: clean

Internal 8.0 spec
- .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md (gitignored, not in npm artifact)
  documents the contract upgrade Cortex 3.0 implements against: required-by-
  default subtype, SubtypeRegistry typing hook, native simplification,
  multi-hop traversal native fast path, brain.fillSubtypes() migration helper.
  Coordinated via PLATFORM-HANDOFF rows CTX-SUBTYPE-PARITY-V2 (7.30 parallel
  work) and CTX-SUBTYPE-8.0-CONTRACT (8.0 spec).
This commit is contained in:
David Snelling 2026-06-05 11:15:52 -07:00
parent 8c68396e71
commit c0d326b36d
14 changed files with 2180 additions and 68 deletions
Download patch file Download diff file Expand all files Collapse all files

700
src/brainy.ts
View file

@ -51,6 +51,7 @@ import {
validateAddParams,
validateUpdateParams,
validateRelateParams,
validateUpdateRelationParams,
validateFindParams,
recordQueryPerformance
} from './utils/paramValidation.js'
@ -66,6 +67,7 @@ import {
RemoveFromMetadataIndexOperation,
RemoveFromGraphIndexOperation,
UpdateNounMetadataOperation,
UpdateVerbMetadataOperation,
DeleteNounMetadataOperation,
DeleteVerbMetadataOperation
} from './transaction/operations/index.js'
@ -85,6 +87,7 @@ import {
AddParams,
UpdateParams,
RelateParams,
UpdateRelationParams,
FindParams,
SimilarParams,
GetRelationsParams,
@ -198,6 +201,15 @@ export class Brainy<T = any> implements BrainyInterface<T> {
*/
private _trackedFields: Map<string, { perType: boolean; values?: Set<string> }> = new Map()
/**
* Per-NounType / per-VerbType subtype enforcement rules registered via
* `brain.requireSubtype(type, options)`. When `required` is true the matching
* write path throws if subtype is missing; when `values` is set the matching
* write path throws if the value is off-vocabulary. Composes with the
* brain-wide `requireSubtype` constructor flag.
*/
private _requiredSubtypes: Map<string, { required: boolean; values?: Set<string> }> = new Map()
// State
private initialized = false
private dimensions?: number
@ -1033,6 +1045,12 @@ export class Brainy<T = any> implements BrainyInterface<T> {
'top-level'
)
// Subtype pairing enforcement (Layer 3 — 7.30.0). Per-type rules registered
// via brain.requireSubtype() compose with the brain-wide strict-mode flag.
// Metadata is passed so infrastructure writes (VFS) can bypass the
// missing-subtype check via the `isVFSEntity` marker.
this.enforceSubtypeOnAdd('add', params.type, params.subtype, params.metadata)
// Generate ID if not provided
const id = params.id || uuidv4()
@ -1511,6 +1529,16 @@ export class Brainy<T = any> implements BrainyInterface<T> {
)
}
// Subtype pairing enforcement on update (7.30.0). The effective NounType after
// the update is `params.type ?? existing.type`; we look it up if needed.
if (params.subtype !== undefined || params.type !== undefined) {
const existing = await this.get(params.id)
const effectiveType = params.type ?? existing?.type
const effectiveSubtype = params.subtype !== undefined ? params.subtype : existing?.subtype
const effectiveMetadata = params.metadata ?? existing?.metadata
this.enforceSubtypeOnAdd('update', effectiveType, effectiveSubtype, effectiveMetadata)
}
// Get existing entity with vectors (fix for regression)
// We need includeVectors: true because:
// 1. SaveNounOperation requires the vector
@ -1853,6 +1881,12 @@ export class Brainy<T = any> implements BrainyInterface<T> {
// Zero-config validation (static import for performance)
validateRelateParams(params)
// Subtype pairing enforcement (Layer 3 — 7.30.0). Per-type rules registered
// via brain.requireSubtype() compose with the brain-wide strict-mode flag.
// Metadata is passed so infrastructure edges (VFS containment) can bypass
// the missing-subtype check via the `isVFSEntity` / `isVFS` marker.
this.enforceSubtypeOnRelate('relate', params.type, params.subtype, params.metadata)
// Verify entities exist
const fromEntity = await this.get(params.from)
const toEntity = await this.get(params.to)
@ -1898,6 +1932,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
const verbMetadata = {
...(params.metadata || {}),
verb: params.type,
...(params.subtype !== undefined && { subtype: params.subtype }),
weight: params.weight ?? 1.0,
createdAt: Date.now(),
...((params as any).data !== undefined && { data: (params as any).data })
@ -1913,6 +1948,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
target: params.to,
verb: params.type,
type: params.type,
...(params.subtype !== undefined && { subtype: params.subtype }),
weight: params.weight ?? 1.0,
metadata: params.metadata,
data: (params as any).data,
@ -2021,6 +2057,115 @@ export class Brainy<T = any> implements BrainyInterface<T> {
})
}
/**
* Update an existing relationship.
*
* Mirror of `update()` for relationships. Supports changing the verb type, the
* sub-classification (`subtype`), weight/confidence, the opaque `data` payload, and
* structured metadata (merge by default; set `merge: false` to replace).
*
* If `type` changes, the relationship is re-indexed in the graph adjacency
* (`RemoveFromGraphIndex` + `AddToGraphIndex`) so traversal by verb type stays
* consistent. The relationship ID is preserved across the type change.
*
* @param params - Update parameters (`id` required; at least one field to change)
* @throws If the relationship doesn't exist
*
* @example Change subtype
* await brain.updateRelation({ id: relId, subtype: 'dotted-line' })
*
* @example Merge metadata
* await brain.updateRelation({ id: relId, metadata: { startDate: '2026-Q3' } })
*
* @example Replace metadata entirely
* await brain.updateRelation({ id: relId, metadata: { only: 'this' }, merge: false })
*/
async updateRelation(params: UpdateRelationParams<T>): Promise<void> {
this.assertWritable('updateRelation')
await this.ensureInitialized()
validateUpdateRelationParams(params)
const existing = await this.storage.getVerb(params.id)
if (!existing) {
throw new Error(`Relation ${params.id} not found`)
}
const existingAny = existing as any
const newVerbType = params.type ?? existingAny.verb ?? existingAny.type
// Subtype pairing enforcement on update (7.30.0). The effective verb type after
// the update may have changed; we check against the new type and the resulting
// subtype value (explicit param or preserved existing).
if (params.subtype !== undefined || params.type !== undefined) {
const effectiveSubtype = params.subtype !== undefined
? params.subtype
: (existingAny.subtype as string | undefined)
const effectiveMetadata = params.metadata ?? (existingAny.metadata as unknown)
this.enforceSubtypeOnRelate('updateRelation', newVerbType as VerbType, effectiveSubtype, effectiveMetadata)
}
const typeChanged = params.type !== undefined && params.type !== (existingAny.verb ?? existingAny.type)
// Merge metadata (mirror of update() for nouns)
const newMetadata = params.merge !== false
? { ...(existingAny.metadata || {}), ...(params.metadata || {}) }
: (params.metadata as any) || existingAny.metadata
// Build updated stored metadata. System fields ALWAYS win — same shape as relate().
const updatedMetadata = {
...newMetadata,
verb: newVerbType,
...(params.subtype !== undefined
? { subtype: params.subtype }
: existingAny.subtype !== undefined && { subtype: existingAny.subtype }),
weight: params.weight ?? existingAny.weight ?? 1.0,
...(params.confidence !== undefined
? { confidence: params.confidence }
: existingAny.confidence !== undefined && { confidence: existingAny.confidence }),
createdAt: existingAny.createdAt,
updatedAt: Date.now(),
...(params.data !== undefined
? { data: params.data }
: existingAny.data !== undefined && { data: existingAny.data })
}
// Build the verb view used by the graph index — top-level fields mirror relate()'s.
const verbForIndex: GraphVerb = {
id: params.id,
vector: existingAny.vector,
sourceId: existingAny.sourceId,
targetId: existingAny.targetId,
source: existingAny.sourceId,
target: existingAny.targetId,
verb: newVerbType,
type: newVerbType,
...(params.subtype !== undefined
? { subtype: params.subtype }
: existingAny.subtype !== undefined && { subtype: existingAny.subtype }),
weight: updatedMetadata.weight,
metadata: newMetadata,
data: updatedMetadata.data,
createdAt: existingAny.createdAt
}
await this.transactionManager.executeTransaction(async (tx) => {
tx.addOperation(
new UpdateVerbMetadataOperation(this.storage, params.id, updatedMetadata)
)
// If the verb type changed, re-index in graph adjacency so traversal-by-type
// stays consistent. The id is preserved across the swap.
if (typeChanged) {
tx.addOperation(
new RemoveFromGraphIndexOperation(this.graphIndex, existing as any)
)
tx.addOperation(
new AddToGraphIndexOperation(this.graphIndex, verbForIndex)
)
}
})
}
/**
* Get relationships between entities
*
@ -2087,6 +2232,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
filter.verbType = Array.isArray(params.type) ? params.type : [params.type]
}
if (params.subtype !== undefined) {
filter.subtype = Array.isArray(params.subtype) ? params.subtype : [params.subtype]
}
if (params.service) {
filter.service = params.service
}
@ -2399,6 +2548,180 @@ export class Brainy<T = any> implements BrainyInterface<T> {
return `__fieldCounts__${name}`
}
/**
* Register subtype enforcement for a specific NounType or VerbType.
*
* Two complementary mechanisms shipped together in 7.30.0:
*
* 1. **Per-type enforcement** (this method): mark a specific type as requiring
* a subtype, optionally with a fixed vocabulary. Writes targeting that
* type without a matching subtype throw at the boundary.
*
* 2. **Brain-wide strict mode**: enable via `new Brainy({ requireSubtype: true })`.
* Every public write path checks the strict-mode rule. See
* `BrainyConfig.requireSubtype`.
*
* Both compose: a per-type registration always applies regardless of the
* brain-wide flag.
*
* @param type - The `NounType` or `VerbType` to register
* @param options.values - Optional vocabulary whitelist (rejects off-vocab values)
* @param options.required - When `true`, subtype is required on `add()`/`relate()`/`update()`/`updateRelation()` for this type (default: `true`)
*
* @example Lock down Person sub-classification
* brain.requireSubtype(NounType.Person, {
* values: ['employee', 'customer', 'vendor'],
* required: true
* })
*
* @example Lock down management relationships
* brain.requireSubtype(VerbType.Manages, {
* values: ['direct', 'dotted-line'],
* required: true
* })
*/
requireSubtype(
type: NounType | VerbType,
options: { values?: string[]; required?: boolean } = {}
): void {
if (type === undefined || type === null) {
throw new Error('requireSubtype: type must be a valid NounType or VerbType')
}
const required = options.required !== false
const valuesSet = options.values && options.values.length > 0
? new Set(options.values)
: undefined
this._requiredSubtypes.set(String(type), { required, values: valuesSet })
}
/**
* Resolve the per-type subtype rule for a given NounType or VerbType.
* Returns `null` when no rule is registered for the type. Used by the
* write-path enforcement hooks (`enforceSubtypeOnAdd`, `enforceSubtypeOnRelate`).
*/
private getSubtypeRule(type: NounType | VerbType | undefined): { required: boolean; values?: Set<string> } | null {
if (type === undefined || type === null) return null
return this._requiredSubtypes.get(String(type)) ?? null
}
/**
* Check whether brain-wide strict mode is on for a given type. Brain-wide
* `requireSubtype: true` enforces on every type; the `{ except: [...] }`
* form allows the listed types through. Used by the write-path enforcement
* hooks for both nouns and verbs.
*/
private brainWideStrictRequiresSubtype(type: NounType | VerbType | undefined): boolean {
const flag = this.config.requireSubtype
if (!flag) return false
if (flag === true) return true
// { except: [...] } form — strict except for listed types
if (typeof flag === 'object' && Array.isArray((flag as any).except)) {
if (type === undefined || type === null) return true
return !(flag as any).except.includes(type)
}
return false
}
/**
* Whether a write should bypass subtype enforcement because it represents
* internal Brainy infrastructure rather than user data. Currently triggers on:
*
* - `metadata.isVFSEntity === true` — Virtual File System root + directories
* + file entities. These are platform plumbing and follow their own
* subtype conventions (`'vfs-root'`, `'vfs-directory'`, `'vfs-file'`).
* - `metadata.isVFS === true` — same intent; older marker.
*
* If user code sets these flags, they opt out of enforcement and accept the
* responsibility. Documented as such on `AddParams.metadata` JSDoc.
*/
private isInfrastructureWrite(metadata: unknown): boolean {
if (!metadata || typeof metadata !== 'object') return false
const m = metadata as Record<string, unknown>
return m.isVFSEntity === true || m.isVFS === true
}
/**
* Enforce subtype rules for a noun write (`add` / `update`).
*
* Walks the per-type registration AND the brain-wide strict-mode flag, and
* throws with a descriptive message on missing or off-vocabulary values.
* Called from `add()` / `addMany()` / `update()` before the storage write.
*
* Skips infrastructure writes (see `isInfrastructureWrite`) so Brainy's own
* VFS root + directories + file entities don't get rejected when strict mode
* is on. Vocabulary rules still apply to user-supplied subtypes — the bypass
* only covers the missing-subtype case for internal plumbing.
*
* @param op - 'add' or 'update' (for error messages)
* @param type - The NounType being written
* @param subtype - The subtype value (or undefined)
* @param metadata - The metadata bag (checked for infrastructure markers)
*/
private enforceSubtypeOnAdd(
op: 'add' | 'update',
type: NounType | undefined,
subtype: string | undefined,
metadata?: unknown
): void {
const rule = this.getSubtypeRule(type)
const strict = this.brainWideStrictRequiresSubtype(type)
const isInfra = this.isInfrastructureWrite(metadata)
if (!rule && !strict) return
if (subtype === undefined || subtype === null || subtype === '') {
if ((rule?.required || strict) && !isInfra) {
throw new Error(
`${op}(): NounType.${type} requires subtype but got ${subtype === undefined ? 'undefined' : 'empty'}. ` +
(rule?.values ? `Allowed values: [${Array.from(rule.values).join(', ')}].` : 'Register vocabulary via brain.requireSubtype().')
)
}
return
}
if (rule?.values && !rule.values.has(subtype)) {
throw new Error(
`${op}(): NounType.${type} subtype '${subtype}' is not in registered vocabulary ` +
`[${Array.from(rule.values).join(', ')}].`
)
}
}
/**
* Enforce subtype rules for a verb write (`relate` / `updateRelation`).
* Mirror of `enforceSubtypeOnAdd` for relationships. Skips infrastructure
* edges (VFS containment, etc.) — same `isVFSEntity` / `isVFS` markers.
*/
private enforceSubtypeOnRelate(
op: 'relate' | 'updateRelation',
verb: VerbType | undefined,
subtype: string | undefined,
metadata?: unknown
): void {
const rule = this.getSubtypeRule(verb)
const strict = this.brainWideStrictRequiresSubtype(verb)
const isInfra = this.isInfrastructureWrite(metadata)
if (!rule && !strict) return
if (subtype === undefined || subtype === null || subtype === '') {
if ((rule?.required || strict) && !isInfra) {
throw new Error(
`${op}(): VerbType.${verb} requires subtype but got ${subtype === undefined ? 'undefined' : 'empty'}. ` +
(rule?.values ? `Allowed values: [${Array.from(rule.values).join(', ')}].` : 'Register vocabulary via brain.requireSubtype().')
)
}
return
}
if (rule?.values && !rule.values.has(subtype)) {
throw new Error(
`${op}(): VerbType.${verb} subtype '${subtype}' is not in registered vocabulary ` +
`[${Array.from(rule.values).join(', ')}].`
)
}
}
/**
* Validate a metadata bag (or top-level field assignment) against any registered
* value whitelists. Called from `add()`/`update()` after the standard zero-config
@ -3115,6 +3438,19 @@ export class Brainy<T = any> implements BrainyInterface<T> {
this.assertWritable('addMany')
await this.ensureInitialized()
// Pre-validate every item against per-type + brain-wide subtype rules BEFORE
// any storage write — atomic-fail semantics: a missing-subtype anywhere in
// the batch fails the whole call, no partial writes. (7.30.0)
for (let i = 0; i < params.items.length; i++) {
const item = params.items[i]
try {
this.enforceSubtypeOnAdd('add', item.type, item.subtype, item.metadata)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
throw new Error(`addMany(): item[${i}] failed subtype enforcement: ${msg}`)
}
}
// Get optimal batch configuration from storage adapter
// This automatically adapts to storage characteristics:
// - GCS: 50 batch size, 100ms delay, sequential
@ -3465,6 +3801,19 @@ export class Brainy<T = any> implements BrainyInterface<T> {
this.assertWritable('relateMany')
await this.ensureInitialized()
// Pre-validate every item against per-type + brain-wide subtype rules BEFORE
// any storage write — atomic-fail semantics: a missing-subtype anywhere in
// the batch fails the whole call, no partial writes. (7.30.0)
for (let i = 0; i < params.items.length; i++) {
const item = params.items[i]
try {
this.enforceSubtypeOnRelate('relate', item.type, item.subtype, item.metadata)
} catch (err) {
const msg = err instanceof Error ? err.message : String(err)
throw new Error(`relateMany(): item[${i}] failed subtype enforcement: ${msg}`)
}
}
// Get optimal batch configuration from storage adapter
// Automatically adapts to storage characteristics
const storageConfig = this.storage.getBatchConfig()
@ -6291,6 +6640,63 @@ export class Brainy<T = any> implements BrainyInterface<T> {
return Object.fromEntries(this.graphIndex.getAllRelationshipCounts())
},
/**
* O(1) subtype counts for a given VerbType. Verb-side mirror of
* `bySubtype`. Returns the count for a single (verb, subtype) pair when
* `subtype` is passed; returns the full subtype → count map when omitted.
* Backed by the persisted `_system/verb-subtype-statistics.json` rollup.
*
* @param verb - The VerbType to count subtypes within
* @param subtype - Optional specific subtype string for O(1) point count
*
* @example
* brain.counts.byRelationshipSubtype(VerbType.Manages)
* // → { direct: 12, 'dotted-line': 3 }
*
* brain.counts.byRelationshipSubtype(VerbType.Manages, 'direct')
* // → 12
*/
byRelationshipSubtype: (verb: VerbType, subtype?: string): number | Record<string, number> => {
const verbSubtypeMap = typeof (this.storage as any).getVerbSubtypeCountsByType === 'function'
? (this.storage as any).getVerbSubtypeCountsByType() as Map<number, Map<string, number>>
: null
if (!verbSubtypeMap) {
return subtype !== undefined ? 0 : {}
}
const verbIdx = TypeUtils.getVerbIndex(verb)
const inner = verbSubtypeMap.get(verbIdx)
if (!inner) {
return subtype !== undefined ? 0 : {}
}
if (subtype !== undefined) {
return inner.get(subtype) || 0
}
const result: Record<string, number> = {}
for (const [k, v] of inner.entries()) result[k] = v
return result
},
/**
* Top N subtypes for a VerbType, sorted by count (descending). Mirror of
* `topSubtypes` for verbs.
*
* @example
* brain.counts.topRelationshipSubtypes(VerbType.Manages, 5)
* // → [['direct', 12], ['dotted-line', 3]]
*/
topRelationshipSubtypes: (verb: VerbType, n: number = 10): Array<[string, number]> => {
const verbSubtypeMap = typeof (this.storage as any).getVerbSubtypeCountsByType === 'function'
? (this.storage as any).getVerbSubtypeCountsByType() as Map<number, Map<string, number>>
: null
if (!verbSubtypeMap) return []
const verbIdx = TypeUtils.getVerbIndex(verb)
const inner = verbSubtypeMap.get(verbIdx)
if (!inner) return []
return Array.from(inner.entries())
.sort((a, b) => b[1] - a[1])
.slice(0, n)
},
// O(1) count by field-value criteria
byCriteria: async (field: string, value: any) => {
return this.metadataIndex.getCountForCriteria(field, value)
@ -6437,6 +6843,30 @@ export class Brainy<T = any> implements BrainyInterface<T> {
return Array.from(inner.keys()).sort()
}
/**
* Distinct subtypes seen for a given VerbType. Mirror of `subtypesOf` for
* relationships. Reads from the verb subtype-statistics rollup — no scan, no
* storage round-trip. Vocabulary is observed (what's actually in the data),
* not registered.
*
* @param verb - The VerbType to enumerate subtypes for
* @returns Sorted list of distinct subtype strings (empty if none)
*
* @example
* const variants = brain.relationshipSubtypesOf(VerbType.Manages)
* // → ['direct', 'dotted-line']
*/
relationshipSubtypesOf(verb: VerbType): string[] {
const verbSubtypeMap = typeof (this.storage as any).getVerbSubtypeCountsByType === 'function'
? (this.storage as any).getVerbSubtypeCountsByType() as Map<number, Map<string, number>>
: null
if (!verbSubtypeMap) return []
const verbIdx = TypeUtils.getVerbIndex(verb)
const inner = verbSubtypeMap.get(verbIdx)
if (!inner) return []
return Array.from(inner.keys()).sort()
}
/**
* Stream-and-rewrite a field across every entity in the brain.
*
@ -6484,6 +6914,13 @@ export class Brainy<T = any> implements BrainyInterface<T> {
readBoth?: boolean
batchSize?: number
onProgress?: (progress: { scanned: number; migrated: number }) => void
/**
* Which entity kind to walk. Defaults to `'noun'` for backward compat with
* 7.29.0. Use `'verb'` to migrate relationship fields, or `'both'` to walk
* nouns then verbs in one pass. Path forms (`'subtype'`, `'metadata.X'`,
* `'data.X'`) are identical on both sides.
*/
entityKind?: 'noun' | 'verb' | 'both'
}): Promise<{
scanned: number
migrated: number
@ -6507,58 +6944,177 @@ export class Brainy<T = any> implements BrainyInterface<T> {
const toPath = this.parseMigrationPath(options.to)
const batchSize = Math.max(1, options.batchSize ?? 100)
const readBoth = options.readBoth === true
const entityKind = options.entityKind ?? 'noun'
let scanned = 0
let migrated = 0
let skipped = 0
const errors: Array<{ id: string; error: string }> = []
let offset = 0
// Stream via storage pagination — same shape used by streaming.entities()
// when no filter is set. Avoids a full in-memory load on large brains.
while (true) {
const page = await this.storage.getNouns({ pagination: { offset, limit: batchSize } })
if (page.items.length === 0) break
for (const noun of page.items) {
scanned++
try {
const entity = await this.convertNounToEntity(noun as any)
const sourceValue = this.readPath(entity, fromPath)
if (sourceValue === undefined || sourceValue === null) {
skipped++
continue
}
const targetValue = this.readPath(entity, toPath)
const sourceCleared = readBoth || fromPath.kind === toPath.kind && fromPath.field === toPath.field
if (targetValue === sourceValue && (readBoth || !this.pathExists(entity, fromPath))) {
// Already migrated and (readBoth || source already gone) → no-op
skipped++
continue
}
const update = this.buildMigrationUpdate(entity, fromPath, toPath, sourceValue, readBoth)
await this.update(update)
migrated++
void sourceCleared
} catch (err) {
errors.push({
id: (noun as any).id ?? '<unknown>',
error: err instanceof Error ? err.message : String(err)
})
}
}
const reportProgress = (): void => {
if (options.onProgress) {
options.onProgress({ scanned, migrated })
}
if (!page.hasMore) break
offset += page.items.length
}
// Walk nouns when entityKind is 'noun' or 'both'.
if (entityKind === 'noun' || entityKind === 'both') {
let offset = 0
while (true) {
const page = await this.storage.getNouns({ pagination: { offset, limit: batchSize } })
if (page.items.length === 0) break
for (const noun of page.items) {
scanned++
try {
const entity = await this.convertNounToEntity(noun as any)
const sourceValue = this.readPath(entity, fromPath)
if (sourceValue === undefined || sourceValue === null) {
skipped++
continue
}
const targetValue = this.readPath(entity, toPath)
if (targetValue === sourceValue && (readBoth || !this.pathExists(entity, fromPath))) {
skipped++
continue
}
const update = this.buildMigrationUpdate(entity, fromPath, toPath, sourceValue, readBoth)
await this.update(update)
migrated++
} catch (err) {
errors.push({
id: (noun as any).id ?? '<unknown>',
error: err instanceof Error ? err.message : String(err)
})
}
}
reportProgress()
if (!page.hasMore) break
offset += page.items.length
}
}
// Walk verbs when entityKind is 'verb' or 'both'. Mirror of the noun loop —
// the path forms (`'subtype'`, `'metadata.X'`, `'data.X'`) and the
// readPath / buildMigrationUpdate helpers all work for verbs because
// `Relation<T>` carries the same shape (top-level standard fields +
// metadata bag + optional data object). updateRelation() is the verb-side
// mutator.
if (entityKind === 'verb' || entityKind === 'both') {
let offset = 0
while (true) {
const page = await this.storage.getVerbs({ pagination: { offset, limit: batchSize } })
if (page.items.length === 0) break
for (const verb of page.items) {
scanned++
try {
const edge = this.verbToRelationLike(verb as any)
const sourceValue = this.readPath(edge, fromPath)
if (sourceValue === undefined || sourceValue === null) {
skipped++
continue
}
const targetValue = this.readPath(edge, toPath)
if (targetValue === sourceValue && (readBoth || !this.pathExists(edge, fromPath))) {
skipped++
continue
}
const update = this.buildRelationMigrationUpdate(edge, fromPath, toPath, sourceValue, readBoth)
await this.updateRelation(update)
migrated++
} catch (err) {
errors.push({
id: (verb as any).id ?? '<unknown>',
error: err instanceof Error ? err.message : String(err)
})
}
}
reportProgress()
if (!page.hasMore) break
offset += page.items.length
}
}
return { scanned, migrated, skipped, errors }
}
/**
* Project a storage verb shape onto the Entity<T>-shaped surface that
* `readPath` / `pathExists` already understand. The migration helpers were
* written for nouns; making them work for verbs is just a shape projection —
* `subtype` / `metadata` / `data` live at the same paths on both sides.
*/
private verbToRelationLike(verb: any): Entity<T> {
return {
id: verb.id,
vector: verb.vector,
type: (verb.verb ?? verb.type) as any,
subtype: verb.subtype,
data: verb.data,
metadata: verb.metadata as T,
service: verb.service,
createdAt: verb.createdAt,
updatedAt: verb.updatedAt,
createdBy: verb.createdBy,
confidence: verb.confidence,
weight: verb.weight
}
}
/**
* Build an `UpdateRelationParams` payload that mirrors `buildMigrationUpdate`
* for verbs. Top-level path → top-level on update; metadata path → metadata
* bag with merge:false; data path → data bag.
*/
private buildRelationMigrationUpdate(
edge: Entity<T>,
from: { kind: 'top' | 'metadata' | 'data'; field: string },
to: { kind: 'top' | 'metadata' | 'data'; field: string },
value: unknown,
readBoth: boolean
): UpdateRelationParams<T> {
const id = edge.id
const update: UpdateRelationParams<T> = { id }
if (to.kind === 'top') {
;(update as any)[to.field] = value
} else if (to.kind === 'metadata') {
const base = (edge.metadata as unknown as Record<string, unknown>) ?? {}
const nextMeta: Record<string, unknown> = { ...base, [to.field]: value }
if (!readBoth && from.kind === 'metadata') {
delete nextMeta[from.field]
}
update.metadata = nextMeta as any
update.merge = false
} else {
const base = (edge.data && typeof edge.data === 'object') ? { ...(edge.data as Record<string, unknown>) } : {}
base[to.field] = value
if (!readBoth && from.kind === 'data') {
delete base[from.field]
}
update.data = base
}
if (!readBoth) {
if (from.kind === 'metadata' && to.kind !== 'metadata') {
const base = (edge.metadata as unknown as Record<string, unknown>) ?? {}
const nextMeta: Record<string, unknown> = { ...base }
delete nextMeta[from.field]
update.metadata = nextMeta as any
update.merge = false
} else if (from.kind === 'data' && to.kind !== 'data') {
const base = (edge.data && typeof edge.data === 'object') ? { ...(edge.data as Record<string, unknown>) } : null
if (base) {
delete base[from.field]
update.data = base
}
} else if (from.kind === 'top' && from.field === 'subtype') {
;(update as any).subtype = undefined
}
}
return update
}
/**
* Parse a dotted path used by `migrateField` into its routing kind + field name.
* `'subtype'` → top-level standard; `'metadata.X'` → metadata; `'data.X'` → data;
@ -7609,6 +8165,18 @@ export class Brainy<T = any> implements BrainyInterface<T> {
const { from, to, depth, direction = 'both' } = params.connected
const via = params.connected.via ?? params.connected.type
const subtypeFilter = params.connected.subtype
// Multi-hop subtype filtering would require per-hop edge enumeration in JS, which doesn't
// scale. The native fast path lands in the next Cortex release (see CTX-SUBTYPE-PARITY-V2).
// For 7.30.0 JS, restrict subtype filtering to depth-1 — explicit error beats silent
// incorrect results.
if (subtypeFilter !== undefined && depth !== undefined && depth > 1) {
throw new Error(
'find({ connected: { subtype, depth > 1 } }) is not yet supported on the JS path. ' +
'Use depth: 1, or wait for Cortex native traversal (CTX-SUBTYPE-PARITY-V2).'
)
}
// GraphConstraints speaks 'in' | 'out' | 'both'; neighbors() speaks 'incoming' | 'outgoing' | 'both'.
const toNeighborDir = (d: 'in' | 'out' | 'both'): 'incoming' | 'outgoing' | 'both' =>
@ -7631,6 +8199,30 @@ export class Brainy<T = any> implements BrainyInterface<T> {
for (const id of await collect(to, toNeighborDir(reverse))) connectedIds.add(id)
}
// Subtype filter (depth-1 only — see guard above). Intersect connectedIds with the set of
// {to} ids whose edge from the anchor carries the matching subtype.
if (subtypeFilter !== undefined) {
const subtypeArr = Array.isArray(subtypeFilter) ? subtypeFilter : [subtypeFilter]
const validIds = new Set<string>()
const matchAnchor = async (anchor: string, reverseLookup: boolean): Promise<void> => {
// Pull all edges from anchor that match via + subtype, then intersect.
const edges = await this.getRelations({
...(reverseLookup ? { to: anchor } : { from: anchor }),
...(via && { type: via as VerbType | VerbType[] }),
subtype: subtypeArr,
limit: 10000
})
for (const e of edges) {
validIds.add(reverseLookup ? e.from : e.to)
}
}
if (from) await matchAnchor(from, false)
if (to) await matchAnchor(to, true)
for (const id of [...connectedIds]) {
if (!validIds.has(id)) connectedIds.delete(id)
}
}
// Filter existing results to only connected entities
if (existingResults.length > 0) {
return existingResults.filter(r => connectedIds.has(r.id))
@ -7921,17 +8513,21 @@ export class Brainy<T = any> implements BrainyInterface<T> {
* Convert verbs to relations (read from top-level)
*/
private verbsToRelations(verbs: GraphVerb[]): Relation<T>[] {
return verbs.map((v) => ({
id: v.id,
from: v.sourceId,
to: v.targetId,
type: (v.verb || v.type) as VerbType,
weight: v.weight ?? 1.0,
data: v.data,
metadata: v.metadata,
service: v.service as string,
createdAt: typeof v.createdAt === 'number' ? v.createdAt : Date.now()
}))
return verbs.map((v) => {
const va = v as any
return {
id: v.id,
from: v.sourceId,
to: v.targetId,
type: (v.verb || v.type) as VerbType,
...(va.subtype !== undefined && { subtype: va.subtype as string }),
weight: v.weight ?? 1.0,
data: v.data,
metadata: v.metadata,
service: v.service as string,
createdAt: typeof v.createdAt === 'number' ? v.createdAt : Date.now()
}
})
}
/**
@ -8463,6 +9059,12 @@ export class Brainy<T = any> implements BrainyInterface<T> {
integrations: config?.integrations ?? undefined as any,
// Migration — disabled by default, opt-in for automatic migration
autoMigrate: config?.autoMigrate ?? false,
// Subtype pairing enforcement (7.30.0) — opt-in.
// false: only per-type rules registered via brain.requireSubtype() apply.
// true: every public write path requires subtype on every type.
// { except: [...] }: strict, but listed types may omit subtype.
// Becomes the default in 8.0.0.
requireSubtype: config?.requireSubtype ?? false,
// Multi-process safety
mode: config?.mode ?? 'writer',
force: config?.force ?? false

67
src/coreTypes.ts
View file

@ -302,6 +302,63 @@ export function resolveEntityField(
return entity.metadata?.[field]
}
/**
* Standard top-level fields on `HNSWVerbWithMetadata`. Parallel to
* `STANDARD_ENTITY_FIELDS` — the source of truth that drives `resolveVerbField`'s
* fast-path branch and the storage destructure-and-resurface pattern in
* `baseStorage.getVerb()`. Verb-specific entries: `verb` (the VerbType),
* `sourceId`, `targetId`. `subtype` is the 7.30 addition.
*/
export const STANDARD_VERB_FIELDS: ReadonlySet<string> = new Set([
'id',
'vector',
'connections',
'verb',
'sourceId',
'targetId',
'subtype',
'confidence',
'weight',
'createdAt',
'updatedAt',
'service',
'createdBy',
'data'
])
/**
* Resolve a field value off a verb by name. Mirror of `resolveEntityField` for
* `HNSWVerbWithMetadata`: standard fields at the top level, custom user fields
* in `verb.metadata`.
*
* Includes the same `type` / `verb` alias treatment as the noun helper
* (`resolveEntityField`'s `noun`/`type` alias) — relation shapes that carry `type`
* instead of `verb` (e.g. the public `Relation<T>` API surface) resolve correctly
* for sorts/groupBys/filters that ask for `verb`.
*
* @param verb - The verb to read from
* @param field - The field name to resolve
* @returns The field value, or undefined if not present
*
* @example
* resolveVerbField(edge, 'createdAt') // reads edge.createdAt (top-level)
* resolveVerbField(edge, 'subtype') // reads edge.subtype (top-level — fast path)
* resolveVerbField(edge, 'customTag') // reads edge.metadata?.customTag
*/
export function resolveVerbField(
verb: HNSWVerbWithMetadata,
field: string
): unknown {
if (field === 'verb' || field === 'type') {
const v = verb as unknown as Record<string, unknown>
return v.verb ?? v.type
}
if (STANDARD_VERB_FIELDS.has(field)) {
return (verb as unknown as Record<string, unknown>)[field]
}
return verb.metadata?.[field]
}
/**
* Combined verb structure for transport/API boundaries
*
@ -322,6 +379,15 @@ export interface HNSWVerbWithMetadata {
sourceId: string
targetId: string
// SUBTYPE — optional per-product sub-classification within a VerbType (e.g. a
// `Manages` relationship might have subtype 'direct' vs 'dotted-line'; a `RelatedTo`
// edge might carry 'spouse' / 'sibling' / 'colleague'). Flat string (no hierarchy) —
// consumers decide the vocabulary. Indexed and rolled up into per-VerbType statistics
// so it's queryable (`getRelations({ verb, subtype })`) and aggregable
// (`groupBy:['subtype']`) on the standard-field fast path, never falling through to
// the metadata fallback.
subtype?: string
// QUALITY METRICS (top-level, explicit)
weight?: number
confidence?: number
@ -354,6 +420,7 @@ export interface GraphVerb {
vector: Vector // Vector representation of the relationship
connections?: Map<number, Set<string>> // Optional connections from HNSW index
type?: string // Optional type of the relationship
subtype?: string // Optional sub-classification within the VerbType (7.30+)
weight?: number // Optional weight of the relationship
confidence?: number // Optional confidence score (0-1)
metadata?: any // Optional metadata for the verb

220
src/storage/baseStorage.ts
View file

@ -236,6 +236,16 @@ export abstract class BaseStorage extends BaseStorageAdapter {
*/
protected subtypeCountsByType = new Map<number, Map<string, number>>()
/**
* Per-VerbType-per-subtype counts. Verb-side mirror of `subtypeCountsByType`.
* Outer key is the VerbType index (matching `verbCountsByType` indexing);
* inner key is the subtype string. Populated incrementally as relationships
* are saved and decremented on delete; persisted to
* `_system/verb-subtype-statistics.json` (same shape as the noun-side rollup
* — `{ counts: { [verbTypeIdx]: { [subtype]: count } }, updatedAt }`).
*/
protected verbSubtypeCountsByType = new Map<number, Map<string, number>>()
/**
* In-memory map from noun id → its `noun` (NounType) value, populated when
* `saveNounMetadata_internal()` runs. `saveNoun_internal()` consults this
@ -258,6 +268,14 @@ export abstract class BaseStorage extends BaseStorageAdapter {
* non-empty subtype get an entry.
*/
protected nounSubtypeByIdCache = new Map<string, string>()
/**
* In-memory map from verb id → `{ verb, subtype }` pair. Verb-side mirror of
* `nounTypeByIdCache` + `nounSubtypeByIdCache`. Lets `deleteVerbMetadata` and
* `updateRelation` decrement the right bucket without a re-read of metadata.
* Sparse — only ids with a non-empty subtype get an entry.
*/
protected verbSubtypeByIdCache = new Map<string, { verb: VerbType; subtype: string }>()
// Total: 676 bytes (99.2% reduction vs Map-based tracking)
// Type caches REMOVED - ID-first paths eliminate need for type lookups!
@ -385,6 +403,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// Load type statistics from storage (if they exist)
await this.loadTypeStatistics()
await this.loadSubtypeStatistics()
await this.loadVerbSubtypeStatistics()
// GraphAdjacencyIndex is now SINGLETON via getGraphIndex()
// - Removed direct creation here to fix dual-ownership bug
@ -1002,7 +1021,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
}
// Combine into HNSWVerbWithMetadata - Extract standard fields to top-level
const { createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadata
const { subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadata
return {
id: verb.id,
@ -1012,6 +1031,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
sourceId: verb.sourceId,
targetId: verb.targetId,
// Standard fields at top-level
subtype: subtype as string | undefined,
createdAt: (createdAt as number) || Date.now(),
updatedAt: (updatedAt as number) || Date.now(),
confidence: confidence as number | undefined,
@ -1076,7 +1096,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
const verb = this.deserializeVerb(vectorData)
// Extract standard fields to top-level
const { createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadataData
const { subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadataData
results.set(id, {
id: verb.id,
@ -1086,6 +1106,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
sourceId: verb.sourceId,
targetId: verb.targetId,
// Standard fields at top-level
subtype: subtype as string | undefined,
createdAt: (createdAt as number) || Date.now(),
updatedAt: (updatedAt as number) || Date.now(),
confidence: confidence as number | undefined,
@ -1548,6 +1569,13 @@ export abstract class BaseStorage extends BaseStorageAdapter {
const filterTargetIds = filter?.targetId
? new Set(Array.isArray(filter.targetId) ? filter.targetId : [filter.targetId])
: null
const filterSubtypes = (filter as any)?.subtype
? new Set(
Array.isArray((filter as any).subtype)
? (filter as any).subtype
: [(filter as any).subtype]
)
: null
// Iterate by shards (0x00-0xFF) instead of types - single pass!
for (let shard = 0; shard < 256 && collectedVerbs.length < targetCount; shard++) {
@ -1586,9 +1614,18 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// Load metadata
const metadata = await this.getVerbMetadata(verb.id)
// Apply subtype filter (requires metadata — checked AFTER load)
if (filterSubtypes) {
const subtype = (metadata as any)?.subtype as string | undefined
if (!subtype || !filterSubtypes.has(subtype)) {
continue
}
}
// Combine verb + metadata
collectedVerbs.push({
...verb,
subtype: (metadata as any)?.subtype as string | undefined,
weight: metadata?.weight,
confidence: metadata?.confidence,
createdAt: metadata?.createdAt
@ -1656,8 +1693,13 @@ export abstract class BaseStorage extends BaseStorageAdapter {
const offset = pagination.offset || 0
const cursor = pagination.cursor
// Optimize for common filter cases to avoid loading all verbs
if (options?.filter) {
// Optimize for common filter cases to avoid loading all verbs.
// NOTE: subtype filter intentionally disqualifies the fast paths and forces
// fallthrough to the full shard scan (which loads metadata and applies the
// subtype check after the load). Subtype is not stored on the raw HNSWVerb;
// it's a metadata field. The graph-index fast paths return verbs without
// metadata, so they can't apply subtype filtering correctly.
if (options?.filter && !(options.filter as any).subtype) {
// CRITICAL VFS FIX: If filtering by sourceId + verbType (most common VFS pattern!)
// This is the query PathResolver.getChildren() uses: getRelations({ from: dirId, type: VerbType.Contains })
if (
@ -2703,6 +2745,34 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// Cache verb type for faster lookups
// Track verb subtype changes: on type or subtype change via updateRelation(),
// decrement the prior bucket before incrementing the new one. Symmetric with
// the delete-path decrement in `deleteVerbMetadata()`.
const priorEntry = this.verbSubtypeByIdCache.get(id)
const priorVerbForSubtype = isNew ? undefined : (existingMetadata?.verb as VerbType | undefined)
const newSubtype = typeof metadata.subtype === 'string' && metadata.subtype.length > 0
? metadata.subtype as string
: undefined
if (priorEntry && (priorEntry.subtype !== newSubtype || priorEntry.verb !== verbType)) {
this.decrementVerbSubtypeCount(priorEntry.verb, priorEntry.subtype)
} else if (!priorEntry && priorVerbForSubtype && !isNew) {
// Edge case: cache miss but metadata existed with a subtype (e.g. reader process startup).
const priorSubFromMeta = (existingMetadata as any)?.subtype as string | undefined
if (priorSubFromMeta && (priorSubFromMeta !== newSubtype || priorVerbForSubtype !== verbType)) {
this.decrementVerbSubtypeCount(priorVerbForSubtype, priorSubFromMeta)
}
}
if (newSubtype) {
if (!priorEntry || priorEntry.subtype !== newSubtype || priorEntry.verb !== verbType) {
this.incrementVerbSubtypeCount(verbType, newSubtype)
}
this.verbSubtypeByIdCache.set(id, { verb: verbType, subtype: newSubtype })
} else if (priorEntry) {
// Subtype cleared by an update — drop the cache entry (decrement done above)
this.verbSubtypeByIdCache.delete(id)
}
// CRITICAL FIX: Increment verb count for new relationships
// This runs AFTER metadata is saved
// Uses synchronous increment since storage operations are already serialized
@ -2744,6 +2814,13 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// Direct O(1) delete with ID-first path
const path = getVerbMetadataPath(id)
await this.deleteObjectFromBranch(path)
// Symmetric verb subtype decrement
const priorEntry = this.verbSubtypeByIdCache.get(id)
if (priorEntry) {
this.verbSubtypeByIdCache.delete(id)
this.decrementVerbSubtypeCount(priorEntry.verb, priorEntry.subtype)
}
}
// ============================================================================
@ -2965,6 +3042,124 @@ export abstract class BaseStorage extends BaseStorageAdapter {
prodLog.info(`[BaseStorage] Rebuilt subtype counts: ${totals} entities across ${this.subtypeCountsByType.size} NounTypes`)
}
/**
* Increment the (verb, subtype) count, creating the inner map on first use.
* Verb-side mirror of `incrementSubtypeCount`. Indexed by VerbType index so it
* lines up with `verbCountsByType` and can be reduced into per-verb totals
* without re-keying.
*/
protected incrementVerbSubtypeCount(verb: VerbType, subtype: string): void {
const verbIdx = TypeUtils.getVerbIndex(verb)
if (verbIdx < 0) return
let inner = this.verbSubtypeCountsByType.get(verbIdx)
if (!inner) {
inner = new Map<string, number>()
this.verbSubtypeCountsByType.set(verbIdx, inner)
}
inner.set(subtype, (inner.get(subtype) || 0) + 1)
}
/**
* Decrement the (verb, subtype) count. Mirror of `decrementSubtypeCount`.
* Deletes the inner key when it reaches 0 and the outer entry when its inner
* map is empty, so the persisted shape stays compact across heavy churn.
*/
protected decrementVerbSubtypeCount(verb: VerbType, subtype: string): void {
const verbIdx = TypeUtils.getVerbIndex(verb)
if (verbIdx < 0) return
const inner = this.verbSubtypeCountsByType.get(verbIdx)
if (!inner) return
const next = (inner.get(subtype) || 0) - 1
if (next <= 0) {
inner.delete(subtype)
if (inner.size === 0) this.verbSubtypeCountsByType.delete(verbIdx)
} else {
inner.set(subtype, next)
}
}
/**
* Load `_system/verb-subtype-statistics.json` into `verbSubtypeCountsByType`.
* Persisted shape mirrors the noun-side rollup:
* `{ counts: { [verbIdx]: { [subtype]: count } }, updatedAt }`. Missing file
* or parse error → start from empty.
*/
protected async loadVerbSubtypeStatistics(): Promise<void> {
try {
const stats = await this.readObjectFromPath(`${SYSTEM_DIR}/verb-subtype-statistics.json`)
if (stats && stats.counts && typeof stats.counts === 'object') {
this.verbSubtypeCountsByType.clear()
for (const [verbKey, subtypeMap] of Object.entries(stats.counts as Record<string, Record<string, number>>)) {
const verbIdx = Number(verbKey)
if (!Number.isInteger(verbIdx) || verbIdx < 0 || verbIdx >= VERB_TYPE_COUNT) continue
const inner = new Map<string, number>()
for (const [subtype, count] of Object.entries(subtypeMap)) {
if (typeof count === 'number' && count > 0) inner.set(subtype, count)
}
if (inner.size > 0) this.verbSubtypeCountsByType.set(verbIdx, inner)
}
}
} catch {
// No existing verb subtype statistics, starting fresh.
}
}
/**
* Save verb subtype statistics to storage. Mirrors `saveSubtypeStatistics`.
* Same persistence cadence (flushCounts + periodic saves alongside other
* type-statistics).
*/
protected async saveVerbSubtypeStatistics(): Promise<void> {
const counts: Record<string, Record<string, number>> = {}
for (const [verbIdx, inner] of this.verbSubtypeCountsByType.entries()) {
const innerObj: Record<string, number> = {}
for (const [subtype, count] of inner.entries()) innerObj[subtype] = count
counts[String(verbIdx)] = innerObj
}
await this.writeObjectToPath(`${SYSTEM_DIR}/verb-subtype-statistics.json`, {
counts,
updatedAt: Date.now()
})
}
/**
* Rebuild verb subtype counts from on-disk metadata. Companion to
* `rebuildSubtypeCounts()`. O(N) over all verbs. Used for poison recovery
* and explicit repair via `brainy inspect repair`.
*/
public async rebuildVerbSubtypeCounts(): Promise<void> {
prodLog.info('[BaseStorage] Rebuilding verb subtype counts from storage...')
this.verbSubtypeCountsByType.clear()
this.verbSubtypeByIdCache.clear()
for (let shard = 0; shard < 256; shard++) {
const shardHex = shard.toString(16).padStart(2, '0')
const shardDir = `entities/verbs/${shardHex}`
try {
const paths = await this.listObjectsInBranch(shardDir)
for (const path of paths) {
if (!path.includes('/metadata.json')) continue
try {
const metadata = await this.readWithInheritance(path)
if (metadata && metadata.verb && typeof metadata.subtype === 'string' && metadata.subtype.length > 0) {
const verb = metadata.verb as VerbType
const subtype = metadata.subtype as string
this.incrementVerbSubtypeCount(verb, subtype)
const segments = path.split('/')
const idSeg = segments[segments.length - 2]
if (idSeg) this.verbSubtypeByIdCache.set(idSeg, { verb, subtype })
}
} catch { /* skip unreadable verbs */ }
}
} catch { /* skip missing shards */ }
}
await this.saveVerbSubtypeStatistics()
const totals = Array.from(this.verbSubtypeCountsByType.values())
.reduce((sum, inner) => sum + Array.from(inner.values()).reduce((s, n) => s + n, 0), 0)
prodLog.info(`[BaseStorage] Rebuilt verb subtype counts: ${totals} relationships across ${this.verbSubtypeCountsByType.size} VerbTypes`)
}
/**
* Persist both counter systems atomically when an explicit flush is
* requested. `super.flushCounts()` writes `entityCounts` (the Map in
@ -2982,6 +3177,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
await super.flushCounts()
await this.saveTypeStatistics()
await this.saveSubtypeStatistics()
await this.saveVerbSubtypeStatistics()
}
/**
@ -3004,6 +3200,15 @@ export abstract class BaseStorage extends BaseStorageAdapter {
return this.subtypeCountsByType
}
/**
* Get verb subtype counts (O(1) access). Verb-side mirror of
* `getSubtypeCountsByType`. Outer key: VerbType index. Inner map: subtype → count.
* Returned map is the live in-memory view — callers must treat it as read-only.
*/
public getVerbSubtypeCountsByType(): Map<number, Map<string, number>> {
return this.verbSubtypeCountsByType
}
/**
* Get verb counts by type (O(1) access to type statistics)
* Exposed for MetadataIndexManager to use as single source of truth
@ -3447,6 +3652,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
results.push({
...verb,
subtype: (metadata as any).subtype as string | undefined,
weight: metadata.weight,
confidence: metadata.confidence,
createdAt: metadata.createdAt
@ -3501,6 +3707,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
results.push({
...verb,
subtype: (metadata as any)?.subtype as string | undefined,
weight: metadata?.weight,
confidence: metadata?.confidence,
createdAt: metadata?.createdAt
@ -3675,6 +3882,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
if (verb && metadata) {
results.push({
...verb,
subtype: (metadata as any).subtype as string | undefined,
weight: metadata.weight,
confidence: metadata.confidence,
createdAt: metadata.createdAt
@ -3722,6 +3930,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
results.push({
...verb,
subtype: (metadata as any)?.subtype as string | undefined,
weight: metadata?.weight,
confidence: metadata?.confidence,
createdAt: metadata?.createdAt
@ -3779,7 +3988,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// Extract standard fields from metadata to top-level
const metadataObj = (metadata || {}) as VerbMetadata
const { createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadataObj
const { subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadataObj
const verbWithMetadata: HNSWVerbWithMetadata = {
id: hnswVerb.id,
@ -3788,6 +3997,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
verb: hnswVerb.verb,
sourceId: hnswVerb.sourceId,
targetId: hnswVerb.targetId,
subtype: subtype as string | undefined,
createdAt: (createdAt as number) || Date.now(),
updatedAt: (updatedAt as number) || Date.now(),
confidence: confidence as number | undefined,

56
src/types/brainy.types.ts
View file

@ -68,6 +68,15 @@ export interface Relation<T = any> {
to: string
/** Relationship type classification (VerbType enum) */
type: VerbType
/**
* Per-product sub-classification within the VerbType (e.g. a `Manages`
* relationship might have `subtype: 'direct'` vs `'dotted-line'`; a `RelatedTo`
* edge might carry `'spouse'` / `'sibling'` / `'colleague'`). Flat string, no
* hierarchy. Top-level standard field — indexed on the fast path and rolled into
* per-VerbType statistics so queries like `getRelations({ verb, subtype })` hit
* the column-store directly.
*/
subtype?: string
/** Connection strength (0-1, default: 1.0) */
weight?: number
/** Opaque content for the relationship (overrides auto-computed vector if provided) */
@ -220,6 +229,14 @@ export interface RelateParams<T = any> {
to: string
/** Relationship type classification (required) */
type: VerbType
/**
* Per-product sub-classification within the VerbType (e.g. a `Manages`
* relationship might have `subtype: 'direct'` or `'dotted-line'`; a `RelatedTo`
* edge might carry `'spouse'` or `'colleague'`). Flat string, no hierarchy.
* Indexed and rolled up into per-VerbType statistics for fast filtering
* (`getRelations({ verb, subtype })`) and aggregation (`groupBy:['subtype']`).
*/
subtype?: string
/** Connection strength (0-1, default: 1.0) */
weight?: number
/** Content for the relationship (optional — overrides auto-computed vector) */
@ -241,7 +258,10 @@ export interface RelateParams<T = any> {
*/
export interface UpdateRelationParams<T = any> {
id: string // Relation to update
type?: VerbType // Change verb type
subtype?: string // Change sub-classification (omit to preserve existing)
weight?: number // New weight
confidence?: number // New confidence (0-1)
data?: any // New content
metadata?: Partial<T> // Metadata to update
merge?: boolean // Merge or replace metadata
@ -334,6 +354,13 @@ export interface GraphConstraints {
from?: string // Connected from this entity
via?: VerbType | VerbType[] // Via these relationship types
type?: VerbType | VerbType[] // Alias for via
/**
* Filter traversal edges by VerbType subtype. Single string for equality,
* array for set membership. Composes with `via` — `{ via: 'manages', subtype:
* 'direct' }` traverses only direct-management edges, not dotted-line. Edges
* without a subtype value are excluded when this filter is set.
*/
subtype?: string | string[]
depth?: number // Max traversal depth (default: 1)
direction?: 'in' | 'out' | 'both' // Direction of traversal (default: 'both')
bidirectional?: boolean // Consider both directions
@ -423,6 +450,15 @@ export interface GetRelationsParams {
*/
type?: VerbType | VerbType[]
/**
* Filter by VerbType subtype
*
* Top-level standard field — uses the fast path, not the metadata fallback.
* Pass a single string for equality (e.g. `subtype: 'direct'`), or an array
* for set membership (e.g. `subtype: ['direct', 'dotted-line']`).
*/
subtype?: string | string[]
/**
* Maximum number of results to return
*
@ -1099,6 +1135,26 @@ export interface BrainyConfig {
// - true: Automatically run pending migrations during init() for small datasets (<10K entities)
autoMigrate?: boolean
/**
* Brain-wide subtype enforcement mode.
*
* Opt-in in 7.30.0 (default: `false`); becomes the default in 8.0.0.
*
* - `false` / `undefined` (default): no brain-wide check. Per-type rules
* registered via `brain.requireSubtype(type, options)` still apply.
* - `true`: every `add()` / `addMany()` / `update()` / `relate()` /
* `relateMany()` / `updateRelation()` rejects writes where the entity's
* NounType (or relationship's VerbType) has no non-empty `subtype` value.
* - `{ except: [NounType.Thing, NounType.Custom] }`: same as `true`, but
* the listed types are allowed through without a subtype. Use for genuine
* catch-all types where no subtype makes sense.
*
* Per-type registrations always compose with the brain-wide flag — a type
* registered with `requireSubtype(type, { required: true })` is always
* enforced regardless of this flag.
*/
requireSubtype?: boolean | { except: Array<import('../types/graphTypes.js').NounType | import('../types/graphTypes.js').VerbType> }
/**
* Process role for multi-process safety on filesystem storage.
*

40
src/utils/paramValidation.ts
View file

@ -5,7 +5,7 @@
* Only enforces universal truths, learns everything else
*/
import { FindParams, AddParams, UpdateParams, RelateParams } from '../types/brainy.types.js'
import { FindParams, AddParams, UpdateParams, RelateParams, UpdateRelationParams } from '../types/brainy.types.js'
import { NounType, VerbType } from '../types/graphTypes.js'
// Dynamic import for Node.js os and fs modules
@ -407,7 +407,7 @@ export function validateRelateParams(params: RelateParams): void {
if (!params.from) {
throw new Error('from entity ID is required')
}
if (!params.to) {
throw new Error('to entity ID is required')
}
@ -421,7 +421,7 @@ export function validateRelateParams(params: RelateParams): void {
} else if (!Object.values(VerbType).includes(params.type)) {
throw new Error(`invalid VerbType: ${params.type}`)
}
// Universal truth: weight must be 0-1
if (params.weight !== undefined) {
if (params.weight < 0 || params.weight > 1) {
@ -430,6 +430,40 @@ export function validateRelateParams(params: RelateParams): void {
}
}
/**
* Validate UpdateRelationParams. Mirror of validateUpdateParams for verbs —
* requires id + at least one field to change; bounds-checks weight/confidence;
* accepts type/subtype/weight/confidence/data/metadata changes.
*/
export function validateUpdateRelationParams(params: UpdateRelationParams): void {
if (!params.id) {
throw new Error('id is required for updateRelation')
}
if (
!params.data &&
!params.metadata &&
!params.type &&
params.subtype === undefined &&
params.weight === undefined &&
params.confidence === undefined
) {
throw new Error('updateRelation: must specify at least one field to update')
}
if (params.type !== undefined && !Object.values(VerbType).includes(params.type)) {
throw new Error(`invalid VerbType: ${params.type}`)
}
if (params.weight !== undefined && (params.weight < 0 || params.weight > 1)) {
throw new Error('weight must be between 0 and 1')
}
if (params.confidence !== undefined && (params.confidence < 0 || params.confidence > 1)) {
throw new Error('confidence must be between 0 and 1')
}
}
/**
* Get current validation configuration
* Useful for debugging and monitoring

4
src/vfs/VirtualFileSystem.ts
View file

@ -245,6 +245,7 @@ export class VirtualFileSystem implements IVirtualFileSystem {
id: rootId, // Fixed ID - storage ensures uniqueness
data: '/',
type: NounType.Collection,
subtype: 'vfs-root', // Standard subtype for the VFS root collection (7.30+)
metadata: this.getRootMetadata()
})
@ -511,6 +512,7 @@ export class VirtualFileSystem implements IVirtualFileSystem {
const entity = await this.brain.add({
data: embeddingData, // Always provide string for embeddings
type: this.getFileNounType(mimeType),
subtype: 'vfs-file', // Standard subtype for VFS file entities (7.30+)
metadata
})
@ -519,6 +521,7 @@ export class VirtualFileSystem implements IVirtualFileSystem {
from: parentId,
to: entity,
type: VerbType.Contains,
subtype: 'vfs-contains', // Standard subtype for VFS containment edges (7.30+)
metadata: { isVFS: true } // Mark as VFS relationship
})
@ -890,6 +893,7 @@ export class VirtualFileSystem implements IVirtualFileSystem {
const entity = await this.brain.add({
data: path, // Directory path as string content
type: NounType.Collection,
subtype: 'vfs-directory', // Standard subtype for VFS directory entities (7.30+)
metadata
})