feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
All checks were successful
CI / Node 22 (push) Successful in 12m9s
CI / Node 24 (push) Successful in 12m4s
CI / Bun (latest) (push) Successful in 12m52s

The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.

- The reserved-name write door DIES: add/update/relate/updateRelation
  metadata bags accept EVERY name (confidence, type, id, data, level,
  content, ...) as ordinary user fields — indexed, filterable, sortable,
  aggregatable, identical to any other field. The remap/enforce/warn
  machinery, the reservedFieldPolicy config (now a typed init refusal),
  and the compile-time metadata key bans are all removed. The one write
  refusal left: keys spelled 'system.*' (namespace forgery), now enforced
  on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
  nested verbatim under 'metadata', sealed by a format stamp — by-name
  storage discrimination is unsound once colliders are admitted. Legacy
  flat records stay readable forever through the shape-aware splitters
  (sound for them: the old door refused colliders). Time travel rides the
  same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
  excludeFields/indexedFields knobs and their silent-[] holes are gone;
  bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
  the frozen 'system.type' column (addToIndex sort, affinity tracking,
  cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
  pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
  (bare 'visibility' was a silent no-op under the law — VFS/system
  entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
  reads the bag first (colliders were absent-shadowed by its own guard)
  and never serves system addresses from the bag; blob history refs read
  the bag shape-aware; migration transforms now receive ONE normalized
  view (engine fields + nested bag) regardless of stored era, and stray
  flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
  gates-green): all ten collider names + plumbing names written as user
  fields, verified verbatim + queryable across live reads, flush+reopen,
  a forced epoch rebuild, and asOf time travel; relation mirror; forgery
  refusals; legacy flat-record compat. 8/8 green.

Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
This commit is contained in:
David Snelling 2026-08-03 16:59:13 -07:00
parent 48a6130a50
commit 24bf6cdbc5
32 changed files with 1355 additions and 1905 deletions

View file

@ -148,7 +148,9 @@ import {
import { NounType, VerbType, TypeUtils } from './types/graphTypes.js'
import {
splitNounMetadataRecord,
splitVerbMetadataRecord
splitVerbMetadataRecord,
buildNounMetadataRecord,
buildVerbMetadataRecord
} from './types/reservedFields.js'
import { BrainyInterface } from './types/brainyInterface.js'
import type { IntegrationHub, IntegrationHubConfig } from './integrations/core/IntegrationHub.js'
@ -746,6 +748,21 @@ export class Brainy<T = any> implements BrainyInterface<T> {
private lazyRebuildPromise: Promise<void> | null = null
constructor(config?: BrainyConfig) {
// The reserved-field write policy died with the field-addressing law:
// every metadata name is the user's now (engine scalars write via their
// dedicated params and read at `system.*`), so there is nothing left for
// the policy to govern. A config still passing it refuses loudly rather
// than being silently ignored.
if (config && 'reservedFieldPolicy' in (config as Record<string, unknown>)) {
throw new Error(
`reservedFieldPolicy was removed by the field-addressing law: metadata field ` +
`names are never reserved anymore — every name in the metadata bag is the ` +
`user's and works like any other field. Set engine scalars via their ` +
`dedicated params (confidence, weight, subtype, …) and query them as ` +
`system.<field>. Remove the reservedFieldPolicy option.`
)
}
// Normalize configuration with defaults
this.config = this.normalizeConfig(config)
@ -2018,12 +2035,6 @@ export class Brainy<T = any> implements BrainyInterface<T> {
// Zero-config validation (static import for performance)
validateAddParams(params)
// Reserved fields arriving via the metadata bag (untyped callers — the
// compile-time guard stops TypeScript callers) are normalized to their
// canonical top-level location BEFORE any enforcement runs, so a
// remapped subtype participates in subtype-pairing enforcement and the
// indexed metadata bag carries only custom fields.
params = this.remapReservedAddMetadata(params)
// Tracked-field vocabulary enforcement (Layer 2). Walks both bags so a
// tracked field declared at top level (e.g. 'subtype') and one declared in
@ -2095,28 +2106,33 @@ export class Brainy<T = any> implements BrainyInterface<T> {
)
}
// Prepare metadata for storage
// data is stored opaquely in the 'data' field - NOT spread into top-level metadata.
// Only metadata fields are queryable via find({ where }).
const storageMetadata = {
...params.metadata,
// Preserve the caller's original (non-UUID) id when normalized, so reads
// can surface it. A real UUID passes through with no _originalId.
...(originalId !== undefined && { [ORIGINAL_ID_KEY]: originalId }),
data: params.data,
noun: params.type,
...(params.subtype !== undefined && { subtype: params.subtype }),
// visibility: stored only when not 'public' (absent === public, keeps records lean)
...(params.visibility !== undefined &&
params.visibility !== 'public' && { visibility: params.visibility }),
service: params.service,
createdAt: Date.now(),
updatedAt: Date.now(),
_rev: 1,
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.weight !== undefined && { weight: params.weight }),
...(params.createdBy && { createdBy: params.createdBy })
}
// Prepare metadata for storage: a v2 nested-bag record — engine fields
// top-level, the user's bag nested VERBATIM (any name, including engine
// spellings like `confidence` or `type`, is the user's and survives
// faithfully; the field-addressing law).
const storageMetadata = buildNounMetadataRecord(
{
data: params.data,
noun: params.type,
...(params.subtype !== undefined && { subtype: params.subtype }),
// visibility: stored only when not 'public' (absent === public, keeps records lean)
...(params.visibility !== undefined &&
params.visibility !== 'public' && { visibility: params.visibility }),
service: params.service,
createdAt: Date.now(),
updatedAt: Date.now(),
_rev: 1,
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.weight !== undefined && { weight: params.weight }),
...(params.createdBy && { createdBy: params.createdBy })
},
{
...params.metadata,
// Preserve the caller's original (non-UUID) id when normalized, so reads
// can surface it. A real UUID passes through with no _originalId.
...(originalId !== undefined && { [ORIGINAL_ID_KEY]: originalId })
}
)
// Build entity structure for indexing (NEW - with top-level fields)
// Optional fields must use conditional spreading to match storageMetadata exactly.
@ -2627,320 +2643,6 @@ export class Brainy<T = any> implements BrainyInterface<T> {
return entity
}
/** One-shot registry for reserved-field warnings (per process, per method+field). */
private static warnedReservedFields = new Set<string>()
/**
* @description Resolve the human-readable "correct write path" guidance for a
* reserved field on a given write method. Single source of truth shared by the
* `'throw'` (Error message) and `'warn'` (one-shot warning) paths so the two
* never drift. The trio `confidence` / `weight` / `subtype` and the
* add()/relate()-time fields `service` / `createdBy` / `visibility` map to a
* dedicated param; everything else is system-managed.
* @param method - The public write method the bag arrived through.
* @param field - The reserved field name found in the metadata bag.
* @returns Guidance naming the correct way to set the field.
*/
private reservedWritePath(
method: 'add' | 'update' | 'relate' | 'updateRelation',
field: string
): string {
const typeParam = "the top-level 'type' param"
switch (field) {
case 'noun':
case 'verb':
return typeParam
case 'data':
return "the top-level 'data' param"
case 'confidence':
return "the 'confidence' param"
case 'weight':
return "the 'weight' param"
case 'subtype':
return "the 'subtype' param"
case 'visibility':
return "the 'visibility' param ('public' | 'internal')"
case 'service':
return method === 'add'
? "the 'service' param of add()"
: method === 'relate'
? "the 'service' param of relate()"
: 'nothing — service is fixed at create time'
case 'createdBy':
return method === 'add'
? "the 'createdBy' param of add()"
: 'nothing — createdBy is system-managed'
case 'createdAt':
return 'nothing — creation time is set automatically'
case 'updatedAt':
return 'nothing — set automatically on every write'
case '_rev':
return method === 'update'
? "the 'ifRev' param for optimistic concurrency"
: 'nothing — revisions are system-managed'
default:
return 'a dedicated top-level param'
}
}
/**
* @description Enforce {@link BrainyConfig.reservedFieldPolicy} for reserved
* fields found inside a metadata bag. Called by every write-path remap once
* the bag has been split and at least one reserved key is present.
*
* - `'throw'` (default): throw a clear Error naming every offending key and
* its correct write path. The caller never reaches the remap.
* - `'warn'`: emit a ONE-SHOT (per method+field, per process) warning for
* EVERY reserved key found both the user-mutable fields that are about to
* be remapped and the system-managed fields that are about to be dropped
* then fall through to the legacy remap.
* - `'remap'`: silent legacy remap, no warning.
*
* @param method - The public write method the bag arrived through.
* @param reserved - The reserved half of the split metadata bag (non-empty).
* @param reservedListName - `'RESERVED_ENTITY_FIELDS'` or
* `'RESERVED_RELATION_FIELDS'` named in the thrown Error for discoverability.
* @returns `true` when the caller should proceed with the legacy remap
* (`'warn'` / `'remap'`); `'throw'` never returns (it throws first).
* @throws {Error} When the policy is `'throw'` and any reserved key is present.
*/
private enforceReservedPolicy(
method: 'add' | 'update' | 'relate' | 'updateRelation',
reserved: Partial<Record<string, unknown>>,
reservedListName: 'RESERVED_ENTITY_FIELDS' | 'RESERVED_RELATION_FIELDS'
): boolean {
const policy = this.config.reservedFieldPolicy ?? 'throw'
const keys = Object.keys(reserved)
if (keys.length === 0) return true
if (policy === 'throw') {
const detail = keys
.map((k) => {
const path = this.reservedWritePath(method, k)
// System-managed fields resolve to a "nothing — …" sentinel; phrase
// those as "is system-managed" rather than "pass it as the nothing".
return path.startsWith('nothing')
? `metadata.${k} is a reserved field (${path.replace(/^nothing\s*—\s*/, '')}) and cannot be set through ${method}()`
: `metadata.${k} is a reserved field — pass it as ${path} to ${method}()`
})
.join('; ')
throw new Error(
`${detail} (reserved: see ${reservedListName}). ` +
`Set reservedFieldPolicy:'remap' to opt into legacy remapping, ` +
`or reservedFieldPolicy:'warn' to remap with a warning.`
)
}
if (policy === 'warn') {
// One-shot warning for EVERY reserved key (today only system-managed ones
// warn — this closes that gap so user-mutable remaps are visible too).
for (const k of keys) {
this.warnReservedRemapped(method, k, this.reservedWritePath(method, k))
}
}
// 'warn' and 'remap' both fall through to the legacy remap.
return true
}
/**
* @description One-shot (per method+field, per process) warning that a
* reserved field arrived inside a metadata bag under the `'warn'` policy. The
* wording is neutral on "remapped vs dropped" `reservedWritePath()` already
* tells the caller where the value goes (a dedicated param, or "nothing").
* @param method - The public write method the bag arrived through.
* @param field - The reserved field name found in the bag.
* @param rightPath - Guidance naming the correct write path.
*/
private warnReservedRemapped(method: string, field: string, rightPath: string): void {
const key = `${method}:${field}`
if (Brainy.warnedReservedFields.has(key)) return
Brainy.warnedReservedFields.add(key)
// System-managed fields resolve to a "nothing — …" sentinel; phrase the
// guidance so it reads cleanly in both the remapped and dropped cases.
const guidance = rightPath.startsWith('nothing')
? `it is ${rightPath.replace(/^nothing\s*—\s*/, '')} and was dropped`
: `set it via ${rightPath} instead`
prodLog.warn(
`[brainy] ${method}(): '${field}' is a reserved field and was found inside the ` +
`metadata bag — ${guidance}. (Legacy remap applied because ` +
`reservedFieldPolicy is 'warn'. This warning is shown once per field per process.)`
)
}
/**
* @description Normalize an `add()` params object with respect to
* Brainy-reserved fields arriving inside `metadata` (untyped callers only
* the compile-time guard on `AddParams.metadata` stops TypeScript callers).
* Governed by {@link BrainyConfig.reservedFieldPolicy} (default `'throw'`):
* `'throw'` rejects the write naming the offending key(s); `'warn'`/`'remap'`
* fall through to the legacy remap, where fields with a dedicated `add()`
* param (`confidence`, `weight`, `subtype`, `visibility`, `service`,
* `createdBy`) are remapped to that param unless the caller also passed it
* explicitly (top-level wins) and system-managed fields (`noun`, `data`,
* `createdAt`, `updatedAt`, `_rev`) are dropped. A remapped `subtype` flows
* through subtype-pairing enforcement exactly like a top-level one.
* @param params - The caller's add params (not mutated).
* @returns Params with reserved fields normalized out of `metadata`.
* @throws {Error} When `reservedFieldPolicy` is `'throw'` and the bag carries a reserved key.
*/
private remapReservedAddMetadata(params: AddParams<T>): AddParams<T> {
const bag = params.metadata as Record<string, unknown> | undefined
if (!bag || typeof bag !== 'object') return params
const { reserved, custom } = splitNounMetadataRecord(bag)
if (Object.keys(reserved).length === 0) return params
// Policy gate: 'throw' (default) throws here; 'warn' warns once per key then
// remaps; 'remap' silently remaps. (Throw never returns.)
this.enforceReservedPolicy('add', reserved, 'RESERVED_ENTITY_FIELDS')
const createdBy = reserved.createdBy as { augmentation?: unknown; version?: unknown } | undefined
const createdByValid =
typeof createdBy === 'object' &&
createdBy !== null &&
typeof createdBy.augmentation === 'string' &&
typeof createdBy.version === 'string'
return {
...params,
metadata: custom as AddParams<T>['metadata'],
...(params.confidence === undefined &&
typeof reserved.confidence === 'number' && { confidence: reserved.confidence }),
...(params.weight === undefined &&
typeof reserved.weight === 'number' && { weight: reserved.weight }),
...(params.subtype === undefined &&
typeof reserved.subtype === 'string' && { subtype: reserved.subtype }),
...(params.visibility === undefined &&
(reserved.visibility === 'public' || reserved.visibility === 'internal') && {
visibility: reserved.visibility as 'public' | 'internal'
}),
...(params.service === undefined &&
typeof reserved.service === 'string' && { service: reserved.service }),
...(params.createdBy === undefined &&
createdByValid && { createdBy: createdBy as { augmentation: string; version: string } })
}
}
/**
* @description Normalize an `update()` params object with respect to
* Brainy-reserved fields arriving inside the metadata patch the `update()`
* mirror of {@link remapReservedAddMetadata}, closing the historical trap
* where `add({metadata:{confidence}})` lifted the field but
* `update({metadata:{confidence}})` silently dropped it (the patch value
* survived the merge and was then clobbered by the preserve-existing
* spread; a production consumer's confidence-evolution writes no-oped until
* read back). Governed by {@link BrainyConfig.reservedFieldPolicy} (default
* `'throw'`): `'throw'` rejects the write; `'warn'`/`'remap'` remap
* user-mutable fields (`confidence`, `weight`, `subtype`) to their dedicated
* param unless the caller also passed it (top-level wins) and drop everything
* else (`noun`, `data`, `createdAt`, `updatedAt`, `service`, `createdBy`,
* `_rev`) as system-managed or fixed at `add()` time.
* @param params - The caller's update params (not mutated).
* @returns Params with reserved fields normalized out of `metadata`.
* @throws {Error} When `reservedFieldPolicy` is `'throw'` and the bag carries a reserved key.
*/
private remapReservedUpdateMetadata(params: UpdateParams<T>): UpdateParams<T> {
const bag = params.metadata as Record<string, unknown> | undefined
if (!bag || typeof bag !== 'object') return params
const { reserved, custom } = splitNounMetadataRecord(bag)
if (Object.keys(reserved).length === 0) return params
// Policy gate: 'throw' (default) throws; 'warn' warns once per key then
// remaps; 'remap' silently remaps.
this.enforceReservedPolicy('update', reserved, 'RESERVED_ENTITY_FIELDS')
return {
...params,
metadata: custom as UpdateParams<T>['metadata'],
...(params.confidence === undefined &&
typeof reserved.confidence === 'number' && { confidence: reserved.confidence }),
...(params.weight === undefined &&
typeof reserved.weight === 'number' && { weight: reserved.weight }),
...(params.subtype === undefined &&
typeof reserved.subtype === 'string' && { subtype: reserved.subtype })
}
}
/**
* @description Normalize a `relate()` params object with respect to
* Brainy-reserved fields arriving inside `metadata` the relationship
* mirror of {@link remapReservedAddMetadata}. Governed by
* {@link BrainyConfig.reservedFieldPolicy} (default `'throw'`): `'throw'`
* rejects the write; `'warn'`/`'remap'` remap fields with a dedicated
* `relate()` param (`confidence`, `weight`, `subtype`, `visibility`,
* `service`) to that param (top-level wins) and drop system-managed fields
* (`verb`, `data`, `createdAt`, `updatedAt`, `createdBy`, `_rev`).
* @param params - The caller's relate params (not mutated).
* @returns Params with reserved fields normalized out of `metadata`.
* @throws {Error} When `reservedFieldPolicy` is `'throw'` and the bag carries a reserved key.
*/
private remapReservedRelateMetadata(params: RelateParams<T>): RelateParams<T> {
const bag = params.metadata as Record<string, unknown> | undefined
if (!bag || typeof bag !== 'object') return params
const { reserved, custom } = splitVerbMetadataRecord(bag)
if (Object.keys(reserved).length === 0) return params
// Policy gate: 'throw' (default) throws; 'warn' warns once per key then
// remaps; 'remap' silently remaps.
this.enforceReservedPolicy('relate', reserved, 'RESERVED_RELATION_FIELDS')
return {
...params,
metadata: custom as RelateParams<T>['metadata'],
...(params.confidence === undefined &&
typeof reserved.confidence === 'number' && { confidence: reserved.confidence }),
...(params.weight === undefined &&
typeof reserved.weight === 'number' && { weight: reserved.weight }),
...(params.subtype === undefined &&
typeof reserved.subtype === 'string' && { subtype: reserved.subtype }),
...(params.visibility === undefined &&
(reserved.visibility === 'public' || reserved.visibility === 'internal') && {
visibility: reserved.visibility as 'public' | 'internal'
}),
...(params.service === undefined &&
typeof reserved.service === 'string' && { service: reserved.service })
}
}
/**
* @description Normalize an `updateRelation()` params object with respect
* to Brainy-reserved fields arriving inside the metadata patch the
* relationship mirror of {@link remapReservedUpdateMetadata}. Governed by
* {@link BrainyConfig.reservedFieldPolicy} (default `'throw'`): `'throw'`
* rejects the write; `'warn'`/`'remap'` remap user-mutable fields
* (`confidence`, `weight`, `subtype`, `visibility`) to their dedicated param
* (top-level wins) and drop everything else.
* @param params - The caller's update-relation params (not mutated).
* @returns Params with reserved fields normalized out of `metadata`.
* @throws {Error} When `reservedFieldPolicy` is `'throw'` and the bag carries a reserved key.
*/
private remapReservedUpdateRelationMetadata(
params: UpdateRelationParams<T>
): UpdateRelationParams<T> {
const bag = params.metadata as Record<string, unknown> | undefined
if (!bag || typeof bag !== 'object') return params
const { reserved, custom } = splitVerbMetadataRecord(bag)
if (Object.keys(reserved).length === 0) return params
// Policy gate: 'throw' (default) throws; 'warn' warns once per key then
// remaps; 'remap' silently remaps.
this.enforceReservedPolicy('updateRelation', reserved, 'RESERVED_RELATION_FIELDS')
return {
...params,
metadata: custom as UpdateRelationParams<T>['metadata'],
...(params.confidence === undefined &&
typeof reserved.confidence === 'number' && { confidence: reserved.confidence }),
...(params.weight === undefined &&
typeof reserved.weight === 'number' && { weight: reserved.weight }),
...(params.subtype === undefined &&
typeof reserved.subtype === 'string' && { subtype: reserved.subtype }),
...(params.visibility === undefined &&
(reserved.visibility === 'public' || reserved.visibility === 'internal') && {
visibility: reserved.visibility as 'public' | 'internal'
})
}
}
/**
* Update an existing entity
@ -3006,12 +2708,6 @@ export class Brainy<T = any> implements BrainyInterface<T> {
// Reserved fields arriving via the metadata patch are remapped to their
// canonical top-level location, mirroring add()'s lift. Without this the
// patch value survived the merge but was then clobbered by the
// preserve-existing spreads below — a silent no-op consumers could only
// detect by reading values back. User-mutable fields (confidence,
// weight, subtype) remap unless the same field was also passed top-level
// (top-level wins); system-managed fields are dropped with a one-shot
// warning naming the right path.
params = this.remapReservedUpdateMetadata(params)
// Tracked-field vocabulary enforcement (Layer 2). Same as add() — the
// metadata bag carries fields registered via trackField(), and subtype is
@ -3078,31 +2774,33 @@ export class Brainy<T = any> implements BrainyInterface<T> {
? { ...existing.metadata, ...params.metadata }
: params.metadata || existing.metadata
// Prepare updated metadata object
// data is stored opaquely in the 'data' field - NOT spread into top-level metadata.
const updatedMetadata = {
...newMetadata,
data: params.data !== undefined ? params.data : existing.data,
noun: params.type || existing.type,
service: existing.service,
createdAt: existing.createdAt,
updatedAt: Date.now(),
_rev: currentRev + 1,
// Update confidence and weight if provided, otherwise preserve existing
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.weight !== undefined && { weight: params.weight }),
...(params.confidence === undefined && existing.confidence !== undefined && { confidence: existing.confidence }),
...(params.weight === undefined && existing.weight !== undefined && { weight: existing.weight }),
// Update subtype if provided, otherwise preserve existing
...(params.subtype !== undefined && { subtype: params.subtype }),
...(params.subtype === undefined && existing.subtype !== undefined && { subtype: existing.subtype }),
// Visibility: take the new value if provided, else preserve existing. Stored only
// when the effective value is not 'public' (absent === public, keeps records lean).
// A change to 'public' therefore drops the field entirely.
...(((params.visibility ?? existing.visibility) ?? 'public') !== 'public' && {
visibility: params.visibility ?? existing.visibility
})
}
// Prepare the updated v2 nested-bag record: engine fields top-level,
// the merged user bag nested verbatim (collider names stay the user's).
const updatedMetadata = buildNounMetadataRecord(
{
data: params.data !== undefined ? params.data : existing.data,
noun: params.type || existing.type,
service: existing.service,
createdAt: existing.createdAt,
updatedAt: Date.now(),
_rev: currentRev + 1,
// Update confidence and weight if provided, otherwise preserve existing
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.weight !== undefined && { weight: params.weight }),
...(params.confidence === undefined && existing.confidence !== undefined && { confidence: existing.confidence }),
...(params.weight === undefined && existing.weight !== undefined && { weight: existing.weight }),
// Update subtype if provided, otherwise preserve existing
...(params.subtype !== undefined && { subtype: params.subtype }),
...(params.subtype === undefined && existing.subtype !== undefined && { subtype: existing.subtype }),
// Visibility: take the new value if provided, else preserve existing. Stored only
// when the effective value is not 'public' (absent === public, keeps records lean).
// A change to 'public' therefore drops the field entirely.
...(((params.visibility ?? existing.visibility) ?? 'public') !== 'public' && {
visibility: params.visibility ?? existing.visibility
})
},
newMetadata as Record<string, unknown>
)
// Build entity structure for metadata index (with top-level fields).
// No `level`: engine plumbing never enters the indexing view (it
@ -4043,9 +3741,6 @@ export class Brainy<T = any> implements BrainyInterface<T> {
// engine-minted UUID — relation ids are never caller-supplied here.)
params = { ...params, from: resolveEntityId(params.from), to: resolveEntityId(params.to) }
// Reserved fields arriving via the metadata bag are normalized to their
// canonical top-level params before enforcement — mirror of add()'s lift.
params = this.remapReservedRelateMetadata(params)
// Subtype pairing enforcement (Layer 3 — 7.30.0). Per-type rules registered
// via brain.requireSubtype() compose with the brain-wide strict-mode flag.
@ -4097,25 +3792,28 @@ export class Brainy<T = any> implements BrainyInterface<T> {
(v, i) => (v + toEntity.vector[i]) / 2
)
// Prepare verb metadata
// User metadata spread FIRST, then system fields ALWAYS win (prevents collision)
// Prepare verb metadata: a v2 nested-bag record — engine fields
// top-level, the user's edge bag nested verbatim (any name is the
// user's; the field-addressing law).
// One timestamp for both createdAt and updatedAt so a never-updated edge reports a
// stable updatedAt (=== createdAt) instead of a fresh Date.now() fabricated per read.
const relateTs = Date.now()
const verbMetadata = {
...(params.metadata || {}),
verb: params.type,
...(params.subtype !== undefined && { subtype: params.subtype }),
// visibility: stored only when not 'public' (absent === public, keeps records lean)
...(params.visibility !== undefined &&
params.visibility !== 'public' && { visibility: params.visibility }),
weight: params.weight ?? 1.0,
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.service !== undefined && { service: params.service }),
createdAt: relateTs,
updatedAt: relateTs,
...(params.data !== undefined && { data: params.data })
}
const verbMetadata = buildVerbMetadataRecord(
{
verb: params.type,
...(params.subtype !== undefined && { subtype: params.subtype }),
// visibility: stored only when not 'public' (absent === public, keeps records lean)
...(params.visibility !== undefined &&
params.visibility !== 'public' && { visibility: params.visibility }),
weight: params.weight ?? 1.0,
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.service !== undefined && { service: params.service }),
createdAt: relateTs,
updatedAt: relateTs,
...(params.data !== undefined && { data: params.data })
},
(params.metadata as Record<string, unknown>) || {}
)
// Save to storage (vector and metadata separately)
const verb: GraphVerb = {
@ -4347,9 +4045,6 @@ export class Brainy<T = any> implements BrainyInterface<T> {
validateUpdateRelationParams(params)
// Reserved fields arriving via the metadata patch are remapped to their
// canonical top-level params — mirror of update()'s normalization.
params = this.remapReservedUpdateRelationMetadata(params)
const existing = await this.storage.getVerb(params.id)
if (!existing) {
@ -4378,32 +4073,36 @@ export class Brainy<T = any> implements BrainyInterface<T> {
? { ...(existingRec.metadata || {}), ...(params.metadata || {}) }
: params.metadata || existingRec.metadata
// Build updated stored metadata. System fields ALWAYS win — same shape as relate().
const updatedMetadata = {
...newMetadata,
verb: newVerbType,
...(params.subtype !== undefined
? { subtype: params.subtype }
: existingRec.subtype !== undefined && { subtype: existingRec.subtype }),
// Visibility: new value if provided, else preserve existing; stored only when the
// effective value is not 'public' (a change to 'public' drops the field).
...(((params.visibility ?? existingRec.visibility) ?? 'public') !== 'public' && {
visibility: params.visibility ?? existingRec.visibility
}),
weight: params.weight ?? existingRec.weight ?? 1.0,
...(params.confidence !== undefined
? { confidence: params.confidence }
: existingRec.confidence !== undefined && { confidence: existingRec.confidence }),
// service/createdBy are fixed at relate() time — always carried forward
// (omitting them here silently erased them on every updateRelation()).
...(existingRec.service !== undefined && { service: existingRec.service }),
...(existingRec.createdBy !== undefined && { createdBy: existingRec.createdBy }),
createdAt: existingRec.createdAt,
updatedAt: Date.now(),
...(params.data !== undefined
? { data: params.data }
: existingRec.data !== undefined && { data: existingRec.data })
}
// Build the updated stored record: v2 nested-bag — engine fields
// top-level, the merged user bag nested verbatim (mirror of update()).
const updatedWeight = params.weight ?? existingRec.weight ?? 1.0
const updatedData =
params.data !== undefined ? params.data : existingRec.data
const updatedMetadata = buildVerbMetadataRecord(
{
verb: newVerbType,
...(params.subtype !== undefined
? { subtype: params.subtype }
: existingRec.subtype !== undefined && { subtype: existingRec.subtype }),
// Visibility: new value if provided, else preserve existing; stored only when the
// effective value is not 'public' (a change to 'public' drops the field).
...(((params.visibility ?? existingRec.visibility) ?? 'public') !== 'public' && {
visibility: params.visibility ?? existingRec.visibility
}),
weight: updatedWeight,
...(params.confidence !== undefined
? { confidence: params.confidence }
: existingRec.confidence !== undefined && { confidence: existingRec.confidence }),
// service/createdBy are fixed at relate() time — always carried forward
// (omitting them here silently erased them on every updateRelation()).
...(existingRec.service !== undefined && { service: existingRec.service }),
...(existingRec.createdBy !== undefined && { createdBy: existingRec.createdBy }),
createdAt: existingRec.createdAt,
updatedAt: Date.now(),
...(updatedData !== undefined && { data: updatedData })
},
newMetadata as Record<string, unknown>
)
// Build the verb view used by the graph index — top-level fields mirror relate()'s.
const verbForIndex: GraphVerb = {
@ -4419,9 +4118,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
...(((params.visibility ?? existingRec.visibility) ?? 'public') !== 'public' && {
visibility: params.visibility ?? existingRec.visibility
}),
weight: updatedMetadata.weight,
weight: updatedWeight,
metadata: newMetadata,
data: updatedMetadata.data,
data: updatedData,
createdAt: existingRec.createdAt
}
@ -6027,8 +5726,12 @@ export class Brainy<T = any> implements BrainyInterface<T> {
): Promise<Set<string>> {
const excluded = this.excludedVisibilityTiers(params)
if (!excluded) return new Set()
// 'system.visibility' — the engine scalar's frozen address. A bare
// 'visibility' key would address the USER's metadata bag under the
// field-addressing law and silently hide nothing (VFS/system entities
// would leak into every default read).
const ids = await this.metadataIndex.getIdsForFilter({
visibility: excluded.length === 1 ? excluded[0] : { oneOf: excluded }
'system.visibility': excluded.length === 1 ? excluded[0] : { oneOf: excluded }
})
return new Set(ids)
}
@ -9282,10 +8985,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
): Promise<string> {
const { op: _discriminator, ...rawParams } = op
validateAddParams(rawParams as AddParams<T>)
// Same reserved-field normalization as add() — the metadata bag is
// cleaned BEFORE enforcement so a remapped subtype participates in
// subtype-pairing enforcement and only custom fields reach the index.
const params = this.remapReservedAddMetadata(rawParams as AddParams<T>)
const params = rawParams as AddParams<T>
this.enforceTrackedFieldValues(params.metadata as Record<string, unknown> | undefined, 'metadata')
this.enforceTrackedFieldValues({ subtype: params.subtype } as Record<string, unknown>, 'top-level')
this.enforceSubtypeOnAdd('add', params.type, params.subtype, params.metadata)
@ -9360,25 +9060,31 @@ export class Brainy<T = any> implements BrainyInterface<T> {
plan.createdNouns.add(id)
const now = Date.now()
const storageMetadata = {
...params.metadata,
// Preserve the caller's original (non-UUID) id when normalized — mirror
// of add(). A real UUID passes through with no _originalId.
...(originalId !== undefined && { [ORIGINAL_ID_KEY]: originalId }),
data: params.data,
noun: params.type,
...(params.subtype !== undefined && { subtype: params.subtype }),
// visibility: stored only when not 'public' (absent === public, keeps records lean)
...(params.visibility !== undefined &&
params.visibility !== 'public' && { visibility: params.visibility }),
service: params.service,
createdAt: now,
updatedAt: now,
_rev: 1,
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.weight !== undefined && { weight: params.weight }),
...(params.createdBy && { createdBy: params.createdBy })
}
// v2 nested-bag record — mirror of add(): engine fields top-level, the
// user's bag nested verbatim (collider names stay the user's).
const storageMetadata = buildNounMetadataRecord(
{
data: params.data,
noun: params.type,
...(params.subtype !== undefined && { subtype: params.subtype }),
// visibility: stored only when not 'public' (absent === public, keeps records lean)
...(params.visibility !== undefined &&
params.visibility !== 'public' && { visibility: params.visibility }),
service: params.service,
createdAt: now,
updatedAt: now,
_rev: 1,
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.weight !== undefined && { weight: params.weight }),
...(params.createdBy && { createdBy: params.createdBy })
},
{
...params.metadata,
// Preserve the caller's original (non-UUID) id when normalized — mirror
// of add(). A real UUID passes through with no _originalId.
...(originalId !== undefined && { [ORIGINAL_ID_KEY]: originalId })
}
)
const entityForIndexing = {
id,
vector,
@ -9441,10 +9147,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
): Promise<string> {
const { op: _discriminator, ...rawParams } = op
validateUpdateParams(rawParams as UpdateParams<T>)
// Same reserved-field normalization as update() — user-mutable fields
// remap to their dedicated param (top-level wins), system-managed fields
// drop with a one-shot warning.
const params = this.remapReservedUpdateMetadata(rawParams as UpdateParams<T>)
const params = rawParams as UpdateParams<T>
// Id normalization (8.0) — mirror of update(): a natural key resolves to the
// canonical UUID add() stored. A real UUID passes through.
params.id = resolveEntityId(params.id)
@ -9496,29 +9199,33 @@ export class Brainy<T = any> implements BrainyInterface<T> {
? { ...existing.metadata, ...params.metadata }
: params.metadata || existing.metadata
const now = Date.now()
const updatedMetadata = {
...newMetadata,
data: params.data !== undefined ? params.data : existing.data,
noun: params.type || existing.type,
service: existing.service,
createdAt: existing.createdAt,
updatedAt: now,
_rev: currentRev + 1,
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.weight !== undefined && { weight: params.weight }),
...(params.confidence === undefined &&
existing.confidence !== undefined && { confidence: existing.confidence }),
...(params.weight === undefined &&
existing.weight !== undefined && { weight: existing.weight }),
...(params.subtype !== undefined && { subtype: params.subtype }),
...(params.subtype === undefined &&
existing.subtype !== undefined && { subtype: existing.subtype }),
// Visibility: new value if provided, else preserve existing; stored only when the
// effective value is not 'public' (a change to 'public' drops the field).
...(((params.visibility ?? existing.visibility) ?? 'public') !== 'public' && {
visibility: params.visibility ?? existing.visibility
})
}
// v2 nested-bag record — mirror of update(): engine fields top-level,
// the merged user bag nested verbatim.
const updatedMetadata = buildNounMetadataRecord(
{
data: params.data !== undefined ? params.data : existing.data,
noun: params.type || existing.type,
service: existing.service,
createdAt: existing.createdAt,
updatedAt: now,
_rev: currentRev + 1,
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.weight !== undefined && { weight: params.weight }),
...(params.confidence === undefined &&
existing.confidence !== undefined && { confidence: existing.confidence }),
...(params.weight === undefined &&
existing.weight !== undefined && { weight: existing.weight }),
...(params.subtype !== undefined && { subtype: params.subtype }),
...(params.subtype === undefined &&
existing.subtype !== undefined && { subtype: existing.subtype }),
// Visibility: new value if provided, else preserve existing; stored only when the
// effective value is not 'public' (a change to 'public' drops the field).
...(((params.visibility ?? existing.visibility) ?? 'public') !== 'public' && {
visibility: params.visibility ?? existing.visibility
})
},
newMetadata as Record<string, unknown>
)
// Register for the authoritative under-mutex CAS re-verify + rev re-stamp
// (see PlannedTransact.casUpdates). The staged UpdateNounMetadataOperation
@ -9739,8 +9446,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
): Promise<string> {
const { op: _discriminator, ...rawParams } = op
validateRelateParams(rawParams as RelateParams<T>)
// Same reserved-field normalization as relate().
const params = this.remapReservedRelateMetadata(rawParams as RelateParams<T>)
const params = rawParams as RelateParams<T>
// Id normalization (8.0) — mirror of relate(): resolve BOTH endpoints to the
// canonical UUID add() stored, so a relate op may reference either side by
// natural key. Real UUIDs pass through. (Relationship ids are engine-minted.)
@ -9790,19 +9496,23 @@ export class Brainy<T = any> implements BrainyInterface<T> {
const id = uuidv4()
const relationVector = fromEntity.vector.map((v, i) => (v + toEntity.vector[i]) / 2)
const now = Date.now()
const verbMetadata = {
...(params.metadata || {}),
verb: params.type,
...(params.subtype !== undefined && { subtype: params.subtype }),
// visibility: stored only when not 'public' (absent === public, keeps records lean)
...(params.visibility !== undefined &&
params.visibility !== 'public' && { visibility: params.visibility }),
weight: params.weight ?? 1.0,
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.service !== undefined && { service: params.service }),
createdAt: now,
...(params.data !== undefined && { data: params.data })
}
// v2 nested-bag record — mirror of relate(): engine fields top-level,
// the user's edge bag nested verbatim.
const verbMetadata = buildVerbMetadataRecord(
{
verb: params.type,
...(params.subtype !== undefined && { subtype: params.subtype }),
// visibility: stored only when not 'public' (absent === public, keeps records lean)
...(params.visibility !== undefined &&
params.visibility !== 'public' && { visibility: params.visibility }),
weight: params.weight ?? 1.0,
...(params.confidence !== undefined && { confidence: params.confidence }),
...(params.service !== undefined && { service: params.service }),
createdAt: now,
...(params.data !== undefined && { data: params.data })
},
(params.metadata as Record<string, unknown>) || {}
)
const verb: GraphVerb = {
id,
vector: relationVector,
@ -15115,12 +14825,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
requireSubtype: config?.requireSubtype ?? true,
// Multi-process safety
mode: config?.mode ?? 'writer',
force: config?.force ?? false,
// Reserved-field-in-metadata-bag policy (8.0 — no silent failures).
// Default 'throw': an untyped caller that smuggles a reserved key past
// the compile guard gets a loud Error naming the correct write path.
// 'warn' = remap + one-shot warning per key; 'remap' = legacy silent remap.
reservedFieldPolicy: config?.reservedFieldPolicy ?? 'throw'
force: config?.force ?? false
}
}