feat(8.0): reserved-field contract — one canonical location, typed prevention, unified read/write
Brainy-owned field names (noun/verb, subtype, createdAt, updatedAt,
confidence, weight, service, data, createdBy, _rev) now have exactly one
home — top level — enforced by three layers driven from a single source of
truth, src/types/reservedFields.ts (RESERVED_ENTITY_FIELDS /
RESERVED_RELATION_FIELDS, exported):
1. Compile time — AddParams/UpdateParams/RelateParams/UpdateRelationParams
metadata (and the transact() ops that extend them) reject a literal
reserved key as a TypeScript error while keeping generic T ergonomics
(typed bags, untyped brains, index-signature shapes, and a documented
exemption for T-declared reserved keys). Pinned by @ts-expect-error
type tests run under vitest typecheck mode on every unit run.
2. Write time — the 7.x update() remap is ported to 8.0 and extended to
every write path: add/update/relate/updateRelation, their transact()
mirrors, and db.with() overlays. User-settable fields lift to their
dedicated param (top-level wins when both are supplied — closes the 7.x
trap where update({metadata:{confidence}}) silently no-oped), and
system-managed fields drop with a one-shot warning naming the right
path. A remapped subtype satisfies subtype-pairing enforcement exactly
like a top-level one.
3. Read time — every storage combine goes through one canonical hydration
helper (hydrateNounWithMetadata / hydrateVerbWithMetadata over
splitNoun/VerbMetadataRecord), so reserved fields surface ONLY top-level
and entity/relation.metadata carry ONLY custom fields on live reads,
batch reads, paginated listings, getRelations by source/target, streamed
verbs, and historical asOf() materialization alike.
Read-path echoes found and fixed (previously the full stored record —
including the verb type key — leaked inside metadata): noun pagination,
verb pagination, getVerbsBySource/ByTarget (adjacency + shard fallback),
getVerbsBySourceBatch (which also dropped subtype/data), and the
filesystem verb stream. getRelations() results now surface
confidence/updatedAt top-level via verbsToRelations, updateRelation() no
longer erases service/createdBy, relate() persists its top-level
confidence/service params, and the dead convertHNSWVerbToGraphVerb echo
path is deleted. Import paths (CLI extract, deduplicator, coordinators,
neural import) write confidence through the dedicated param instead of the
bag. UpdateRelationParams is now exported from the package root.
Documented for consumers in docs/concepts/consistency-model.md ("Reserved
fields") and RELEASES.md. Regression tests ported from the 7.x fix and
extended to the full 8.0 contract (17 runtime tests + 41 type-level
assertions); full unit suite 1427/1427, db-mvcc integration 24/24.
This commit is contained in:
parent
c44678390e
commit
970e08c466
18 changed files with 1688 additions and 452 deletions
19
RELEASES.md
19
RELEASES.md
|
|
@ -198,6 +198,25 @@ this rename; only the API surface moved.
|
||||||
through earlier pins and are not reported by `db.since()`. Writes you want
|
through earlier pins and are not reported by `db.since()`. Writes you want
|
||||||
to travel back through go through `transact()`. This is the documented
|
to travel back through go through `transact()`. This is the documented
|
||||||
contract, stated rather than papered over.
|
contract, stated rather than papered over.
|
||||||
|
- **Reserved fields have one canonical location — enforced at every layer.**
|
||||||
|
The Brainy-owned field names (`RESERVED_ENTITY_FIELDS`:
|
||||||
|
`noun`/`subtype`/`createdAt`/`updatedAt`/`confidence`/`weight`/`service`/
|
||||||
|
`data`/`createdBy`/`_rev`; verb mirror `RESERVED_RELATION_FIELDS` with
|
||||||
|
`verb` for the type key) are now (1) a **compile error** inside any
|
||||||
|
`metadata` param — `add`/`update`/`relate`/`updateRelation` and the
|
||||||
|
matching `transact()` ops; (2) **normalized at write time** for untyped
|
||||||
|
callers — user-settable fields remap to their dedicated top-level param
|
||||||
|
(top-level wins; `update({metadata:{confidence}})` no longer silently
|
||||||
|
no-ops, closing a 7.x trap), system-managed fields drop with a one-shot
|
||||||
|
warning naming the right path; (3) **split at read time** through one
|
||||||
|
canonical helper, so `entity.metadata` / `relation.metadata` contain ONLY
|
||||||
|
custom fields on every read path — `get`, `find`, `getRelations` (several
|
||||||
|
paginated/by-source/by-target paths previously echoed the full stored
|
||||||
|
record, including the `verb` type key, inside `metadata`), batch reads,
|
||||||
|
and historical `asOf()` reads. `getRelations()` results now also surface
|
||||||
|
`confidence`/`updatedAt` top-level, and `updateRelation()` no longer
|
||||||
|
erases a relationship's `service`/`createdBy`. See "Reserved fields" in
|
||||||
|
`docs/concepts/consistency-model.md`.
|
||||||
- **Unchanged from 7.31:** per-entity `_rev`, `update({ ifRev })`
|
- **Unchanged from 7.31:** per-entity `_rev`, `update({ ifRev })`
|
||||||
(`RevisionConflictError`), and `add({ ifAbsent })` work exactly as before,
|
(`RevisionConflictError`), and `add({ ifAbsent })` work exactly as before,
|
||||||
and `ifRev` is also accepted on `transact()` update operations (a conflict
|
and `ifRev` is also accepted on `transact()` update operations (a conflict
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,7 @@ public: true
|
||||||
category: concepts
|
category: concepts
|
||||||
template: concept
|
template: concept
|
||||||
order: 4
|
order: 4
|
||||||
description: The exact guarantees behind Brainy's Db API — snapshot isolation, atomic transactions, two levels of compare-and-swap, time travel, retention, snapshots, and crash recovery.
|
description: The exact guarantees behind Brainy's Db API — snapshot isolation, atomic transactions, two levels of compare-and-swap, time travel, retention, snapshots, crash recovery, and the reserved-field contract.
|
||||||
next:
|
next:
|
||||||
- guides/snapshots-and-time-travel
|
- guides/snapshots-and-time-travel
|
||||||
- guides/optimistic-concurrency
|
- guides/optimistic-concurrency
|
||||||
|
|
@ -251,6 +251,55 @@ Two rules keep snapshots honest:
|
||||||
self-contained **read-only** store with the full query surface, including
|
self-contained **read-only** store with the full query surface, including
|
||||||
vector search.
|
vector search.
|
||||||
|
|
||||||
|
## Reserved fields
|
||||||
|
|
||||||
|
Some field names belong to Brainy, not to your metadata. They live at **top
|
||||||
|
level** on every entity and relationship, have dedicated write paths, and may
|
||||||
|
never appear inside a `metadata` bag:
|
||||||
|
|
||||||
|
| Entities (nouns) | Relationships (verbs) | Canonical write path |
|
||||||
|
|---|---|---|
|
||||||
|
| `noun` | `verb` | the `type` param of `add()` / `relate()` |
|
||||||
|
| `subtype` | `subtype` | the `subtype` param |
|
||||||
|
| `confidence` | `confidence` | the `confidence` param |
|
||||||
|
| `weight` | `weight` | the `weight` param |
|
||||||
|
| `service` | `service` | the `service` param (fixed at create time) |
|
||||||
|
| `data` | `data` | the `data` param |
|
||||||
|
| `createdBy` | `createdBy` | the `createdBy` param of `add()` (system-managed on verbs) |
|
||||||
|
| `createdAt`, `updatedAt`, `_rev` | `createdAt`, `updatedAt`, `_rev` | system-managed (`ifRev` for CAS) |
|
||||||
|
|
||||||
|
The canonical machine-readable lists are exported as
|
||||||
|
`RESERVED_ENTITY_FIELDS` and `RESERVED_RELATION_FIELDS` (defined in
|
||||||
|
`src/types/reservedFields.ts`, the single source of truth). Three layers
|
||||||
|
enforce the contract:
|
||||||
|
|
||||||
|
1. **Compile time** — every `metadata` param (`add`, `update`, `relate`,
|
||||||
|
`updateRelation`, and the matching `transact()` operations) rejects a
|
||||||
|
literal reserved key as a TypeScript error.
|
||||||
|
2. **Write time** — untyped (JavaScript) callers that pass one anyway are
|
||||||
|
normalized: user-settable fields (`confidence`, `weight`, `subtype`, and
|
||||||
|
`service`/`createdBy` at create time) are remapped to their dedicated
|
||||||
|
param — **top-level wins** when both are supplied — and system-managed
|
||||||
|
fields are dropped with a one-shot warning naming the correct write path.
|
||||||
|
`update({ metadata: { confidence: 0.9 } })` therefore behaves exactly
|
||||||
|
like `update({ confidence: 0.9 })`.
|
||||||
|
3. **Read time** — every read path (`get`, `find`, `search`,
|
||||||
|
`getRelations`, batch reads, and historical `asOf()` materialization)
|
||||||
|
surfaces reserved fields **only at top level**: `entity.metadata` and
|
||||||
|
`relation.metadata` contain only your custom fields, always.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const id = await brain.add({
|
||||||
|
type: 'document', subtype: 'invoice',
|
||||||
|
data: 'Invoice #42', confidence: 0.95, // reserved → top-level params
|
||||||
|
metadata: { customer: 'acme', total: 129.5 } // custom fields only
|
||||||
|
})
|
||||||
|
|
||||||
|
const entity = await brain.get(id)
|
||||||
|
entity.confidence // 0.95 — top level
|
||||||
|
entity.metadata // { customer: 'acme', total: 129.5 }
|
||||||
|
```
|
||||||
|
|
||||||
## What is not guaranteed
|
## What is not guaranteed
|
||||||
|
|
||||||
Stated plainly, so nothing surprises you in production:
|
Stated plainly, so nothing surprises you in production:
|
||||||
|
|
|
||||||
443
src/brainy.ts
443
src/brainy.ts
|
|
@ -100,6 +100,10 @@ import {
|
||||||
FillSubtypesResult
|
FillSubtypesResult
|
||||||
} from './types/brainy.types.js'
|
} from './types/brainy.types.js'
|
||||||
import { NounType, VerbType, TypeUtils } from './types/graphTypes.js'
|
import { NounType, VerbType, TypeUtils } from './types/graphTypes.js'
|
||||||
|
import {
|
||||||
|
splitNounMetadataRecord,
|
||||||
|
splitVerbMetadataRecord
|
||||||
|
} from './types/reservedFields.js'
|
||||||
import { BrainyInterface } from './types/brainyInterface.js'
|
import { BrainyInterface } from './types/brainyInterface.js'
|
||||||
import type { IntegrationHub } from './integrations/core/IntegrationHub.js'
|
import type { IntegrationHub } from './integrations/core/IntegrationHub.js'
|
||||||
import { MigrationRunner } from './migration/MigrationRunner.js'
|
import { MigrationRunner } from './migration/MigrationRunner.js'
|
||||||
|
|
@ -1163,6 +1167,13 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
// Zero-config validation (static import for performance)
|
// Zero-config validation (static import for performance)
|
||||||
validateAddParams(params)
|
validateAddParams(params)
|
||||||
|
|
||||||
|
// Reserved fields arriving via the metadata bag (untyped callers — the
|
||||||
|
// compile-time guard stops TypeScript callers) are normalized to their
|
||||||
|
// canonical top-level location BEFORE any enforcement runs, so a
|
||||||
|
// remapped subtype participates in subtype-pairing enforcement and the
|
||||||
|
// indexed metadata bag carries only custom fields.
|
||||||
|
params = this.remapReservedAddMetadata(params)
|
||||||
|
|
||||||
// Tracked-field vocabulary enforcement (Layer 2). Walks both bags so a
|
// Tracked-field vocabulary enforcement (Layer 2). Walks both bags so a
|
||||||
// tracked field declared at top level (e.g. 'subtype') and one declared in
|
// tracked field declared at top level (e.g. 'subtype') and one declared in
|
||||||
// metadata (e.g. 'status') both validate.
|
// metadata (e.g. 'status') both validate.
|
||||||
|
|
@ -1574,34 +1585,291 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
// Metadata-only entity (no vector loading)
|
// Metadata-only entity (no vector loading)
|
||||||
// This is 76-81% faster for operations that don't need semantic similarity
|
// This is 76-81% faster for operations that don't need semantic similarity
|
||||||
|
|
||||||
// Extract standard fields, rest are custom metadata
|
// Canonical reserved/custom split — same single source of truth as every
|
||||||
// Same destructuring as baseStorage.getNoun() to ensure consistency
|
// other read path (see src/types/reservedFields.ts).
|
||||||
const { noun, subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, _rev, ...customMetadata } = metadata
|
const { reserved, custom } = splitNounMetadataRecord(metadata)
|
||||||
|
|
||||||
const entity: Entity<T> = {
|
const entity: Entity<T> = {
|
||||||
id,
|
id,
|
||||||
vector: [], // Stub vector (empty array - vectors not loaded for metadata-only)
|
vector: [], // Stub vector (empty array - vectors not loaded for metadata-only)
|
||||||
type: noun as NounType || NounType.Thing,
|
type: (reserved.noun as NounType) || NounType.Thing,
|
||||||
subtype,
|
subtype: reserved.subtype as string | undefined,
|
||||||
|
|
||||||
// Standard fields from metadata
|
// Standard fields from metadata
|
||||||
confidence,
|
confidence: reserved.confidence as number | undefined,
|
||||||
weight,
|
weight: reserved.weight as number | undefined,
|
||||||
createdAt: createdAt || Date.now(),
|
createdAt: (reserved.createdAt as number) || Date.now(),
|
||||||
updatedAt: updatedAt || Date.now(),
|
updatedAt: (reserved.updatedAt as number) || Date.now(),
|
||||||
service,
|
service: reserved.service as string | undefined,
|
||||||
data,
|
data: reserved.data,
|
||||||
createdBy,
|
createdBy: reserved.createdBy as Entity<T>['createdBy'],
|
||||||
// 7.31.0 — surface revision counter (defaults to 1 for pre-7.31.0 entities)
|
// 7.31.0 — surface revision counter (defaults to 1 for pre-7.31.0 entities)
|
||||||
_rev: typeof _rev === 'number' ? _rev : 1,
|
_rev: typeof reserved._rev === 'number' ? reserved._rev : 1,
|
||||||
|
|
||||||
// Custom user fields (standard fields removed, only custom remain)
|
// Custom user fields (standard fields removed, only custom remain)
|
||||||
metadata: customMetadata as T
|
metadata: custom as T
|
||||||
}
|
}
|
||||||
|
|
||||||
return entity
|
return entity
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** One-shot registry for reserved-field drop warnings (per process, per method+field). */
|
||||||
|
private static warnedReservedFields = new Set<string>()
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Warn once per process (per method+field) that a reserved
|
||||||
|
* field arrived inside a metadata bag and was dropped, naming the correct
|
||||||
|
* write path. Only dropped (system-managed) fields warn — user-settable
|
||||||
|
* fields are remapped to their dedicated param and honored silently.
|
||||||
|
* @param method - The public write method the bag arrived through.
|
||||||
|
* @param field - The reserved field name that was dropped.
|
||||||
|
* @param rightPath - Human guidance naming the correct write path.
|
||||||
|
*/
|
||||||
|
private warnDroppedReservedField(method: string, field: string, rightPath: string): void {
|
||||||
|
const key = `${method}:${field}`
|
||||||
|
if (Brainy.warnedReservedFields.has(key)) return
|
||||||
|
Brainy.warnedReservedFields.add(key)
|
||||||
|
prodLog.warn(
|
||||||
|
`[brainy] ${method}(): '${field}' is a reserved field and cannot be set ` +
|
||||||
|
`through the metadata bag — use ${rightPath}. The value was ignored. ` +
|
||||||
|
`(This warning is shown once per field per process.)`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Emit the one-shot drop warnings for system-managed entity
|
||||||
|
* fields found in a metadata bag. Shared by the `add` and `update` remaps
|
||||||
|
* (live calls and their `transact()` mirrors).
|
||||||
|
* @param method - `'add'` or `'update'` (the contract is identical for the
|
||||||
|
* matching `transact()` operation).
|
||||||
|
* @param reserved - The reserved half of the split metadata bag.
|
||||||
|
*/
|
||||||
|
private warnDroppedReservedEntityFields(
|
||||||
|
method: 'add' | 'update',
|
||||||
|
reserved: Partial<Record<string, unknown>>
|
||||||
|
): void {
|
||||||
|
if (reserved.noun !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, 'noun', "the top-level 'type' param")
|
||||||
|
}
|
||||||
|
if (reserved.data !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, 'data', "the top-level 'data' param")
|
||||||
|
}
|
||||||
|
if (reserved.createdAt !== undefined) {
|
||||||
|
this.warnDroppedReservedField(
|
||||||
|
method,
|
||||||
|
'createdAt',
|
||||||
|
method === 'add'
|
||||||
|
? 'nothing — creation time is set automatically'
|
||||||
|
: 'nothing — creation time is immutable'
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (reserved.updatedAt !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, 'updatedAt', 'nothing — set automatically on every write')
|
||||||
|
}
|
||||||
|
if (reserved._rev !== undefined) {
|
||||||
|
this.warnDroppedReservedField(
|
||||||
|
method,
|
||||||
|
'_rev',
|
||||||
|
method === 'update'
|
||||||
|
? "the 'ifRev' param for optimistic concurrency"
|
||||||
|
: 'nothing — revisions are system-managed'
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (method === 'update') {
|
||||||
|
if (reserved.service !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, 'service', 'nothing — fixed at add() time')
|
||||||
|
}
|
||||||
|
if (reserved.createdBy !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, 'createdBy', 'nothing — fixed at add() time')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Normalize an `add()` params object with respect to
|
||||||
|
* Brainy-reserved fields arriving inside `metadata` (untyped callers only —
|
||||||
|
* the compile-time guard on `AddParams.metadata` stops TypeScript callers).
|
||||||
|
* Fields with a dedicated `add()` param (`confidence`, `weight`, `subtype`,
|
||||||
|
* `service`, `createdBy`) are remapped to that param unless the caller also
|
||||||
|
* passed it explicitly (top-level wins); system-managed fields (`noun`,
|
||||||
|
* `data`, `createdAt`, `updatedAt`, `_rev`) are dropped with a one-shot
|
||||||
|
* warning naming the correct write path. The remapped `subtype` flows
|
||||||
|
* through subtype-pairing enforcement exactly like a top-level one.
|
||||||
|
* @param params - The caller's add params (not mutated).
|
||||||
|
* @returns Params with reserved fields normalized out of `metadata`.
|
||||||
|
*/
|
||||||
|
private remapReservedAddMetadata(params: AddParams<T>): AddParams<T> {
|
||||||
|
const bag = params.metadata as Record<string, unknown> | undefined
|
||||||
|
if (!bag || typeof bag !== 'object') return params
|
||||||
|
const { reserved, custom } = splitNounMetadataRecord(bag)
|
||||||
|
if (Object.keys(reserved).length === 0) return params
|
||||||
|
|
||||||
|
this.warnDroppedReservedEntityFields('add', reserved)
|
||||||
|
|
||||||
|
const createdBy = reserved.createdBy as { augmentation?: unknown; version?: unknown } | undefined
|
||||||
|
const createdByValid =
|
||||||
|
typeof createdBy === 'object' &&
|
||||||
|
createdBy !== null &&
|
||||||
|
typeof createdBy.augmentation === 'string' &&
|
||||||
|
typeof createdBy.version === 'string'
|
||||||
|
|
||||||
|
return {
|
||||||
|
...params,
|
||||||
|
metadata: custom as AddParams<T>['metadata'],
|
||||||
|
...(params.confidence === undefined &&
|
||||||
|
typeof reserved.confidence === 'number' && { confidence: reserved.confidence }),
|
||||||
|
...(params.weight === undefined &&
|
||||||
|
typeof reserved.weight === 'number' && { weight: reserved.weight }),
|
||||||
|
...(params.subtype === undefined &&
|
||||||
|
typeof reserved.subtype === 'string' && { subtype: reserved.subtype }),
|
||||||
|
...(params.service === undefined &&
|
||||||
|
typeof reserved.service === 'string' && { service: reserved.service }),
|
||||||
|
...(params.createdBy === undefined &&
|
||||||
|
createdByValid && { createdBy: createdBy as { augmentation: string; version: string } })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Normalize an `update()` params object with respect to
|
||||||
|
* Brainy-reserved fields arriving inside the metadata patch — the `update()`
|
||||||
|
* mirror of {@link remapReservedAddMetadata}, closing the historical trap
|
||||||
|
* where `add({metadata:{confidence}})` lifted the field but
|
||||||
|
* `update({metadata:{confidence}})` silently dropped it (the patch value
|
||||||
|
* survived the merge and was then clobbered by the preserve-existing
|
||||||
|
* spread; a production consumer's confidence-evolution writes no-oped until
|
||||||
|
* read back). User-mutable fields (`confidence`, `weight`, `subtype`)
|
||||||
|
* remap to their dedicated param unless the caller also passed it
|
||||||
|
* (top-level wins); everything else (`noun`, `data`, `createdAt`,
|
||||||
|
* `updatedAt`, `service`, `createdBy`, `_rev`) is system-managed or fixed
|
||||||
|
* at `add()` time and is dropped with a one-shot warning.
|
||||||
|
* @param params - The caller's update params (not mutated).
|
||||||
|
* @returns Params with reserved fields normalized out of `metadata`.
|
||||||
|
*/
|
||||||
|
private remapReservedUpdateMetadata(params: UpdateParams<T>): UpdateParams<T> {
|
||||||
|
const bag = params.metadata as Record<string, unknown> | undefined
|
||||||
|
if (!bag || typeof bag !== 'object') return params
|
||||||
|
const { reserved, custom } = splitNounMetadataRecord(bag)
|
||||||
|
if (Object.keys(reserved).length === 0) return params
|
||||||
|
|
||||||
|
this.warnDroppedReservedEntityFields('update', reserved)
|
||||||
|
|
||||||
|
return {
|
||||||
|
...params,
|
||||||
|
metadata: custom as UpdateParams<T>['metadata'],
|
||||||
|
...(params.confidence === undefined &&
|
||||||
|
typeof reserved.confidence === 'number' && { confidence: reserved.confidence }),
|
||||||
|
...(params.weight === undefined &&
|
||||||
|
typeof reserved.weight === 'number' && { weight: reserved.weight }),
|
||||||
|
...(params.subtype === undefined &&
|
||||||
|
typeof reserved.subtype === 'string' && { subtype: reserved.subtype })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Emit the one-shot drop warnings for system-managed
|
||||||
|
* relationship fields found in a metadata bag. Shared by the `relate` and
|
||||||
|
* `updateRelation` remaps (live calls and the `transact()` relate mirror).
|
||||||
|
* @param method - `'relate'` or `'updateRelation'`.
|
||||||
|
* @param reserved - The reserved half of the split metadata bag.
|
||||||
|
*/
|
||||||
|
private warnDroppedReservedRelationFields(
|
||||||
|
method: 'relate' | 'updateRelation',
|
||||||
|
reserved: Partial<Record<string, unknown>>
|
||||||
|
): void {
|
||||||
|
if (reserved.verb !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, 'verb', "the top-level 'type' param")
|
||||||
|
}
|
||||||
|
if (reserved.data !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, 'data', "the top-level 'data' param")
|
||||||
|
}
|
||||||
|
if (reserved.createdAt !== undefined) {
|
||||||
|
this.warnDroppedReservedField(
|
||||||
|
method,
|
||||||
|
'createdAt',
|
||||||
|
method === 'relate'
|
||||||
|
? 'nothing — creation time is set automatically'
|
||||||
|
: 'nothing — creation time is immutable'
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (reserved.updatedAt !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, 'updatedAt', 'nothing — set automatically on every write')
|
||||||
|
}
|
||||||
|
if (reserved.createdBy !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, 'createdBy', 'nothing — system-managed')
|
||||||
|
}
|
||||||
|
if (reserved._rev !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, '_rev', 'nothing — system-managed')
|
||||||
|
}
|
||||||
|
if (method === 'updateRelation' && reserved.service !== undefined) {
|
||||||
|
this.warnDroppedReservedField(method, 'service', 'nothing — fixed at relate() time')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Normalize a `relate()` params object with respect to
|
||||||
|
* Brainy-reserved fields arriving inside `metadata` — the relationship
|
||||||
|
* mirror of {@link remapReservedAddMetadata}. Fields with a dedicated
|
||||||
|
* `relate()` param (`confidence`, `weight`, `subtype`, `service`) remap to
|
||||||
|
* that param (top-level wins); system-managed fields (`verb`, `data`,
|
||||||
|
* `createdAt`, `updatedAt`, `createdBy`, `_rev`) are dropped with a
|
||||||
|
* one-shot warning.
|
||||||
|
* @param params - The caller's relate params (not mutated).
|
||||||
|
* @returns Params with reserved fields normalized out of `metadata`.
|
||||||
|
*/
|
||||||
|
private remapReservedRelateMetadata(params: RelateParams<T>): RelateParams<T> {
|
||||||
|
const bag = params.metadata as Record<string, unknown> | undefined
|
||||||
|
if (!bag || typeof bag !== 'object') return params
|
||||||
|
const { reserved, custom } = splitVerbMetadataRecord(bag)
|
||||||
|
if (Object.keys(reserved).length === 0) return params
|
||||||
|
|
||||||
|
this.warnDroppedReservedRelationFields('relate', reserved)
|
||||||
|
|
||||||
|
return {
|
||||||
|
...params,
|
||||||
|
metadata: custom as RelateParams<T>['metadata'],
|
||||||
|
...(params.confidence === undefined &&
|
||||||
|
typeof reserved.confidence === 'number' && { confidence: reserved.confidence }),
|
||||||
|
...(params.weight === undefined &&
|
||||||
|
typeof reserved.weight === 'number' && { weight: reserved.weight }),
|
||||||
|
...(params.subtype === undefined &&
|
||||||
|
typeof reserved.subtype === 'string' && { subtype: reserved.subtype }),
|
||||||
|
...(params.service === undefined &&
|
||||||
|
typeof reserved.service === 'string' && { service: reserved.service })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Normalize an `updateRelation()` params object with respect
|
||||||
|
* to Brainy-reserved fields arriving inside the metadata patch — the
|
||||||
|
* relationship mirror of {@link remapReservedUpdateMetadata}. User-mutable
|
||||||
|
* fields (`confidence`, `weight`, `subtype`) remap to their dedicated
|
||||||
|
* param (top-level wins); everything else is dropped with a one-shot
|
||||||
|
* warning.
|
||||||
|
* @param params - The caller's update-relation params (not mutated).
|
||||||
|
* @returns Params with reserved fields normalized out of `metadata`.
|
||||||
|
*/
|
||||||
|
private remapReservedUpdateRelationMetadata(
|
||||||
|
params: UpdateRelationParams<T>
|
||||||
|
): UpdateRelationParams<T> {
|
||||||
|
const bag = params.metadata as Record<string, unknown> | undefined
|
||||||
|
if (!bag || typeof bag !== 'object') return params
|
||||||
|
const { reserved, custom } = splitVerbMetadataRecord(bag)
|
||||||
|
if (Object.keys(reserved).length === 0) return params
|
||||||
|
|
||||||
|
this.warnDroppedReservedRelationFields('updateRelation', reserved)
|
||||||
|
|
||||||
|
return {
|
||||||
|
...params,
|
||||||
|
metadata: custom as UpdateRelationParams<T>['metadata'],
|
||||||
|
...(params.confidence === undefined &&
|
||||||
|
typeof reserved.confidence === 'number' && { confidence: reserved.confidence }),
|
||||||
|
...(params.weight === undefined &&
|
||||||
|
typeof reserved.weight === 'number' && { weight: reserved.weight }),
|
||||||
|
...(params.subtype === undefined &&
|
||||||
|
typeof reserved.subtype === 'string' && { subtype: reserved.subtype })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Update an existing entity
|
* Update an existing entity
|
||||||
*
|
*
|
||||||
|
|
@ -1659,6 +1927,16 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
// Zero-config validation (static import for performance)
|
// Zero-config validation (static import for performance)
|
||||||
validateUpdateParams(params)
|
validateUpdateParams(params)
|
||||||
|
|
||||||
|
// Reserved fields arriving via the metadata patch are remapped to their
|
||||||
|
// canonical top-level location, mirroring add()'s lift. Without this the
|
||||||
|
// patch value survived the merge but was then clobbered by the
|
||||||
|
// preserve-existing spreads below — a silent no-op consumers could only
|
||||||
|
// detect by reading values back. User-mutable fields (confidence,
|
||||||
|
// weight, subtype) remap unless the same field was also passed top-level
|
||||||
|
// (top-level wins); system-managed fields are dropped with a one-shot
|
||||||
|
// warning naming the right path.
|
||||||
|
params = this.remapReservedUpdateMetadata(params)
|
||||||
|
|
||||||
// Tracked-field vocabulary enforcement (Layer 2). Same as add() — the
|
// Tracked-field vocabulary enforcement (Layer 2). Same as add() — the
|
||||||
// metadata bag carries fields registered via trackField(), and subtype is
|
// metadata bag carries fields registered via trackField(), and subtype is
|
||||||
// a tracked top-level candidate.
|
// a tracked top-level candidate.
|
||||||
|
|
@ -1690,10 +1968,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
}
|
}
|
||||||
|
|
||||||
// ifRev (7.31.0) — optimistic concurrency. If caller supplied ifRev, the persisted
|
// ifRev (7.31.0) — optimistic concurrency. If caller supplied ifRev, the persisted
|
||||||
// _rev must match exactly. Pre-7.31.0 entities without _rev are treated as rev 1.
|
// _rev must match exactly. `_rev` is reserved: every read path surfaces it ONLY
|
||||||
const currentRev = typeof (existing.metadata as any)?._rev === 'number'
|
// top-level (entities without one are read as rev 1), so that is the one place
|
||||||
? (existing.metadata as any)._rev
|
// to look.
|
||||||
: (typeof (existing as any)._rev === 'number' ? (existing as any)._rev : 1)
|
const currentRev = typeof existing._rev === 'number' ? existing._rev : 1
|
||||||
if (typeof params.ifRev === 'number' && params.ifRev !== currentRev) {
|
if (typeof params.ifRev === 'number' && params.ifRev !== currentRev) {
|
||||||
throw new RevisionConflictError(params.id, params.ifRev, currentRev)
|
throw new RevisionConflictError(params.id, params.ifRev, currentRev)
|
||||||
}
|
}
|
||||||
|
|
@ -1900,13 +2178,15 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
|
|
||||||
// Aggregation hook (outside transaction — derived data)
|
// Aggregation hook (outside transaction — derived data)
|
||||||
if (this._aggregationIndex && metadata) {
|
if (this._aggregationIndex && metadata) {
|
||||||
// Reconstruct entity-like object from stored metadata
|
// Reconstruct entity-like object from stored metadata via the
|
||||||
const { noun, createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadata
|
// canonical reserved/custom split (the hand-rolled destructure here
|
||||||
|
// missed subtype/_rev, leaking them into the aggregation view).
|
||||||
|
const { reserved, custom } = splitNounMetadataRecord(metadata)
|
||||||
const entityForAgg = {
|
const entityForAgg = {
|
||||||
type: noun,
|
type: reserved.noun,
|
||||||
service,
|
service: reserved.service,
|
||||||
data,
|
data: reserved.data,
|
||||||
metadata: customMetadata
|
metadata: custom
|
||||||
}
|
}
|
||||||
this._aggregationIndex.onEntityDeleted(id, entityForAgg)
|
this._aggregationIndex.onEntityDeleted(id, entityForAgg)
|
||||||
}
|
}
|
||||||
|
|
@ -2168,6 +2448,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
// Zero-config validation (static import for performance)
|
// Zero-config validation (static import for performance)
|
||||||
validateRelateParams(params)
|
validateRelateParams(params)
|
||||||
|
|
||||||
|
// Reserved fields arriving via the metadata bag are normalized to their
|
||||||
|
// canonical top-level params before enforcement — mirror of add()'s lift.
|
||||||
|
params = this.remapReservedRelateMetadata(params)
|
||||||
|
|
||||||
// Subtype pairing enforcement (Layer 3 — 7.30.0). Per-type rules registered
|
// Subtype pairing enforcement (Layer 3 — 7.30.0). Per-type rules registered
|
||||||
// via brain.requireSubtype() compose with the brain-wide strict-mode flag.
|
// via brain.requireSubtype() compose with the brain-wide strict-mode flag.
|
||||||
// Metadata is passed so infrastructure edges (VFS containment) can bypass
|
// Metadata is passed so infrastructure edges (VFS containment) can bypass
|
||||||
|
|
@ -2225,8 +2509,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
verb: params.type,
|
verb: params.type,
|
||||||
...(params.subtype !== undefined && { subtype: params.subtype }),
|
...(params.subtype !== undefined && { subtype: params.subtype }),
|
||||||
weight: params.weight ?? 1.0,
|
weight: params.weight ?? 1.0,
|
||||||
|
...(params.confidence !== undefined && { confidence: params.confidence }),
|
||||||
|
...(params.service !== undefined && { service: params.service }),
|
||||||
createdAt: Date.now(),
|
createdAt: Date.now(),
|
||||||
...((params as any).data !== undefined && { data: (params as any).data })
|
...(params.data !== undefined && { data: params.data })
|
||||||
}
|
}
|
||||||
|
|
||||||
// Save to storage (vector and metadata separately)
|
// Save to storage (vector and metadata separately)
|
||||||
|
|
@ -2239,8 +2525,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
type: params.type,
|
type: params.type,
|
||||||
...(params.subtype !== undefined && { subtype: params.subtype }),
|
...(params.subtype !== undefined && { subtype: params.subtype }),
|
||||||
weight: params.weight ?? 1.0,
|
weight: params.weight ?? 1.0,
|
||||||
|
...(params.confidence !== undefined && { confidence: params.confidence }),
|
||||||
|
...(params.service !== undefined && { service: params.service }),
|
||||||
metadata: params.metadata,
|
metadata: params.metadata,
|
||||||
data: (params as any).data,
|
data: params.data,
|
||||||
createdAt: Date.now()
|
createdAt: Date.now()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -2392,6 +2680,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
|
|
||||||
validateUpdateRelationParams(params)
|
validateUpdateRelationParams(params)
|
||||||
|
|
||||||
|
// Reserved fields arriving via the metadata patch are remapped to their
|
||||||
|
// canonical top-level params — mirror of update()'s normalization.
|
||||||
|
params = this.remapReservedUpdateRelationMetadata(params)
|
||||||
|
|
||||||
const existing = await this.storage.getVerb(params.id)
|
const existing = await this.storage.getVerb(params.id)
|
||||||
if (!existing) {
|
if (!existing) {
|
||||||
throw new Error(`Relation ${params.id} not found`)
|
throw new Error(`Relation ${params.id} not found`)
|
||||||
|
|
@ -2428,6 +2720,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
...(params.confidence !== undefined
|
...(params.confidence !== undefined
|
||||||
? { confidence: params.confidence }
|
? { confidence: params.confidence }
|
||||||
: existingAny.confidence !== undefined && { confidence: existingAny.confidence }),
|
: existingAny.confidence !== undefined && { confidence: existingAny.confidence }),
|
||||||
|
// service/createdBy are fixed at relate() time — always carried forward
|
||||||
|
// (omitting them here silently erased them on every updateRelation()).
|
||||||
|
...(existingAny.service !== undefined && { service: existingAny.service }),
|
||||||
|
...(existingAny.createdBy !== undefined && { createdBy: existingAny.createdBy }),
|
||||||
createdAt: existingAny.createdAt,
|
createdAt: existingAny.createdAt,
|
||||||
updatedAt: Date.now(),
|
updatedAt: Date.now(),
|
||||||
...(params.data !== undefined
|
...(params.data !== undefined
|
||||||
|
|
@ -5030,31 +5326,25 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
const {
|
// Canonical reserved/custom split — same single source of truth as the
|
||||||
verb,
|
// live verb read paths (see src/types/reservedFields.ts).
|
||||||
subtype,
|
const { reserved, custom } = splitVerbMetadataRecord(
|
||||||
createdAt,
|
record.metadata as Record<string, unknown>
|
||||||
updatedAt,
|
)
|
||||||
confidence,
|
|
||||||
weight,
|
|
||||||
service,
|
|
||||||
data,
|
|
||||||
...customMetadata
|
|
||||||
} = record.metadata as Record<string, any>
|
|
||||||
|
|
||||||
return {
|
return {
|
||||||
id,
|
id,
|
||||||
from: core.sourceId,
|
from: core.sourceId,
|
||||||
to: core.targetId,
|
to: core.targetId,
|
||||||
type: (verb ?? core.verb) as VerbType,
|
type: (reserved.verb ?? core.verb) as VerbType,
|
||||||
...(subtype !== undefined && { subtype: subtype as string }),
|
...(reserved.subtype !== undefined && { subtype: reserved.subtype as string }),
|
||||||
weight: (weight as number) ?? 1.0,
|
weight: (reserved.weight as number) ?? 1.0,
|
||||||
data,
|
data: reserved.data,
|
||||||
metadata: customMetadata as T,
|
metadata: custom as T,
|
||||||
service: service as string,
|
service: reserved.service as string,
|
||||||
createdAt: typeof createdAt === 'number' ? createdAt : Date.now(),
|
createdAt: typeof reserved.createdAt === 'number' ? reserved.createdAt : Date.now(),
|
||||||
...(typeof updatedAt === 'number' && { updatedAt }),
|
...(typeof reserved.updatedAt === 'number' && { updatedAt: reserved.updatedAt }),
|
||||||
...(typeof confidence === 'number' && { confidence })
|
...(typeof reserved.confidence === 'number' && { confidence: reserved.confidence })
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -5318,8 +5608,12 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
state: TxPlanState,
|
state: TxPlanState,
|
||||||
plan: PlannedTransact
|
plan: PlannedTransact
|
||||||
): Promise<string> {
|
): Promise<string> {
|
||||||
const { op: _discriminator, ...params } = op
|
const { op: _discriminator, ...rawParams } = op
|
||||||
validateAddParams(params as AddParams<T>)
|
validateAddParams(rawParams as AddParams<T>)
|
||||||
|
// Same reserved-field normalization as add() — the metadata bag is
|
||||||
|
// cleaned BEFORE enforcement so a remapped subtype participates in
|
||||||
|
// subtype-pairing enforcement and only custom fields reach the index.
|
||||||
|
const params = this.remapReservedAddMetadata(rawParams as AddParams<T>)
|
||||||
this.enforceTrackedFieldValues(params.metadata as Record<string, unknown> | undefined, 'metadata')
|
this.enforceTrackedFieldValues(params.metadata as Record<string, unknown> | undefined, 'metadata')
|
||||||
this.enforceTrackedFieldValues({ subtype: params.subtype } as Record<string, unknown>, 'top-level')
|
this.enforceTrackedFieldValues({ subtype: params.subtype } as Record<string, unknown>, 'top-level')
|
||||||
this.enforceSubtypeOnAdd('add', params.type, params.subtype, params.metadata)
|
this.enforceSubtypeOnAdd('add', params.type, params.subtype, params.metadata)
|
||||||
|
|
@ -5404,8 +5698,12 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
state: TxPlanState,
|
state: TxPlanState,
|
||||||
plan: PlannedTransact
|
plan: PlannedTransact
|
||||||
): Promise<string> {
|
): Promise<string> {
|
||||||
const { op: _discriminator, ...params } = op
|
const { op: _discriminator, ...rawParams } = op
|
||||||
validateUpdateParams(params as UpdateParams<T>)
|
validateUpdateParams(rawParams as UpdateParams<T>)
|
||||||
|
// Same reserved-field normalization as update() — user-mutable fields
|
||||||
|
// remap to their dedicated param (top-level wins), system-managed fields
|
||||||
|
// drop with a one-shot warning.
|
||||||
|
const params = this.remapReservedUpdateMetadata(rawParams as UpdateParams<T>)
|
||||||
this.enforceTrackedFieldValues(params.metadata as Record<string, unknown> | undefined, 'metadata')
|
this.enforceTrackedFieldValues(params.metadata as Record<string, unknown> | undefined, 'metadata')
|
||||||
if (params.subtype !== undefined) {
|
if (params.subtype !== undefined) {
|
||||||
this.enforceTrackedFieldValues({ subtype: params.subtype } as Record<string, unknown>, 'top-level')
|
this.enforceTrackedFieldValues({ subtype: params.subtype } as Record<string, unknown>, 'top-level')
|
||||||
|
|
@ -5424,13 +5722,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
}
|
}
|
||||||
|
|
||||||
// ifRev CAS — identical resolution to update(); a conflict rejects the
|
// ifRev CAS — identical resolution to update(); a conflict rejects the
|
||||||
// WHOLE batch before anything is staged or applied.
|
// WHOLE batch before anything is staged or applied. `_rev` is reserved:
|
||||||
const currentRev =
|
// every read path surfaces it ONLY top-level.
|
||||||
typeof (existing.metadata as any)?._rev === 'number'
|
const currentRev = typeof existing._rev === 'number' ? existing._rev : 1
|
||||||
? (existing.metadata as any)._rev
|
|
||||||
: typeof existing._rev === 'number'
|
|
||||||
? existing._rev
|
|
||||||
: 1
|
|
||||||
if (typeof params.ifRev === 'number' && params.ifRev !== currentRev) {
|
if (typeof params.ifRev === 'number' && params.ifRev !== currentRev) {
|
||||||
throw new RevisionConflictError(params.id, params.ifRev, currentRev)
|
throw new RevisionConflictError(params.id, params.ifRev, currentRev)
|
||||||
}
|
}
|
||||||
|
|
@ -5601,8 +5895,14 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
plan.touchedNouns.push(id)
|
plan.touchedNouns.push(id)
|
||||||
|
|
||||||
if (metadata) {
|
if (metadata) {
|
||||||
const { noun: nounType, createdAt, updatedAt, confidence, weight, service, data, createdBy, ...customMetadata } = metadata
|
// Canonical reserved/custom split — mirror of delete()'s aggregation hook.
|
||||||
const entityForAgg = { type: nounType, service, data, metadata: customMetadata }
|
const { reserved, custom } = splitNounMetadataRecord(metadata)
|
||||||
|
const entityForAgg = {
|
||||||
|
type: reserved.noun,
|
||||||
|
service: reserved.service,
|
||||||
|
data: reserved.data,
|
||||||
|
metadata: custom
|
||||||
|
}
|
||||||
plan.postCommit.push(() => {
|
plan.postCommit.push(() => {
|
||||||
if (this._aggregationIndex) {
|
if (this._aggregationIndex) {
|
||||||
this._aggregationIndex.onEntityDeleted(id, entityForAgg)
|
this._aggregationIndex.onEntityDeleted(id, entityForAgg)
|
||||||
|
|
@ -5626,8 +5926,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
state: TxPlanState,
|
state: TxPlanState,
|
||||||
plan: PlannedTransact
|
plan: PlannedTransact
|
||||||
): Promise<string> {
|
): Promise<string> {
|
||||||
const { op: _discriminator, ...params } = op
|
const { op: _discriminator, ...rawParams } = op
|
||||||
validateRelateParams(params as RelateParams<T>)
|
validateRelateParams(rawParams as RelateParams<T>)
|
||||||
|
// Same reserved-field normalization as relate().
|
||||||
|
const params = this.remapReservedRelateMetadata(rawParams as RelateParams<T>)
|
||||||
this.enforceSubtypeOnRelate('relate', params.type, params.subtype, params.metadata)
|
this.enforceSubtypeOnRelate('relate', params.type, params.subtype, params.metadata)
|
||||||
|
|
||||||
const fromEntity = await this.planGetEntity(state, params.from)
|
const fromEntity = await this.planGetEntity(state, params.from)
|
||||||
|
|
@ -5677,6 +5979,8 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
verb: params.type,
|
verb: params.type,
|
||||||
...(params.subtype !== undefined && { subtype: params.subtype }),
|
...(params.subtype !== undefined && { subtype: params.subtype }),
|
||||||
weight: params.weight ?? 1.0,
|
weight: params.weight ?? 1.0,
|
||||||
|
...(params.confidence !== undefined && { confidence: params.confidence }),
|
||||||
|
...(params.service !== undefined && { service: params.service }),
|
||||||
createdAt: now,
|
createdAt: now,
|
||||||
...(params.data !== undefined && { data: params.data })
|
...(params.data !== undefined && { data: params.data })
|
||||||
}
|
}
|
||||||
|
|
@ -5689,6 +5993,8 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
type: params.type,
|
type: params.type,
|
||||||
...(params.subtype !== undefined && { subtype: params.subtype }),
|
...(params.subtype !== undefined && { subtype: params.subtype }),
|
||||||
weight: params.weight ?? 1.0,
|
weight: params.weight ?? 1.0,
|
||||||
|
...(params.confidence !== undefined && { confidence: params.confidence }),
|
||||||
|
...(params.service !== undefined && { service: params.service }),
|
||||||
metadata: params.metadata,
|
metadata: params.metadata,
|
||||||
data: params.data,
|
data: params.data,
|
||||||
createdAt: now
|
createdAt: now
|
||||||
|
|
@ -8238,7 +8544,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
if (!readBoth && from.kind === 'metadata') {
|
if (!readBoth && from.kind === 'metadata') {
|
||||||
delete nextMeta[from.field]
|
delete nextMeta[from.field]
|
||||||
}
|
}
|
||||||
update.metadata = nextMeta as T
|
update.metadata = nextMeta as UpdateParams<T>['metadata']
|
||||||
update.merge = false
|
update.merge = false
|
||||||
} else {
|
} else {
|
||||||
// data path — only meaningful when data is an object
|
// data path — only meaningful when data is an object
|
||||||
|
|
@ -8256,7 +8562,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
const base = (entity.metadata as unknown as Record<string, unknown>) ?? {}
|
const base = (entity.metadata as unknown as Record<string, unknown>) ?? {}
|
||||||
const nextMeta: Record<string, unknown> = { ...base }
|
const nextMeta: Record<string, unknown> = { ...base }
|
||||||
delete nextMeta[from.field]
|
delete nextMeta[from.field]
|
||||||
update.metadata = nextMeta as T
|
update.metadata = nextMeta as UpdateParams<T>['metadata']
|
||||||
update.merge = false
|
update.merge = false
|
||||||
} else if (from.kind === 'data' && to.kind !== 'data') {
|
} else if (from.kind === 'data' && to.kind !== 'data') {
|
||||||
const base = (entity.data && typeof entity.data === 'object') ? { ...(entity.data as Record<string, unknown>) } : null
|
const base = (entity.data && typeof entity.data === 'object') ? { ...(entity.data as Record<string, unknown>) } : null
|
||||||
|
|
@ -9560,7 +9866,12 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Convert verbs to relations (read from top-level)
|
* Convert storage-shape verbs to public `Relation`s. The storage layer has
|
||||||
|
* already done the canonical reserved/custom split (every combine goes
|
||||||
|
* through `hydrateVerbWithMetadata` — see src/types/reservedFields.ts), so
|
||||||
|
* this is a pure top-level field mapping: `metadata` carries ONLY custom
|
||||||
|
* fields, and every reserved field (`subtype`, `weight`, `confidence`,
|
||||||
|
* timestamps, `service`, `data`) surfaces at top level.
|
||||||
*/
|
*/
|
||||||
private verbsToRelations(verbs: GraphVerb[]): Relation<T>[] {
|
private verbsToRelations(verbs: GraphVerb[]): Relation<T>[] {
|
||||||
return verbs.map((v) => {
|
return verbs.map((v) => {
|
||||||
|
|
@ -9572,10 +9883,12 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
type: (v.verb || v.type) as VerbType,
|
type: (v.verb || v.type) as VerbType,
|
||||||
...(va.subtype !== undefined && { subtype: va.subtype as string }),
|
...(va.subtype !== undefined && { subtype: va.subtype as string }),
|
||||||
weight: v.weight ?? 1.0,
|
weight: v.weight ?? 1.0,
|
||||||
|
...(typeof va.confidence === 'number' && { confidence: va.confidence as number }),
|
||||||
data: v.data,
|
data: v.data,
|
||||||
metadata: v.metadata,
|
metadata: v.metadata,
|
||||||
service: v.service as string,
|
service: v.service as string,
|
||||||
createdAt: typeof v.createdAt === 'number' ? v.createdAt : Date.now()
|
createdAt: typeof v.createdAt === 'number' ? v.createdAt : Date.now(),
|
||||||
|
...(typeof va.updatedAt === 'number' && { updatedAt: va.updatedAt as number })
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -324,16 +324,16 @@ export const importCommands = {
|
||||||
|
|
||||||
const extractedEntities = await brain.extract(text)
|
const extractedEntities = await brain.extract(text)
|
||||||
|
|
||||||
// Add extracted entities
|
// Add extracted entities (ExtractedEntity is fully typed —
|
||||||
|
// text/type/confidence come straight off the extraction result)
|
||||||
for (const extracted of extractedEntities) {
|
for (const extracted of extractedEntities) {
|
||||||
const type = (extracted as any).type || NounType.Thing
|
|
||||||
await brain.add({
|
await brain.add({
|
||||||
data: (extracted as any).content || (extracted as any).text,
|
data: extracted.text,
|
||||||
type: type,
|
type: extracted.type || NounType.Thing,
|
||||||
|
confidence: extracted.confidence, // reserved field — dedicated param, not metadata
|
||||||
metadata: {
|
metadata: {
|
||||||
extractedFrom: entity.id,
|
extractedFrom: entity.id,
|
||||||
extractionMethod: 'nlp',
|
extractionMethod: 'nlp'
|
||||||
confidence: (extracted as any).confidence
|
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
entitiesExtracted++
|
entitiesExtracted++
|
||||||
|
|
|
||||||
84
src/db/db.ts
84
src/db/db.ts
|
|
@ -56,6 +56,10 @@ import type {
|
||||||
Relation,
|
Relation,
|
||||||
Result
|
Result
|
||||||
} from '../types/brainy.types.js'
|
} from '../types/brainy.types.js'
|
||||||
|
import {
|
||||||
|
splitNounMetadataRecord,
|
||||||
|
splitVerbMetadataRecord
|
||||||
|
} from '../types/reservedFields.js'
|
||||||
import { v4 as uuidv4 } from '../universal/uuid.js'
|
import { v4 as uuidv4 } from '../universal/uuid.js'
|
||||||
import { SpeculativeOverlayError } from './errors.js'
|
import { SpeculativeOverlayError } from './errors.js'
|
||||||
import type { GenerationStore } from './generationStore.js'
|
import type { GenerationStore } from './generationStore.js'
|
||||||
|
|
@ -544,20 +548,38 @@ export class Db<T = any> {
|
||||||
for (const op of ops) {
|
for (const op of ops) {
|
||||||
switch (op.op) {
|
switch (op.op) {
|
||||||
case 'add': {
|
case 'add': {
|
||||||
|
// Reserved-field normalization — mirror of the brain.transact()
|
||||||
|
// write path: user-settable fields lift to their dedicated field
|
||||||
|
// (top-level wins), system-managed fields drop, and the entity's
|
||||||
|
// metadata bag carries ONLY custom fields. Speculative views skip
|
||||||
|
// the one-shot warnings — committing the same ops through
|
||||||
|
// `brain.transact()` warns on the real write path.
|
||||||
|
const { reserved, custom } = splitNounMetadataRecord(
|
||||||
|
op.metadata as Record<string, unknown> | undefined
|
||||||
|
)
|
||||||
|
const confidence =
|
||||||
|
op.confidence ?? (typeof reserved.confidence === 'number' ? reserved.confidence : undefined)
|
||||||
|
const weight =
|
||||||
|
op.weight ?? (typeof reserved.weight === 'number' ? reserved.weight : undefined)
|
||||||
|
const subtype =
|
||||||
|
op.subtype ?? (typeof reserved.subtype === 'string' ? reserved.subtype : undefined)
|
||||||
|
const service =
|
||||||
|
op.service ?? (typeof reserved.service === 'string' ? reserved.service : undefined)
|
||||||
|
|
||||||
const id = op.id ?? uuidv4()
|
const id = op.id ?? uuidv4()
|
||||||
const now = Date.now()
|
const now = Date.now()
|
||||||
overlay.nouns.set(id, {
|
overlay.nouns.set(id, {
|
||||||
id,
|
id,
|
||||||
vector: op.vector ?? [],
|
vector: op.vector ?? [],
|
||||||
type: op.type,
|
type: op.type,
|
||||||
...(op.subtype !== undefined && { subtype: op.subtype }),
|
...(subtype !== undefined && { subtype }),
|
||||||
data: op.data,
|
data: op.data,
|
||||||
metadata: (op.metadata ?? {}) as T,
|
metadata: custom as T,
|
||||||
...(op.service !== undefined && { service: op.service }),
|
...(service !== undefined && { service }),
|
||||||
createdAt: now,
|
createdAt: now,
|
||||||
updatedAt: now,
|
updatedAt: now,
|
||||||
...(op.confidence !== undefined && { confidence: op.confidence }),
|
...(confidence !== undefined && { confidence }),
|
||||||
...(op.weight !== undefined && { weight: op.weight }),
|
...(weight !== undefined && { weight }),
|
||||||
_rev: 1
|
_rev: 1
|
||||||
})
|
})
|
||||||
break
|
break
|
||||||
|
|
@ -567,18 +589,28 @@ export class Db<T = any> {
|
||||||
if (!base) {
|
if (!base) {
|
||||||
throw new Error(`with(): entity ${op.id} not found at generation ${this.gen}`)
|
throw new Error(`with(): entity ${op.id} not found at generation ${this.gen}`)
|
||||||
}
|
}
|
||||||
|
// Same reserved-field normalization as the committed update path.
|
||||||
|
const { reserved, custom } = splitNounMetadataRecord(
|
||||||
|
op.metadata as Record<string, unknown> | undefined
|
||||||
|
)
|
||||||
|
const confidence =
|
||||||
|
op.confidence ?? (typeof reserved.confidence === 'number' ? reserved.confidence : undefined)
|
||||||
|
const weight =
|
||||||
|
op.weight ?? (typeof reserved.weight === 'number' ? reserved.weight : undefined)
|
||||||
|
const subtype =
|
||||||
|
op.subtype ?? (typeof reserved.subtype === 'string' ? reserved.subtype : undefined)
|
||||||
const mergedMetadata =
|
const mergedMetadata =
|
||||||
op.merge !== false
|
op.merge !== false
|
||||||
? ({ ...(base.metadata as object), ...(op.metadata as object) } as T)
|
? ({ ...(base.metadata as object), ...custom } as T)
|
||||||
: ((op.metadata ?? base.metadata) as T)
|
: ((op.metadata !== undefined ? custom : base.metadata) as T)
|
||||||
overlay.nouns.set(op.id, {
|
overlay.nouns.set(op.id, {
|
||||||
...base,
|
...base,
|
||||||
...(op.type !== undefined && { type: op.type }),
|
...(op.type !== undefined && { type: op.type }),
|
||||||
...(op.subtype !== undefined && { subtype: op.subtype }),
|
...(subtype !== undefined && { subtype }),
|
||||||
...(op.data !== undefined && { data: op.data }),
|
...(op.data !== undefined && { data: op.data }),
|
||||||
...(op.vector !== undefined && { vector: op.vector }),
|
...(op.vector !== undefined && { vector: op.vector }),
|
||||||
...(op.confidence !== undefined && { confidence: op.confidence }),
|
...(confidence !== undefined && { confidence }),
|
||||||
...(op.weight !== undefined && { weight: op.weight }),
|
...(weight !== undefined && { weight }),
|
||||||
metadata: mergedMetadata,
|
metadata: mergedMetadata,
|
||||||
updatedAt: Date.now(),
|
updatedAt: Date.now(),
|
||||||
_rev: (base._rev ?? 1) + 1
|
_rev: (base._rev ?? 1) + 1
|
||||||
|
|
@ -617,17 +649,32 @@ export class Db<T = any> {
|
||||||
}
|
}
|
||||||
if (duplicate) break
|
if (duplicate) break
|
||||||
|
|
||||||
|
// Reserved-field normalization — relationship mirror of the add
|
||||||
|
// op above (and of the committed relate() path).
|
||||||
|
const { reserved, custom } = splitVerbMetadataRecord(
|
||||||
|
op.metadata as Record<string, unknown> | undefined
|
||||||
|
)
|
||||||
|
const confidence =
|
||||||
|
op.confidence ?? (typeof reserved.confidence === 'number' ? reserved.confidence : undefined)
|
||||||
|
const weight =
|
||||||
|
op.weight ?? (typeof reserved.weight === 'number' ? reserved.weight : undefined)
|
||||||
|
const subtype =
|
||||||
|
op.subtype ?? (typeof reserved.subtype === 'string' ? reserved.subtype : undefined)
|
||||||
|
const service =
|
||||||
|
op.service ?? (typeof reserved.service === 'string' ? reserved.service : undefined)
|
||||||
|
|
||||||
const id = uuidv4()
|
const id = uuidv4()
|
||||||
overlay.verbs.set(id, {
|
overlay.verbs.set(id, {
|
||||||
id,
|
id,
|
||||||
from: op.from,
|
from: op.from,
|
||||||
to: op.to,
|
to: op.to,
|
||||||
type: op.type,
|
type: op.type,
|
||||||
...(op.subtype !== undefined && { subtype: op.subtype }),
|
...(subtype !== undefined && { subtype }),
|
||||||
weight: op.weight ?? 1.0,
|
weight: weight ?? 1.0,
|
||||||
|
...(confidence !== undefined && { confidence }),
|
||||||
...(op.data !== undefined && { data: op.data }),
|
...(op.data !== undefined && { data: op.data }),
|
||||||
metadata: op.metadata,
|
metadata: custom as T,
|
||||||
...(op.service !== undefined && { service: op.service }),
|
...(service !== undefined && { service }),
|
||||||
createdAt: Date.now()
|
createdAt: Date.now()
|
||||||
})
|
})
|
||||||
if (op.bidirectional) {
|
if (op.bidirectional) {
|
||||||
|
|
@ -637,11 +684,12 @@ export class Db<T = any> {
|
||||||
from: op.to,
|
from: op.to,
|
||||||
to: op.from,
|
to: op.from,
|
||||||
type: op.type,
|
type: op.type,
|
||||||
...(op.subtype !== undefined && { subtype: op.subtype }),
|
...(subtype !== undefined && { subtype }),
|
||||||
weight: op.weight ?? 1.0,
|
weight: weight ?? 1.0,
|
||||||
|
...(confidence !== undefined && { confidence }),
|
||||||
...(op.data !== undefined && { data: op.data }),
|
...(op.data !== undefined && { data: op.data }),
|
||||||
metadata: op.metadata,
|
metadata: custom as T,
|
||||||
...(op.service !== undefined && { service: op.service }),
|
...(service !== undefined && { service }),
|
||||||
createdAt: Date.now()
|
createdAt: Date.now()
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -155,7 +155,15 @@ export class EntityDeduplicator {
|
||||||
throw new Error(`Entity ${existingId} not found`)
|
throw new Error(`Entity ${existingId} not found`)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Merge metadata
|
// Update confidence (weighted average) — `confidence` is a reserved
|
||||||
|
// top-level field, so it travels via the dedicated update() param, not
|
||||||
|
// the metadata bag.
|
||||||
|
const mergedConfidence = this.mergeConfidence(
|
||||||
|
existing.confidence ?? 0.5,
|
||||||
|
candidate.confidence
|
||||||
|
)
|
||||||
|
|
||||||
|
// Merge metadata (custom fields only — reserved fields are top-level)
|
||||||
const mergedMetadata = {
|
const mergedMetadata = {
|
||||||
...existing.metadata,
|
...existing.metadata,
|
||||||
// Track provenance
|
// Track provenance
|
||||||
|
|
@ -168,11 +176,6 @@ export class EntityDeduplicator {
|
||||||
...(existing.metadata?.vfsPaths || [existing.metadata?.vfsPath]).filter(Boolean),
|
...(existing.metadata?.vfsPaths || [existing.metadata?.vfsPath]).filter(Boolean),
|
||||||
candidate.metadata?.vfsPath
|
candidate.metadata?.vfsPath
|
||||||
].filter(Boolean),
|
].filter(Boolean),
|
||||||
// Update confidence (weighted average)
|
|
||||||
confidence: this.mergeConfidence(
|
|
||||||
existing.metadata?.confidence || 0.5,
|
|
||||||
candidate.confidence
|
|
||||||
),
|
|
||||||
// Merge other metadata
|
// Merge other metadata
|
||||||
...this.mergeMetadataFields(existing.metadata, candidate.metadata),
|
...this.mergeMetadataFields(existing.metadata, candidate.metadata),
|
||||||
// Track last update
|
// Track last update
|
||||||
|
|
@ -183,6 +186,7 @@ export class EntityDeduplicator {
|
||||||
// Update entity
|
// Update entity
|
||||||
await this.brain.update({
|
await this.brain.update({
|
||||||
id: existingId,
|
id: existingId,
|
||||||
|
confidence: mergedConfidence,
|
||||||
metadata: mergedMetadata,
|
metadata: mergedMetadata,
|
||||||
merge: true
|
merge: true
|
||||||
})
|
})
|
||||||
|
|
@ -191,7 +195,7 @@ export class EntityDeduplicator {
|
||||||
mergedEntityId: existingId,
|
mergedEntityId: existingId,
|
||||||
wasMerged: true,
|
wasMerged: true,
|
||||||
mergedWith: existing.metadata?.name || existingId,
|
mergedWith: existing.metadata?.name || existingId,
|
||||||
confidence: mergedMetadata.confidence,
|
confidence: mergedConfidence,
|
||||||
provenance: mergedMetadata.imports
|
provenance: mergedMetadata.imports
|
||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
|
@ -218,17 +222,18 @@ export class EntityDeduplicator {
|
||||||
// No duplicate found, create new entity. Preserve any subtype the candidate
|
// No duplicate found, create new entity. Preserve any subtype the candidate
|
||||||
// carried (set by the extractor or upstream importer), else fall back to
|
// carried (set by the extractor or upstream importer), else fall back to
|
||||||
// `'imported'` so enforcement doesn't fire (added 7.30.1).
|
// `'imported'` so enforcement doesn't fire (added 7.30.1).
|
||||||
|
// `confidence` is a reserved top-level field (dedicated add() param);
|
||||||
|
// creation time is system-managed — neither belongs in the metadata bag.
|
||||||
const entityId = await this.brain.add({
|
const entityId = await this.brain.add({
|
||||||
data: candidate.description || candidate.name,
|
data: candidate.description || candidate.name,
|
||||||
type: candidate.type,
|
type: candidate.type,
|
||||||
subtype: (candidate as any).subtype ?? 'imported',
|
subtype: (candidate as any).subtype ?? 'imported',
|
||||||
|
confidence: candidate.confidence,
|
||||||
metadata: {
|
metadata: {
|
||||||
...candidate.metadata,
|
...candidate.metadata,
|
||||||
name: candidate.name,
|
name: candidate.name,
|
||||||
confidence: candidate.confidence,
|
|
||||||
imports: [importSource],
|
imports: [importSource],
|
||||||
vfsPaths: [candidate.metadata?.vfsPath].filter(Boolean),
|
vfsPaths: [candidate.metadata?.vfsPath].filter(Boolean),
|
||||||
createdAt: Date.now(),
|
|
||||||
mergeCount: 0
|
mergeCount: 0
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
|
||||||
|
|
@ -1051,7 +1051,6 @@ export class ImportCoordinator {
|
||||||
...(trackingContext && {
|
...(trackingContext && {
|
||||||
importIds: [trackingContext.importId],
|
importIds: [trackingContext.importId],
|
||||||
projectId: trackingContext.projectId,
|
projectId: trackingContext.projectId,
|
||||||
createdAt: Date.now(),
|
|
||||||
importFormat: trackingContext.importFormat,
|
importFormat: trackingContext.importFormat,
|
||||||
...trackingContext.customMetadata
|
...trackingContext.customMetadata
|
||||||
})
|
})
|
||||||
|
|
@ -1133,11 +1132,12 @@ export class ImportCoordinator {
|
||||||
rowNumber: row.rowNumber,
|
rowNumber: row.rowNumber,
|
||||||
extractedAt: Date.now(),
|
extractedAt: Date.now(),
|
||||||
format: sourceInfo?.format,
|
format: sourceInfo?.format,
|
||||||
// Import tracking metadata
|
// Import tracking metadata (`createdAt` is reserved — the
|
||||||
|
// relationship's own creation time is system-managed, and the
|
||||||
|
// import timestamp already travels as `extractedAt`)
|
||||||
...(trackingContext && {
|
...(trackingContext && {
|
||||||
importIds: [trackingContext.importId],
|
importIds: [trackingContext.importId],
|
||||||
projectId: trackingContext.projectId,
|
projectId: trackingContext.projectId,
|
||||||
createdAt: Date.now(),
|
|
||||||
importFormat: trackingContext.importFormat,
|
importFormat: trackingContext.importFormat,
|
||||||
...trackingContext.customMetadata
|
...trackingContext.customMetadata
|
||||||
})
|
})
|
||||||
|
|
|
||||||
|
|
@ -194,10 +194,10 @@ export class SmartImportOrchestrator {
|
||||||
data: extracted.entity.description,
|
data: extracted.entity.description,
|
||||||
type: extracted.entity.type,
|
type: extracted.entity.type,
|
||||||
subtype: (extracted.entity as any).subtype ?? options.defaultSubtype ?? 'imported',
|
subtype: (extracted.entity as any).subtype ?? options.defaultSubtype ?? 'imported',
|
||||||
|
confidence: extracted.entity.confidence, // reserved field — dedicated param, not metadata
|
||||||
metadata: {
|
metadata: {
|
||||||
...extracted.entity.metadata,
|
...extracted.entity.metadata,
|
||||||
name: extracted.entity.name,
|
name: extracted.entity.name,
|
||||||
confidence: extracted.entity.confidence,
|
|
||||||
importedFrom: 'smart-import'
|
importedFrom: 'smart-import'
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
@ -602,7 +602,8 @@ export class SmartImportOrchestrator {
|
||||||
data: extracted.entity.description,
|
data: extracted.entity.description,
|
||||||
type: extracted.entity.type,
|
type: extracted.entity.type,
|
||||||
subtype: (extracted.entity as any).subtype ?? options.defaultSubtype ?? 'imported',
|
subtype: (extracted.entity as any).subtype ?? options.defaultSubtype ?? 'imported',
|
||||||
metadata: { ...extracted.entity.metadata, name: extracted.entity.name, confidence: extracted.entity.confidence, importedFrom: 'smart-import' }
|
confidence: extracted.entity.confidence, // reserved field — dedicated param, not metadata
|
||||||
|
metadata: { ...extracted.entity.metadata, name: extracted.entity.name, importedFrom: 'smart-import' }
|
||||||
})
|
})
|
||||||
result.entityIds.push(entityId)
|
result.entityIds.push(entityId)
|
||||||
result.stats.entitiesCreated++
|
result.stats.entitiesCreated++
|
||||||
|
|
@ -617,7 +618,7 @@ export class SmartImportOrchestrator {
|
||||||
onProgress?.({ phase: 'creating', message: 'Preparing relationships...', processed: 0, total: result.extraction.rows.length, entities: result.entityIds.length, relationships: 0 })
|
onProgress?.({ phase: 'creating', message: 'Preparing relationships...', processed: 0, total: result.extraction.rows.length, entities: result.entityIds.length, relationships: 0 })
|
||||||
|
|
||||||
// Collect all relationship parameters
|
// Collect all relationship parameters
|
||||||
const relationshipParams: Array<{from: string; to: string; type: VerbType; subtype?: string; metadata?: any}> = []
|
const relationshipParams: Array<{from: string; to: string; type: VerbType; subtype?: string; confidence?: number; metadata?: any}> = []
|
||||||
|
|
||||||
for (const extracted of result.extraction.rows) {
|
for (const extracted of result.extraction.rows) {
|
||||||
for (const rel of extracted.relationships) {
|
for (const rel of extracted.relationships) {
|
||||||
|
|
@ -635,7 +636,8 @@ export class SmartImportOrchestrator {
|
||||||
result.entityIds.push(toEntityId)
|
result.entityIds.push(toEntityId)
|
||||||
}
|
}
|
||||||
// Relationship subtype precedence: extractor → caller default → `'imported'` (7.30.1).
|
// Relationship subtype precedence: extractor → caller default → `'imported'` (7.30.1).
|
||||||
relationshipParams.push({ from: extracted.entity.id, to: toEntityId, type: rel.type, subtype: (rel as any).subtype ?? options.defaultSubtype ?? 'imported', metadata: { confidence: rel.confidence, evidence: rel.evidence } })
|
// `confidence` is a reserved top-level field — dedicated relate() param, not metadata
|
||||||
|
relationshipParams.push({ from: extracted.entity.id, to: toEntityId, type: rel.type, subtype: (rel as any).subtype ?? options.defaultSubtype ?? 'imported', confidence: rel.confidence, metadata: { evidence: rel.evidence } })
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
result.errors.push(`Failed to prepare relationship: ${error.message}`)
|
result.errors.push(`Failed to prepare relationship: ${error.message}`)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
21
src/index.ts
21
src/index.ts
|
|
@ -26,6 +26,7 @@ export type {
|
||||||
AddParams,
|
AddParams,
|
||||||
UpdateParams,
|
UpdateParams,
|
||||||
RelateParams,
|
RelateParams,
|
||||||
|
UpdateRelationParams,
|
||||||
FindParams,
|
FindParams,
|
||||||
SubtypeRegistry,
|
SubtypeRegistry,
|
||||||
FillSubtypeRule,
|
FillSubtypeRule,
|
||||||
|
|
@ -44,6 +45,26 @@ export type {
|
||||||
AggregationProvider
|
AggregationProvider
|
||||||
} from './types/brainy.types.js'
|
} from './types/brainy.types.js'
|
||||||
|
|
||||||
|
// Reserved-field contract — the canonical list of Brainy-owned field names
|
||||||
|
// that may never appear inside a `metadata` bag (see docs/concepts/consistency-model.md)
|
||||||
|
export {
|
||||||
|
RESERVED_ENTITY_FIELDS,
|
||||||
|
RESERVED_RELATION_FIELDS,
|
||||||
|
splitNounMetadataRecord,
|
||||||
|
splitVerbMetadataRecord
|
||||||
|
} from './types/reservedFields.js'
|
||||||
|
export type {
|
||||||
|
ReservedEntityField,
|
||||||
|
ReservedRelationField,
|
||||||
|
EntityMetadataInput,
|
||||||
|
EntityMetadataPatch,
|
||||||
|
RelationMetadataInput,
|
||||||
|
RelationMetadataPatch,
|
||||||
|
NoReservedEntityKeys,
|
||||||
|
NoReservedRelationKeys,
|
||||||
|
SplitMetadataRecord
|
||||||
|
} from './types/reservedFields.js'
|
||||||
|
|
||||||
// Export Aggregation Engine
|
// Export Aggregation Engine
|
||||||
export { AggregationIndex, AggregateMaterializer, bucketTimestamp, parseBucketRange } from './aggregation/index.js'
|
export { AggregationIndex, AggregateMaterializer, bucketTimestamp, parseBucketRange } from './aggregation/index.js'
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -808,13 +808,12 @@ export class NeuralImport {
|
||||||
type: relationship.verbType as VerbType,
|
type: relationship.verbType as VerbType,
|
||||||
subtype: (relationship as any).subtype ?? options.defaultSubtype ?? 'extracted',
|
subtype: (relationship as any).subtype ?? options.defaultSubtype ?? 'extracted',
|
||||||
weight: relationship.weight,
|
weight: relationship.weight,
|
||||||
|
confidence: relationship.confidence, // reserved field — dedicated param, not metadata
|
||||||
metadata: {
|
metadata: {
|
||||||
confidence: relationship.confidence,
|
context: relationship.context,
|
||||||
context: relationship.context,
|
...relationship.metadata
|
||||||
...relationship.metadata
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
)
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
spinner.succeed(this.colors.success(
|
spinner.succeed(this.colors.success(
|
||||||
|
|
|
||||||
|
|
@ -4,11 +4,8 @@
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import {
|
import {
|
||||||
GraphVerb,
|
|
||||||
HNSWNoun,
|
HNSWNoun,
|
||||||
HNSWVerb,
|
HNSWVerb,
|
||||||
NounMetadata,
|
|
||||||
VerbMetadata,
|
|
||||||
HNSWNounWithMetadata,
|
HNSWNounWithMetadata,
|
||||||
HNSWVerbWithMetadata,
|
HNSWVerbWithMetadata,
|
||||||
StatisticsData,
|
StatisticsData,
|
||||||
|
|
@ -2866,26 +2863,22 @@ export class FileSystemStorage extends BaseStorage {
|
||||||
connections = connectionsMap
|
connections = connectionsMap
|
||||||
}
|
}
|
||||||
|
|
||||||
// Extract standard fields from metadata to top-level
|
// Canonical hydration (single source of truth:
|
||||||
const metadataObj = (metadata || {}) as VerbMetadata
|
// src/types/reservedFields.ts) — reserved fields top-level, ONLY
|
||||||
const { createdAt, updatedAt, confidence, weight, service, data: dataField, createdBy, ...customMetadata } = metadataObj
|
// custom fields in `metadata`. The previous hand-rolled
|
||||||
|
// destructure here missed `subtype` (so streamed verbs lost it)
|
||||||
const verbWithMetadata: HNSWVerbWithMetadata = {
|
// and `verb` (so the type key echoed inside custom metadata).
|
||||||
id: edge.id,
|
const verbWithMetadata: HNSWVerbWithMetadata = this.hydrateVerbWithMetadata(
|
||||||
vector: edge.vector,
|
{
|
||||||
connections: connections || new Map(),
|
id: edge.id,
|
||||||
verb: edge.verb,
|
vector: edge.vector,
|
||||||
sourceId: edge.sourceId,
|
connections: connections || new Map(),
|
||||||
targetId: edge.targetId,
|
verb: edge.verb,
|
||||||
createdAt: (createdAt as number) || Date.now(),
|
sourceId: edge.sourceId,
|
||||||
updatedAt: (updatedAt as number) || Date.now(),
|
targetId: edge.targetId
|
||||||
confidence: confidence as number | undefined,
|
},
|
||||||
weight: weight as number | undefined,
|
metadata
|
||||||
service: service as string | undefined,
|
)
|
||||||
data: dataField as Record<string, any> | undefined,
|
|
||||||
createdBy,
|
|
||||||
metadata: customMetadata
|
|
||||||
}
|
|
||||||
|
|
||||||
// Apply filters
|
// Apply filters
|
||||||
if (options.filter) {
|
if (options.filter) {
|
||||||
|
|
|
||||||
|
|
@ -30,6 +30,32 @@ import { BlobStorage, type BlobStoreAdapter } from './blobStorage.js'
|
||||||
import { unwrapBinaryData } from './binaryDataCodec.js'
|
import { unwrapBinaryData } from './binaryDataCodec.js'
|
||||||
import { prodLog } from '../utils/logger.js'
|
import { prodLog } from '../utils/logger.js'
|
||||||
import { MetadataWriteBuffer } from '../utils/metadataWriteBuffer.js'
|
import { MetadataWriteBuffer } from '../utils/metadataWriteBuffer.js'
|
||||||
|
import {
|
||||||
|
splitNounMetadataRecord,
|
||||||
|
splitVerbMetadataRecord
|
||||||
|
} from '../types/reservedFields.js'
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Normalize a stored timestamp value to epoch milliseconds. Brainy 8.0 writes
|
||||||
|
* plain numbers; records written by pre-8.0 cloud adapters may carry the
|
||||||
|
* `{ seconds, nanoseconds }` object form. Anything else falls back to now —
|
||||||
|
* matching the long-standing `|| Date.now()` combine behavior.
|
||||||
|
* @param value - The raw `createdAt`/`updatedAt` value from a stored metadata record.
|
||||||
|
* @returns Epoch milliseconds.
|
||||||
|
*/
|
||||||
|
function normalizeStoredTimestamp(value: unknown): number {
|
||||||
|
if (typeof value === 'number' && value > 0) {
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
if (
|
||||||
|
value !== null &&
|
||||||
|
typeof value === 'object' &&
|
||||||
|
typeof (value as { seconds?: unknown }).seconds === 'number'
|
||||||
|
) {
|
||||||
|
return (value as { seconds: number }).seconds * 1000
|
||||||
|
}
|
||||||
|
return Date.now()
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Storage key analysis result
|
* Storage key analysis result
|
||||||
|
|
@ -969,6 +995,74 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
await this.saveNoun_internal(noun)
|
await this.saveNoun_internal(noun)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Hydrate a deserialized noun (pure HNSW vector data) with its stored flat
|
||||||
|
* metadata record — THE canonical noun combine for every storage read path.
|
||||||
|
* The record is split through `splitNounMetadataRecord` (single source of
|
||||||
|
* truth: src/types/reservedFields.ts): reserved fields surface ONLY at
|
||||||
|
* top level and `metadata` carries ONLY the consumer's custom fields.
|
||||||
|
* Adding a combine site that bypasses this helper reintroduces the
|
||||||
|
* reserved-field echo bug — don't.
|
||||||
|
*
|
||||||
|
* @param noun - The deserialized HNSW noun (id/vector/connections/level).
|
||||||
|
* @param metadata - The stored flat metadata record (reserved + custom keys).
|
||||||
|
* @returns The combined noun with reserved fields top-level, custom fields in `metadata`.
|
||||||
|
*/
|
||||||
|
protected hydrateNounWithMetadata(
|
||||||
|
noun: HNSWNoun,
|
||||||
|
metadata: Record<string, unknown> | null | undefined
|
||||||
|
): HNSWNounWithMetadata {
|
||||||
|
const { reserved, custom } = splitNounMetadataRecord(metadata)
|
||||||
|
return {
|
||||||
|
...noun,
|
||||||
|
// Standard fields at top-level
|
||||||
|
type: (reserved.noun as NounType) || NounType.Thing,
|
||||||
|
subtype: reserved.subtype as string | undefined,
|
||||||
|
createdAt: normalizeStoredTimestamp(reserved.createdAt),
|
||||||
|
updatedAt: normalizeStoredTimestamp(reserved.updatedAt),
|
||||||
|
confidence: reserved.confidence as number | undefined,
|
||||||
|
weight: reserved.weight as number | undefined,
|
||||||
|
service: reserved.service as string | undefined,
|
||||||
|
data: reserved.data as Record<string, any> | undefined,
|
||||||
|
createdBy: reserved.createdBy as HNSWNounWithMetadata['createdBy'],
|
||||||
|
_rev: typeof reserved._rev === 'number' ? reserved._rev : 1,
|
||||||
|
// Only custom user fields remain in metadata
|
||||||
|
metadata: custom
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Hydrate a deserialized verb (structural core) with its stored flat
|
||||||
|
* metadata record — THE canonical verb combine, the relationship mirror of
|
||||||
|
* {@link hydrateNounWithMetadata}. Splitting through
|
||||||
|
* `splitVerbMetadataRecord` extracts `verb` too, so the type key never
|
||||||
|
* echoes inside `metadata`.
|
||||||
|
*
|
||||||
|
* @param verb - The deserialized HNSW verb (id/vector/connections/verb/sourceId/targetId).
|
||||||
|
* @param metadata - The stored flat metadata record (reserved + custom keys).
|
||||||
|
* @returns The combined verb with reserved fields top-level, custom fields in `metadata`.
|
||||||
|
*/
|
||||||
|
protected hydrateVerbWithMetadata(
|
||||||
|
verb: HNSWVerb,
|
||||||
|
metadata: Record<string, unknown> | null | undefined
|
||||||
|
): HNSWVerbWithMetadata {
|
||||||
|
const { reserved, custom } = splitVerbMetadataRecord(metadata)
|
||||||
|
return {
|
||||||
|
...verb,
|
||||||
|
// Standard fields at top-level
|
||||||
|
subtype: reserved.subtype as string | undefined,
|
||||||
|
createdAt: normalizeStoredTimestamp(reserved.createdAt),
|
||||||
|
updatedAt: normalizeStoredTimestamp(reserved.updatedAt),
|
||||||
|
confidence: reserved.confidence as number | undefined,
|
||||||
|
weight: reserved.weight as number | undefined,
|
||||||
|
service: reserved.service as string | undefined,
|
||||||
|
data: reserved.data as Record<string, any> | undefined,
|
||||||
|
createdBy: reserved.createdBy as HNSWVerbWithMetadata['createdBy'],
|
||||||
|
// Only custom user fields remain in metadata
|
||||||
|
metadata: custom
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Get a noun from storage (returns combined HNSWNounWithMetadata)
|
* Get a noun from storage (returns combined HNSWNounWithMetadata)
|
||||||
* @param id Entity ID
|
* @param id Entity ID
|
||||||
|
|
@ -990,28 +1084,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
return null
|
return null
|
||||||
}
|
}
|
||||||
|
|
||||||
// Combine into HNSWNounWithMetadata - Extract standard fields to top-level
|
return this.hydrateNounWithMetadata(vector, metadata)
|
||||||
const { noun, subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, _rev, ...customMetadata } = metadata
|
|
||||||
|
|
||||||
return {
|
|
||||||
id: vector.id,
|
|
||||||
vector: vector.vector,
|
|
||||||
connections: vector.connections,
|
|
||||||
level: vector.level,
|
|
||||||
// Standard fields at top-level
|
|
||||||
type: (noun as NounType) || NounType.Thing,
|
|
||||||
subtype: subtype as string | undefined,
|
|
||||||
createdAt: (createdAt as number) || Date.now(),
|
|
||||||
updatedAt: (updatedAt as number) || Date.now(),
|
|
||||||
confidence: confidence as number | undefined,
|
|
||||||
weight: weight as number | undefined,
|
|
||||||
service: service as string | undefined,
|
|
||||||
data: data as Record<string, any> | undefined,
|
|
||||||
createdBy,
|
|
||||||
_rev: typeof _rev === 'number' ? _rev : 1,
|
|
||||||
// Only custom user fields remain in metadata
|
|
||||||
metadata: customMetadata
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|
@ -1025,29 +1098,12 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
// Internal method returns HNSWNoun[], need to combine with metadata
|
// Internal method returns HNSWNoun[], need to combine with metadata
|
||||||
const nouns = await this.getNounsByNounType_internal(nounType)
|
const nouns = await this.getNounsByNounType_internal(nounType)
|
||||||
|
|
||||||
// Combine each noun with its metadata - Extract standard fields to top-level
|
// Combine each noun with its metadata via the canonical hydration helper
|
||||||
const nounsWithMetadata: HNSWNounWithMetadata[] = []
|
const nounsWithMetadata: HNSWNounWithMetadata[] = []
|
||||||
for (const noun of nouns) {
|
for (const noun of nouns) {
|
||||||
const metadata = await this.getNounMetadata(noun.id)
|
const metadata = await this.getNounMetadata(noun.id)
|
||||||
if (metadata) {
|
if (metadata) {
|
||||||
const { noun: nounType, subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, _rev, ...customMetadata } = metadata
|
nounsWithMetadata.push(this.hydrateNounWithMetadata(noun, metadata))
|
||||||
|
|
||||||
nounsWithMetadata.push({
|
|
||||||
...noun,
|
|
||||||
// Standard fields at top-level
|
|
||||||
type: (nounType as NounType) || NounType.Thing,
|
|
||||||
subtype: subtype as string | undefined,
|
|
||||||
createdAt: (createdAt as number) || Date.now(),
|
|
||||||
updatedAt: (updatedAt as number) || Date.now(),
|
|
||||||
confidence: confidence as number | undefined,
|
|
||||||
weight: weight as number | undefined,
|
|
||||||
service: service as string | undefined,
|
|
||||||
data: data as Record<string, any> | undefined,
|
|
||||||
createdBy,
|
|
||||||
_rev: typeof _rev === 'number' ? _rev : 1,
|
|
||||||
// Only custom user fields in metadata
|
|
||||||
metadata: customMetadata
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -1109,28 +1165,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
return null
|
return null
|
||||||
}
|
}
|
||||||
|
|
||||||
// Combine into HNSWVerbWithMetadata - Extract standard fields to top-level
|
return this.hydrateVerbWithMetadata(verb, metadata)
|
||||||
const { subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, _rev, ...customMetadata } = metadata
|
|
||||||
|
|
||||||
return {
|
|
||||||
id: verb.id,
|
|
||||||
vector: verb.vector,
|
|
||||||
connections: verb.connections,
|
|
||||||
verb: verb.verb,
|
|
||||||
sourceId: verb.sourceId,
|
|
||||||
targetId: verb.targetId,
|
|
||||||
// Standard fields at top-level
|
|
||||||
subtype: subtype as string | undefined,
|
|
||||||
createdAt: (createdAt as number) || Date.now(),
|
|
||||||
updatedAt: (updatedAt as number) || Date.now(),
|
|
||||||
confidence: confidence as number | undefined,
|
|
||||||
weight: weight as number | undefined,
|
|
||||||
service: service as string | undefined,
|
|
||||||
data: data as Record<string, any> | undefined,
|
|
||||||
createdBy,
|
|
||||||
// Only custom user fields remain in metadata
|
|
||||||
metadata: customMetadata
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|
@ -1181,98 +1216,15 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
const metadataData = metadataResults.get(metadataPath)
|
const metadataData = metadataResults.get(metadataPath)
|
||||||
|
|
||||||
if (vectorData && metadataData) {
|
if (vectorData && metadataData) {
|
||||||
// Deserialize verb
|
// Deserialize, then combine via the canonical hydration helper
|
||||||
const verb = this.deserializeVerb(vectorData)
|
const verb = this.deserializeVerb(vectorData)
|
||||||
|
results.set(id, this.hydrateVerbWithMetadata(verb, metadataData))
|
||||||
// Extract standard fields to top-level
|
|
||||||
const { subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, _rev, ...customMetadata } = metadataData
|
|
||||||
|
|
||||||
results.set(id, {
|
|
||||||
id: verb.id,
|
|
||||||
vector: verb.vector,
|
|
||||||
connections: verb.connections,
|
|
||||||
verb: verb.verb,
|
|
||||||
sourceId: verb.sourceId,
|
|
||||||
targetId: verb.targetId,
|
|
||||||
// Standard fields at top-level
|
|
||||||
subtype: subtype as string | undefined,
|
|
||||||
createdAt: (createdAt as number) || Date.now(),
|
|
||||||
updatedAt: (updatedAt as number) || Date.now(),
|
|
||||||
confidence: confidence as number | undefined,
|
|
||||||
weight: weight as number | undefined,
|
|
||||||
service: service as string | undefined,
|
|
||||||
data: data as Record<string, any> | undefined,
|
|
||||||
createdBy,
|
|
||||||
// Only custom user fields remain in metadata
|
|
||||||
metadata: customMetadata
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return results
|
return results
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Convert an HNSW verb (raw vector store entry) into a `GraphVerb` shape by
|
|
||||||
* combining with the verb's metadata. Used by internal code paths that need
|
|
||||||
* the graph-layer shape rather than the storage-layer shape.
|
|
||||||
*/
|
|
||||||
protected async convertHNSWVerbToGraphVerb(hnswVerb: HNSWVerb): Promise<GraphVerb | null> {
|
|
||||||
try {
|
|
||||||
// Load metadata
|
|
||||||
const metadata = await this.getVerbMetadata(hnswVerb.id)
|
|
||||||
|
|
||||||
// Create default timestamp in Firestore format
|
|
||||||
const defaultTimestamp = {
|
|
||||||
seconds: Math.floor(Date.now() / 1000),
|
|
||||||
nanoseconds: (Date.now() % 1000) * 1000000
|
|
||||||
}
|
|
||||||
|
|
||||||
// Create default createdBy if not present
|
|
||||||
const defaultCreatedBy = {
|
|
||||||
augmentation: 'unknown',
|
|
||||||
version: '1.0'
|
|
||||||
}
|
|
||||||
|
|
||||||
// Convert flexible timestamp to Firestore format for GraphVerb
|
|
||||||
const normalizeTimestamp = (ts: any) => {
|
|
||||||
if (!ts) return defaultTimestamp
|
|
||||||
if (typeof ts === 'number') {
|
|
||||||
return {
|
|
||||||
seconds: Math.floor(ts / 1000),
|
|
||||||
nanoseconds: (ts % 1000) * 1000000
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return ts
|
|
||||||
}
|
|
||||||
|
|
||||||
return {
|
|
||||||
id: hnswVerb.id,
|
|
||||||
vector: hnswVerb.vector,
|
|
||||||
|
|
||||||
// CORE FIELDS from HNSWVerb
|
|
||||||
verb: hnswVerb.verb,
|
|
||||||
sourceId: hnswVerb.sourceId,
|
|
||||||
targetId: hnswVerb.targetId,
|
|
||||||
|
|
||||||
// Alias for ergonomic access
|
|
||||||
type: hnswVerb.verb,
|
|
||||||
|
|
||||||
// Optional fields from metadata file
|
|
||||||
weight: metadata?.weight || 1.0,
|
|
||||||
metadata: metadata as any || {},
|
|
||||||
createdAt: normalizeTimestamp(metadata?.createdAt),
|
|
||||||
updatedAt: normalizeTimestamp(metadata?.updatedAt),
|
|
||||||
createdBy: metadata?.createdBy || defaultCreatedBy,
|
|
||||||
data: metadata?.data as Record<string, any> | undefined,
|
|
||||||
embedding: hnswVerb.vector
|
|
||||||
}
|
|
||||||
} catch (error) {
|
|
||||||
prodLog.error(`Failed to convert HNSWVerb to GraphVerb for ${hnswVerb.id}:`, error)
|
|
||||||
return null
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Internal method for loading all verbs - used by performance optimizations
|
* Internal method for loading all verbs - used by performance optimizations
|
||||||
* @internal - Do not use directly, use getVerbs() with pagination instead
|
* @internal - Do not use directly, use getVerbs() with pagination instead
|
||||||
|
|
@ -1571,27 +1523,10 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Combine noun + metadata. Subtype is surfaced to top-level here
|
// Combine noun + metadata via the canonical hydration helper —
|
||||||
// (mirrors what `getNoun()` already does) so callers — including
|
// reserved fields top-level, ONLY custom fields in `metadata`
|
||||||
// the 7.30.1 `brain.audit()` diagnostic — see a consistent shape
|
// (this site previously echoed the full flat record).
|
||||||
// regardless of which getter path they reach.
|
collectedNouns.push(this.hydrateNounWithMetadata(deserialized, metadata))
|
||||||
collectedNouns.push({
|
|
||||||
...deserialized,
|
|
||||||
type: (metadata.noun || 'thing') as NounType,
|
|
||||||
subtype: (metadata as any).subtype as string | undefined,
|
|
||||||
confidence: metadata.confidence,
|
|
||||||
weight: metadata.weight,
|
|
||||||
createdAt: metadata.createdAt
|
|
||||||
? (typeof metadata.createdAt === 'number' ? metadata.createdAt : metadata.createdAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
updatedAt: metadata.updatedAt
|
|
||||||
? (typeof metadata.updatedAt === 'number' ? metadata.updatedAt : metadata.updatedAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
service: metadata.service,
|
|
||||||
data: metadata.data as Record<string, any> | undefined,
|
|
||||||
createdBy: metadata.createdBy,
|
|
||||||
metadata: metadata || ({} as NounMetadata)
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
|
@ -1723,22 +1658,10 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Combine verb + metadata
|
// Combine verb + metadata via the canonical hydration helper —
|
||||||
collectedVerbs.push({
|
// reserved fields top-level, ONLY custom fields in `metadata`
|
||||||
...verb,
|
// (this site previously echoed the full flat record).
|
||||||
subtype: (metadata as any)?.subtype as string | undefined,
|
collectedVerbs.push(this.hydrateVerbWithMetadata(verb, metadata))
|
||||||
weight: metadata?.weight,
|
|
||||||
confidence: metadata?.confidence,
|
|
||||||
createdAt: metadata?.createdAt
|
|
||||||
? (typeof metadata.createdAt === 'number' ? metadata.createdAt : metadata.createdAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
updatedAt: metadata?.updatedAt
|
|
||||||
? (typeof metadata.updatedAt === 'number' ? metadata.updatedAt : metadata.updatedAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
service: metadata?.service,
|
|
||||||
createdBy: metadata?.createdBy,
|
|
||||||
metadata: metadata || ({} as VerbMetadata)
|
|
||||||
})
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
// Skip verbs that fail to load
|
// Skip verbs that fail to load
|
||||||
}
|
}
|
||||||
|
|
@ -2571,31 +2494,9 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
const metadataData = metadataResults.get(metadataPath)
|
const metadataData = metadataResults.get(metadataPath)
|
||||||
|
|
||||||
if (vectorData && metadataData) {
|
if (vectorData && metadataData) {
|
||||||
// Deserialize noun
|
// Deserialize, then combine via the canonical hydration helper
|
||||||
const noun = this.deserializeNoun(vectorData)
|
const noun = this.deserializeNoun(vectorData)
|
||||||
|
results.set(id, this.hydrateNounWithMetadata(noun, metadataData))
|
||||||
// Extract standard fields to top-level
|
|
||||||
const { noun: nounType, subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, _rev, ...customMetadata } = metadataData
|
|
||||||
|
|
||||||
results.set(id, {
|
|
||||||
id: noun.id,
|
|
||||||
vector: noun.vector,
|
|
||||||
connections: noun.connections,
|
|
||||||
level: noun.level,
|
|
||||||
// Standard fields at top-level
|
|
||||||
type: (nounType as NounType) || NounType.Thing,
|
|
||||||
subtype: subtype as string | undefined,
|
|
||||||
createdAt: (createdAt as number) || Date.now(),
|
|
||||||
updatedAt: (updatedAt as number) || Date.now(),
|
|
||||||
confidence: confidence as number | undefined,
|
|
||||||
weight: weight as number | undefined,
|
|
||||||
service: service as string | undefined,
|
|
||||||
data: data as Record<string, any> | undefined,
|
|
||||||
createdBy,
|
|
||||||
_rev: typeof _rev === 'number' ? _rev : 1,
|
|
||||||
// Only custom user fields remain in metadata
|
|
||||||
metadata: customMetadata
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -3734,24 +3635,11 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
const metadata = metadataMap.get(metadataPath)
|
const metadata = metadataMap.get(metadataPath)
|
||||||
|
|
||||||
if (rawVerb && metadata) {
|
if (rawVerb && metadata) {
|
||||||
// CRITICAL - Deserialize connections Map from JSON storage format
|
// CRITICAL - Deserialize connections Map from JSON storage format,
|
||||||
|
// then combine via the canonical hydration helper (reserved fields
|
||||||
|
// top-level, ONLY custom fields in `metadata`).
|
||||||
const verb = this.deserializeVerb(rawVerb)
|
const verb = this.deserializeVerb(rawVerb)
|
||||||
|
results.push(this.hydrateVerbWithMetadata(verb, metadata))
|
||||||
results.push({
|
|
||||||
...verb,
|
|
||||||
subtype: (metadata as any).subtype as string | undefined,
|
|
||||||
weight: metadata.weight,
|
|
||||||
confidence: metadata.confidence,
|
|
||||||
createdAt: metadata.createdAt
|
|
||||||
? (typeof metadata.createdAt === 'number' ? metadata.createdAt : metadata.createdAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
updatedAt: metadata.updatedAt
|
|
||||||
? (typeof metadata.updatedAt === 'number' ? metadata.updatedAt : metadata.updatedAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
service: metadata.service,
|
|
||||||
createdBy: metadata.createdBy,
|
|
||||||
metadata: metadata || {} as VerbMetadata
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -3791,22 +3679,9 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
if (verb.sourceId === sourceId) {
|
if (verb.sourceId === sourceId) {
|
||||||
const metadataPath = getVerbMetadataPath(verb.id)
|
const metadataPath = getVerbMetadataPath(verb.id)
|
||||||
const metadata = await this.readCanonicalObject(metadataPath)
|
const metadata = await this.readCanonicalObject(metadataPath)
|
||||||
|
// Canonical hydration — reserved fields top-level, ONLY custom
|
||||||
results.push({
|
// fields in `metadata`.
|
||||||
...verb,
|
results.push(this.hydrateVerbWithMetadata(verb, metadata))
|
||||||
subtype: (metadata as any)?.subtype as string | undefined,
|
|
||||||
weight: metadata?.weight,
|
|
||||||
confidence: metadata?.confidence,
|
|
||||||
createdAt: metadata?.createdAt
|
|
||||||
? (typeof metadata.createdAt === 'number' ? metadata.createdAt : metadata.createdAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
updatedAt: metadata?.updatedAt
|
|
||||||
? (typeof metadata.updatedAt === 'number' ? metadata.updatedAt : metadata.updatedAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
service: metadata?.service,
|
|
||||||
createdBy: metadata?.createdBy,
|
|
||||||
metadata: metadata || {} as VerbMetadata
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
// Skip verbs that fail to load
|
// Skip verbs that fail to load
|
||||||
|
|
@ -3919,20 +3794,10 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
const metadataPath = getVerbMetadataPath(verbId)
|
const metadataPath = getVerbMetadataPath(verbId)
|
||||||
const metadata = metadataMap.get(metadataPath) || {}
|
const metadata = metadataMap.get(metadataPath) || {}
|
||||||
|
|
||||||
const hydratedVerb: HNSWVerbWithMetadata = {
|
// Canonical hydration — reserved fields top-level (including
|
||||||
...verbData,
|
// subtype/data, which this site previously dropped), ONLY custom
|
||||||
weight: metadata?.weight,
|
// fields in `metadata`.
|
||||||
confidence: metadata?.confidence,
|
const hydratedVerb = this.hydrateVerbWithMetadata(verbData, metadata)
|
||||||
createdAt: metadata?.createdAt
|
|
||||||
? (typeof metadata.createdAt === 'number' ? metadata.createdAt : metadata.createdAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
updatedAt: metadata?.updatedAt
|
|
||||||
? (typeof metadata.updatedAt === 'number' ? metadata.updatedAt : metadata.updatedAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
service: metadata?.service,
|
|
||||||
createdBy: metadata?.createdBy,
|
|
||||||
metadata: metadata as VerbMetadata
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add to results for this sourceId
|
// Add to results for this sourceId
|
||||||
const sourceVerbs = results.get(verbData.sourceId)!
|
const sourceVerbs = results.get(verbData.sourceId)!
|
||||||
|
|
@ -3976,21 +3841,9 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
const metadata = await this.getVerbMetadata(verbId)
|
const metadata = await this.getVerbMetadata(verbId)
|
||||||
|
|
||||||
if (verb && metadata) {
|
if (verb && metadata) {
|
||||||
results.push({
|
// Canonical hydration — reserved fields top-level, ONLY custom
|
||||||
...verb,
|
// fields in `metadata`.
|
||||||
subtype: (metadata as any).subtype as string | undefined,
|
results.push(this.hydrateVerbWithMetadata(verb, metadata))
|
||||||
weight: metadata.weight,
|
|
||||||
confidence: metadata.confidence,
|
|
||||||
createdAt: metadata.createdAt
|
|
||||||
? (typeof metadata.createdAt === 'number' ? metadata.createdAt : metadata.createdAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
updatedAt: metadata.updatedAt
|
|
||||||
? (typeof metadata.updatedAt === 'number' ? metadata.updatedAt : metadata.updatedAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
service: metadata.service,
|
|
||||||
createdBy: metadata.createdBy,
|
|
||||||
metadata: metadata || {} as VerbMetadata
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -4023,22 +3876,9 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
if (verb.targetId === targetId) {
|
if (verb.targetId === targetId) {
|
||||||
const metadataPath = getVerbMetadataPath(verb.id)
|
const metadataPath = getVerbMetadataPath(verb.id)
|
||||||
const metadata = await this.readCanonicalObject(metadataPath)
|
const metadata = await this.readCanonicalObject(metadataPath)
|
||||||
|
// Canonical hydration — reserved fields top-level, ONLY custom
|
||||||
results.push({
|
// fields in `metadata`.
|
||||||
...verb,
|
results.push(this.hydrateVerbWithMetadata(verb, metadata))
|
||||||
subtype: (metadata as any)?.subtype as string | undefined,
|
|
||||||
weight: metadata?.weight,
|
|
||||||
confidence: metadata?.confidence,
|
|
||||||
createdAt: metadata?.createdAt
|
|
||||||
? (typeof metadata.createdAt === 'number' ? metadata.createdAt : metadata.createdAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
updatedAt: metadata?.updatedAt
|
|
||||||
? (typeof metadata.updatedAt === 'number' ? metadata.updatedAt : metadata.updatedAt.seconds * 1000)
|
|
||||||
: Date.now(),
|
|
||||||
service: metadata?.service,
|
|
||||||
createdBy: metadata?.createdBy,
|
|
||||||
metadata: metadata || {} as VerbMetadata
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
// Skip verbs that fail to load
|
// Skip verbs that fail to load
|
||||||
|
|
@ -4079,32 +3919,15 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
// Filter by verb type
|
// Filter by verb type
|
||||||
if (hnswVerb.verb !== verbType) continue
|
if (hnswVerb.verb !== verbType) continue
|
||||||
|
|
||||||
// Load metadata separately (optional)
|
// Load metadata separately (optional), then combine via the
|
||||||
|
// canonical hydration helper (defensive vector copy preserved)
|
||||||
const metadata = await this.getVerbMetadata(hnswVerb.id)
|
const metadata = await this.getVerbMetadata(hnswVerb.id)
|
||||||
|
verbs.push(
|
||||||
// Extract standard fields from metadata to top-level
|
this.hydrateVerbWithMetadata(
|
||||||
const metadataObj = (metadata || {}) as VerbMetadata
|
{ ...hnswVerb, vector: [...hnswVerb.vector] },
|
||||||
const { subtype, createdAt, updatedAt, confidence, weight, service, data, createdBy, _rev, ...customMetadata } = metadataObj
|
metadata
|
||||||
|
)
|
||||||
const verbWithMetadata: HNSWVerbWithMetadata = {
|
)
|
||||||
id: hnswVerb.id,
|
|
||||||
vector: [...hnswVerb.vector],
|
|
||||||
connections: hnswVerb.connections, // Already deserialized
|
|
||||||
verb: hnswVerb.verb,
|
|
||||||
sourceId: hnswVerb.sourceId,
|
|
||||||
targetId: hnswVerb.targetId,
|
|
||||||
subtype: subtype as string | undefined,
|
|
||||||
createdAt: (createdAt as number) || Date.now(),
|
|
||||||
updatedAt: (updatedAt as number) || Date.now(),
|
|
||||||
confidence: confidence as number | undefined,
|
|
||||||
weight: weight as number | undefined,
|
|
||||||
service: service as string | undefined,
|
|
||||||
data: data as Record<string, any> | undefined,
|
|
||||||
createdBy,
|
|
||||||
metadata: customMetadata
|
|
||||||
}
|
|
||||||
|
|
||||||
verbs.push(verbWithMetadata)
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
// Skip verbs that fail to load
|
// Skip verbs that fail to load
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -6,6 +6,12 @@
|
||||||
|
|
||||||
import { StorageAdapter, Vector } from '../coreTypes.js'
|
import { StorageAdapter, Vector } from '../coreTypes.js'
|
||||||
import { NounType, VerbType } from './graphTypes.js'
|
import { NounType, VerbType } from './graphTypes.js'
|
||||||
|
import type {
|
||||||
|
EntityMetadataInput,
|
||||||
|
EntityMetadataPatch,
|
||||||
|
RelationMetadataInput,
|
||||||
|
RelationMetadataPatch
|
||||||
|
} from './reservedFields.js'
|
||||||
|
|
||||||
// ============= Core Types =============
|
// ============= Core Types =============
|
||||||
|
|
||||||
|
|
@ -283,8 +289,18 @@ export interface AddParams<T = any> {
|
||||||
* aggregation on the standard-field fast path.
|
* aggregation on the standard-field fast path.
|
||||||
*/
|
*/
|
||||||
subtype?: string
|
subtype?: string
|
||||||
/** Structured queryable fields — indexed by MetadataIndex, used in `where` filters */
|
/**
|
||||||
metadata?: T
|
* Structured queryable fields — indexed by MetadataIndex, used in `where` filters.
|
||||||
|
*
|
||||||
|
* Reserved entity fields (`RESERVED_ENTITY_FIELDS` — `noun`, `subtype`, `createdAt`,
|
||||||
|
* `updatedAt`, `confidence`, `weight`, `service`, `data`, `createdBy`, `_rev`) may NOT
|
||||||
|
* appear here — they have dedicated top-level params and the type makes a literal
|
||||||
|
* reserved key a compile error. Untyped (JavaScript) callers that pass one anyway are
|
||||||
|
* normalized at write time: user-settable fields remap to their top-level param
|
||||||
|
* (top-level wins when both are supplied), system-managed fields are dropped with a
|
||||||
|
* one-shot warning.
|
||||||
|
*/
|
||||||
|
metadata?: EntityMetadataInput<T>
|
||||||
/** Custom entity ID (auto-generated UUID v4 if not provided) */
|
/** Custom entity ID (auto-generated UUID v4 if not provided) */
|
||||||
id?: string
|
id?: string
|
||||||
/** Pre-computed embedding vector (skips auto-embedding when provided) */
|
/** Pre-computed embedding vector (skips auto-embedding when provided) */
|
||||||
|
|
@ -314,7 +330,15 @@ export interface UpdateParams<T = any> {
|
||||||
data?: any // New content to re-embed
|
data?: any // New content to re-embed
|
||||||
type?: NounType // Change type
|
type?: NounType // Change type
|
||||||
subtype?: string // Change subtype (set to '' or null-equivalent via dedicated unset is future work)
|
subtype?: string // Change subtype (set to '' or null-equivalent via dedicated unset is future work)
|
||||||
metadata?: Partial<T> // Metadata to update
|
/**
|
||||||
|
* Metadata fields to merge (or replace when `merge: false`). Reserved entity
|
||||||
|
* fields (`RESERVED_ENTITY_FIELDS`) may NOT appear here — `confidence` /
|
||||||
|
* `weight` / `subtype` have dedicated params on this call, and the rest are
|
||||||
|
* system-managed. A literal reserved key is a compile error; untyped callers
|
||||||
|
* are normalized at write time (remap user-settable, drop system-managed
|
||||||
|
* with a one-shot warning).
|
||||||
|
*/
|
||||||
|
metadata?: EntityMetadataPatch<T>
|
||||||
merge?: boolean // Merge or replace metadata (default: true)
|
merge?: boolean // Merge or replace metadata (default: true)
|
||||||
vector?: Vector // New pre-computed vector
|
vector?: Vector // New pre-computed vector
|
||||||
confidence?: number // Update type classification confidence
|
confidence?: number // Update type classification confidence
|
||||||
|
|
@ -355,8 +379,14 @@ export interface RelateParams<T = any> {
|
||||||
weight?: number
|
weight?: number
|
||||||
/** Content for the relationship (optional — overrides auto-computed vector) */
|
/** Content for the relationship (optional — overrides auto-computed vector) */
|
||||||
data?: any
|
data?: any
|
||||||
/** Structured queryable fields on the edge */
|
/**
|
||||||
metadata?: T
|
* Structured queryable fields on the edge. Reserved relationship fields
|
||||||
|
* (`RESERVED_RELATION_FIELDS` — `verb`, `subtype`, `createdAt`, `updatedAt`,
|
||||||
|
* `confidence`, `weight`, `service`, `data`, `createdBy`, `_rev`) may NOT
|
||||||
|
* appear here — they have dedicated params. A literal reserved key is a
|
||||||
|
* compile error; untyped callers are normalized at write time.
|
||||||
|
*/
|
||||||
|
metadata?: RelationMetadataInput<T>
|
||||||
/** Create reverse edge too (default: false) */
|
/** Create reverse edge too (default: false) */
|
||||||
bidirectional?: boolean
|
bidirectional?: boolean
|
||||||
/** Multi-tenancy service identifier */
|
/** Multi-tenancy service identifier */
|
||||||
|
|
@ -377,7 +407,13 @@ export interface UpdateRelationParams<T = any> {
|
||||||
weight?: number // New weight
|
weight?: number // New weight
|
||||||
confidence?: number // New confidence (0-1)
|
confidence?: number // New confidence (0-1)
|
||||||
data?: any // New content
|
data?: any // New content
|
||||||
metadata?: Partial<T> // Metadata to update
|
/**
|
||||||
|
* Metadata fields to merge (or replace when `merge: false`). Reserved
|
||||||
|
* relationship fields (`RESERVED_RELATION_FIELDS`) may NOT appear here —
|
||||||
|
* a literal reserved key is a compile error; untyped callers are
|
||||||
|
* normalized at write time.
|
||||||
|
*/
|
||||||
|
metadata?: RelationMetadataPatch<T>
|
||||||
merge?: boolean // Merge or replace metadata
|
merge?: boolean // Merge or replace metadata
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
250
src/types/reservedFields.ts
Normal file
250
src/types/reservedFields.ts
Normal file
|
|
@ -0,0 +1,250 @@
|
||||||
|
/**
|
||||||
|
* @module types/reservedFields
|
||||||
|
* @description The canonical reserved-field contract — ONE place that defines
|
||||||
|
* which keys belong to Brainy (top-level entity/relationship fields) and may
|
||||||
|
* therefore never live inside a `metadata` bag.
|
||||||
|
*
|
||||||
|
* Three layers enforce the contract, all driven by the constants below:
|
||||||
|
*
|
||||||
|
* 1. **Compile time** — `AddParams.metadata`, `UpdateParams.metadata`,
|
||||||
|
* `RelateParams.metadata` and `UpdateRelationParams.metadata` are typed so
|
||||||
|
* a literal reserved key is a TypeScript error (see
|
||||||
|
* {@link EntityMetadataInput} / {@link RelationMetadataInput}).
|
||||||
|
* 2. **Write time** — for untyped (JavaScript) callers that smuggle a
|
||||||
|
* reserved key past the compiler anyway, every write path normalizes the
|
||||||
|
* bag: user-mutable fields are remapped to their dedicated top-level
|
||||||
|
* param (top-level wins when both are supplied) and system-managed fields
|
||||||
|
* are dropped with a one-shot warning naming the correct write path.
|
||||||
|
* 3. **Read time** — every read path splits the stored flat record through
|
||||||
|
* {@link splitNounMetadataRecord} / {@link splitVerbMetadataRecord}, so a
|
||||||
|
* reserved field is surfaced ONLY at top level and `entity.metadata` /
|
||||||
|
* `relation.metadata` contain ONLY custom fields, always — live reads,
|
||||||
|
* batch reads, and historical (`asOf`) reads alike.
|
||||||
|
*
|
||||||
|
* Documented for consumers in `docs/concepts/consistency-model.md`
|
||||||
|
* ("Reserved fields").
|
||||||
|
*/
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Entity (noun) field names reserved by Brainy. These keys are
|
||||||
|
* stored in the flat per-entity metadata record alongside custom fields, but
|
||||||
|
* they belong to Brainy: every read path extracts them to top-level
|
||||||
|
* `Entity` fields, and no write path accepts them inside `metadata`.
|
||||||
|
*
|
||||||
|
* | Key | Canonical write path |
|
||||||
|
* |-----|----------------------|
|
||||||
|
* | `noun` | the `type` param of `add()` / `update()` (stored under the key `noun`) |
|
||||||
|
* | `subtype` | the `subtype` param |
|
||||||
|
* | `createdAt` | system-managed — set once at `add()` time |
|
||||||
|
* | `updatedAt` | system-managed — set on every write |
|
||||||
|
* | `confidence` | the `confidence` param |
|
||||||
|
* | `weight` | the `weight` param |
|
||||||
|
* | `service` | the `service` param of `add()` (immutable afterwards) |
|
||||||
|
* | `data` | the `data` param |
|
||||||
|
* | `createdBy` | the `createdBy` param of `add()` (immutable afterwards) |
|
||||||
|
* | `_rev` | system-managed revision counter — pass `ifRev` to `update()` for CAS |
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* import { RESERVED_ENTITY_FIELDS } from '@soulcraft/brainy'
|
||||||
|
* const isReserved = (key: string) =>
|
||||||
|
* (RESERVED_ENTITY_FIELDS as readonly string[]).includes(key)
|
||||||
|
*/
|
||||||
|
export const RESERVED_ENTITY_FIELDS = [
|
||||||
|
'noun',
|
||||||
|
'subtype',
|
||||||
|
'createdAt',
|
||||||
|
'updatedAt',
|
||||||
|
'confidence',
|
||||||
|
'weight',
|
||||||
|
'service',
|
||||||
|
'data',
|
||||||
|
'createdBy',
|
||||||
|
'_rev'
|
||||||
|
] as const
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Union of the entity field names reserved by Brainy — the
|
||||||
|
* element type of {@link RESERVED_ENTITY_FIELDS}.
|
||||||
|
*/
|
||||||
|
export type ReservedEntityField = (typeof RESERVED_ENTITY_FIELDS)[number]
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Relationship (verb) field names reserved by Brainy — the verb
|
||||||
|
* mirror of {@link RESERVED_ENTITY_FIELDS}. The stored flat record keys the
|
||||||
|
* relationship type under `verb` (the public `Relation` field is `type`);
|
||||||
|
* everything else matches the entity list.
|
||||||
|
*
|
||||||
|
* | Key | Canonical write path |
|
||||||
|
* |-----|----------------------|
|
||||||
|
* | `verb` | the `type` param of `relate()` / `updateRelation()` (stored under the key `verb`) |
|
||||||
|
* | `subtype` | the `subtype` param |
|
||||||
|
* | `createdAt` | system-managed — set once at `relate()` time |
|
||||||
|
* | `updatedAt` | system-managed — set on every write |
|
||||||
|
* | `confidence` | the `confidence` param |
|
||||||
|
* | `weight` | the `weight` param |
|
||||||
|
* | `service` | the `service` param of `relate()` (immutable afterwards) |
|
||||||
|
* | `data` | the `data` param |
|
||||||
|
* | `createdBy` | system-managed |
|
||||||
|
* | `_rev` | system-managed |
|
||||||
|
*/
|
||||||
|
export const RESERVED_RELATION_FIELDS = [
|
||||||
|
'verb',
|
||||||
|
'subtype',
|
||||||
|
'createdAt',
|
||||||
|
'updatedAt',
|
||||||
|
'confidence',
|
||||||
|
'weight',
|
||||||
|
'service',
|
||||||
|
'data',
|
||||||
|
'createdBy',
|
||||||
|
'_rev'
|
||||||
|
] as const
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Union of the relationship field names reserved by Brainy —
|
||||||
|
* the element type of {@link RESERVED_RELATION_FIELDS}.
|
||||||
|
*/
|
||||||
|
export type ReservedRelationField = (typeof RESERVED_RELATION_FIELDS)[number]
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description `true` when `T` is exactly `any` (the classic
|
||||||
|
* `0 extends 1 & T` probe — only `any` absorbs the impossible intersection).
|
||||||
|
* Used to keep the reserved-key guard active for untyped brains, where a
|
||||||
|
* plain `T & guard` intersection would collapse to `any` and check nothing.
|
||||||
|
*/
|
||||||
|
type IsAny<T> = 0 extends 1 & T ? true : false
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Compile-time tripwire: marks every reserved entity key as
|
||||||
|
* `never` so an object literal carrying one fails to type-check. Keys that
|
||||||
|
* `T` itself declares (including via an index signature, where
|
||||||
|
* `keyof T = string`) are exempted — a consumer who *explicitly* types a
|
||||||
|
* reserved key into their metadata shape keeps a working (if unwise) type,
|
||||||
|
* and index-signature metadata types remain assignable.
|
||||||
|
*/
|
||||||
|
export type NoReservedEntityKeys<T> = {
|
||||||
|
readonly [K in ReservedEntityField as K extends keyof T ? never : K]?: never
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Relationship mirror of {@link NoReservedEntityKeys}.
|
||||||
|
*/
|
||||||
|
export type NoReservedRelationKeys<T> = {
|
||||||
|
readonly [K in ReservedRelationField as K extends keyof T ? never : K]?: never
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description The metadata bag shape for untyped brains (`T = any`): an
|
||||||
|
* open index signature (any custom key, any value — exactly the pre-8.0
|
||||||
|
* latitude) intersected with the reserved-key guard, whose declared
|
||||||
|
* `?: never` properties take precedence over the index signature so a
|
||||||
|
* literal reserved key is still a compile error.
|
||||||
|
*/
|
||||||
|
type OpenBag<Guard> = { [key: string]: any } & Guard
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description The type of `AddParams.metadata`: the consumer's metadata
|
||||||
|
* shape `T` with reserved entity keys forbidden at compile time. For untyped
|
||||||
|
* brains (`T = any`) the bag stays open ({@link OpenBag}), so arbitrary
|
||||||
|
* custom fields remain legal while literal reserved keys still error.
|
||||||
|
*/
|
||||||
|
export type EntityMetadataInput<T> = IsAny<T> extends true
|
||||||
|
? OpenBag<NoReservedEntityKeys<object>>
|
||||||
|
: T & NoReservedEntityKeys<T>
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description The type of `UpdateParams.metadata`: a partial patch of the
|
||||||
|
* consumer's metadata shape with reserved entity keys forbidden at compile
|
||||||
|
* time. Same `T = any` handling as {@link EntityMetadataInput}.
|
||||||
|
*/
|
||||||
|
export type EntityMetadataPatch<T> = IsAny<T> extends true
|
||||||
|
? OpenBag<NoReservedEntityKeys<object>>
|
||||||
|
: Partial<T> & NoReservedEntityKeys<T>
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description The type of `RelateParams.metadata`: the consumer's edge
|
||||||
|
* metadata shape with reserved relationship keys forbidden at compile time.
|
||||||
|
*/
|
||||||
|
export type RelationMetadataInput<T> = IsAny<T> extends true
|
||||||
|
? OpenBag<NoReservedRelationKeys<object>>
|
||||||
|
: T & NoReservedRelationKeys<T>
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description The type of `UpdateRelationParams.metadata`: a partial patch
|
||||||
|
* of the consumer's edge metadata shape with reserved relationship keys
|
||||||
|
* forbidden at compile time.
|
||||||
|
*/
|
||||||
|
export type RelationMetadataPatch<T> = IsAny<T> extends true
|
||||||
|
? OpenBag<NoReservedRelationKeys<object>>
|
||||||
|
: Partial<T> & NoReservedRelationKeys<T>
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Result of splitting a stored flat metadata record into its
|
||||||
|
* reserved (Brainy-owned) and custom (consumer-owned) halves.
|
||||||
|
*/
|
||||||
|
export interface SplitMetadataRecord<F extends string> {
|
||||||
|
/** The reserved fields present in the record, keyed by reserved name. */
|
||||||
|
reserved: Partial<Record<F, unknown>>
|
||||||
|
/** Every other key — the consumer's custom metadata, and nothing else. */
|
||||||
|
custom: Record<string, unknown>
|
||||||
|
}
|
||||||
|
|
||||||
|
const RESERVED_ENTITY_SET: ReadonlySet<string> = new Set(RESERVED_ENTITY_FIELDS)
|
||||||
|
const RESERVED_RELATION_SET: ReadonlySet<string> = new Set(RESERVED_RELATION_FIELDS)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Shared splitter — partitions a record's keys against a
|
||||||
|
* reserved-name set. `null`/`undefined` records split to two empty objects.
|
||||||
|
* @param record - The stored flat metadata record (reserved + custom keys mixed).
|
||||||
|
* @param reservedSet - The reserved-name set to partition against.
|
||||||
|
* @returns The `{ reserved, custom }` halves.
|
||||||
|
*/
|
||||||
|
function splitRecord<F extends string>(
|
||||||
|
record: Record<string, unknown> | null | undefined,
|
||||||
|
reservedSet: ReadonlySet<string>
|
||||||
|
): SplitMetadataRecord<F> {
|
||||||
|
const reserved: Record<string, unknown> = {}
|
||||||
|
const custom: Record<string, unknown> = {}
|
||||||
|
if (record && typeof record === 'object') {
|
||||||
|
for (const [key, value] of Object.entries(record)) {
|
||||||
|
if (reservedSet.has(key)) {
|
||||||
|
reserved[key] = value
|
||||||
|
} else {
|
||||||
|
custom[key] = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { reserved: reserved as Partial<Record<F, unknown>>, custom }
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Split a stored entity (noun) flat metadata record into
|
||||||
|
* reserved fields and custom metadata — THE canonical read-side split. Every
|
||||||
|
* entity read path (live `get()`, batch reads, paginated listings, and
|
||||||
|
* historical `asOf()` materialization) goes through this function, so the
|
||||||
|
* reserved list can never drift between read paths.
|
||||||
|
* @param record - The stored flat metadata record.
|
||||||
|
* @returns `reserved` (Brainy-owned fields) and `custom` (the consumer's metadata bag).
|
||||||
|
* @example
|
||||||
|
* const { reserved, custom } = splitNounMetadataRecord(stored)
|
||||||
|
* // reserved.noun → entity.type, reserved.confidence → entity.confidence, …
|
||||||
|
* // custom → entity.metadata (custom fields only, always)
|
||||||
|
*/
|
||||||
|
export function splitNounMetadataRecord(
|
||||||
|
record: Record<string, unknown> | null | undefined
|
||||||
|
): SplitMetadataRecord<ReservedEntityField> {
|
||||||
|
return splitRecord(record, RESERVED_ENTITY_SET)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Split a stored relationship (verb) flat metadata record into
|
||||||
|
* reserved fields and custom metadata — the verb mirror of
|
||||||
|
* {@link splitNounMetadataRecord}, used by every relationship read path.
|
||||||
|
* @param record - The stored flat metadata record.
|
||||||
|
* @returns `reserved` (Brainy-owned fields) and `custom` (the consumer's metadata bag).
|
||||||
|
*/
|
||||||
|
export function splitVerbMetadataRecord(
|
||||||
|
record: Record<string, unknown> | null | undefined
|
||||||
|
): SplitMetadataRecord<ReservedRelationField> {
|
||||||
|
return splitRecord(record, RESERVED_RELATION_SET)
|
||||||
|
}
|
||||||
10
tests/configs/tsconfig.typecheck.json
Normal file
10
tests/configs/tsconfig.typecheck.json
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
{
|
||||||
|
"extends": "../../tsconfig.json",
|
||||||
|
"compilerOptions": {
|
||||||
|
"noEmit": true,
|
||||||
|
"declaration": false,
|
||||||
|
"sourceMap": false,
|
||||||
|
"rootDir": "../.."
|
||||||
|
},
|
||||||
|
"include": ["../unit/types/**/*.test-d.ts", "../../src/**/*.d.ts"]
|
||||||
|
}
|
||||||
|
|
@ -34,6 +34,16 @@ export default defineConfig({
|
||||||
'node_modules/**'
|
'node_modules/**'
|
||||||
],
|
],
|
||||||
|
|
||||||
|
// Compile-time tests (*.test-d.ts) — validates the reserved-metadata-key
|
||||||
|
// guard (@ts-expect-error assertions in tests/unit/types/) with tsc on
|
||||||
|
// every unit run. See src/types/reservedFields.ts for the contract.
|
||||||
|
typecheck: {
|
||||||
|
enabled: true,
|
||||||
|
checker: 'tsc',
|
||||||
|
include: ['tests/unit/types/**/*.test-d.ts'],
|
||||||
|
tsconfig: './tests/configs/tsconfig.typecheck.json'
|
||||||
|
},
|
||||||
|
|
||||||
// Use 'forks' for process isolation — 'threads' causes vitest internal
|
// Use 'forks' for process isolation — 'threads' causes vitest internal
|
||||||
// "Timeout calling onTaskUpdate" RPC errors under heavy parallel load
|
// "Timeout calling onTaskUpdate" RPC errors under heavy parallel load
|
||||||
pool: 'forks',
|
pool: 'forks',
|
||||||
|
|
|
||||||
393
tests/unit/brainy/update-reserved-metadata-remap.test.ts
Normal file
393
tests/unit/brainy/update-reserved-metadata-remap.test.ts
Normal file
|
|
@ -0,0 +1,393 @@
|
||||||
|
/**
|
||||||
|
* @module tests/unit/brainy/update-reserved-metadata-remap
|
||||||
|
* @description Regression tests for the reserved-field metadata-bag trap,
|
||||||
|
* ported from the 7.x fix and extended to the full 8.0 contract.
|
||||||
|
*
|
||||||
|
* History: `add({metadata: {confidence}})` lifted reserved fields to their
|
||||||
|
* canonical top-level location, but `update({metadata: {confidence}})`
|
||||||
|
* silently dropped the same shape — the patch value survived the merge and
|
||||||
|
* was then clobbered by the preserve-existing spread. A production
|
||||||
|
* consumer's confidence-evolution writes no-oped for weeks before being
|
||||||
|
* caught by reading values back.
|
||||||
|
*
|
||||||
|
* 8.0 contract under test (every write path, entities AND relationships):
|
||||||
|
* - user-mutable reserved fields (`confidence`, `weight`, `subtype` — plus
|
||||||
|
* `service`/`createdBy` at add()/relate() time) remap from the metadata
|
||||||
|
* bag to their dedicated top-level param, with top-level winning when both
|
||||||
|
* are present;
|
||||||
|
* - system-managed reserved fields (`createdAt`, `_rev`, `noun`/`verb`,
|
||||||
|
* `data`, …) are dropped from the bag (one-shot warning);
|
||||||
|
* - the same normalization applies to `transact()` operations and `with()`
|
||||||
|
* speculative views;
|
||||||
|
* - reads NEVER echo a reserved field inside `metadata`.
|
||||||
|
*
|
||||||
|
* TypeScript callers can't write these shapes at all (compile-time guard on
|
||||||
|
* the metadata param types — see tests/unit/types/reserved-metadata-keys.test-d.ts);
|
||||||
|
* these tests simulate untyped (JavaScript) callers, hence the `as object`
|
||||||
|
* widenings on the metadata literals.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
|
||||||
|
import { Brainy } from '../../../src/index.js'
|
||||||
|
import { NounType, VerbType } from '../../../src/types/graphTypes.js'
|
||||||
|
import { createTestConfig } from '../../helpers/test-factory.js'
|
||||||
|
|
||||||
|
describe('reserved-field metadata remap (8.0 contract)', () => {
|
||||||
|
let brain: Brainy
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
brain = new Brainy(createTestConfig())
|
||||||
|
await brain.init()
|
||||||
|
})
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('update() — the ported 7.x regression', () => {
|
||||||
|
it('remaps metadata.confidence to the top-level field (the production repro)', async () => {
|
||||||
|
const id = await brain.add({
|
||||||
|
type: NounType.Concept,
|
||||||
|
subtype: 'general',
|
||||||
|
data: 'x',
|
||||||
|
metadata: { confidence: 0.8 } as object
|
||||||
|
})
|
||||||
|
|
||||||
|
// Top-level write works (always did)
|
||||||
|
await brain.update({ id, confidence: 0.42 })
|
||||||
|
let entity = await brain.get(id)
|
||||||
|
expect(entity?.confidence).toBe(0.42)
|
||||||
|
|
||||||
|
// Metadata-patch write — silently dropped pre-fix, remapped now
|
||||||
|
await brain.update({ id, metadata: { confidence: 0.33 } as object })
|
||||||
|
entity = await brain.get(id)
|
||||||
|
expect(entity?.confidence).toBe(0.33)
|
||||||
|
// The reserved key must not linger inside the metadata bag
|
||||||
|
expect((entity?.metadata as Record<string, unknown>)?.confidence).toBeUndefined()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('remaps metadata.weight and metadata.subtype the same way', async () => {
|
||||||
|
const id = await brain.add({
|
||||||
|
type: NounType.Concept,
|
||||||
|
subtype: 'general',
|
||||||
|
data: 'y',
|
||||||
|
metadata: {}
|
||||||
|
})
|
||||||
|
|
||||||
|
await brain.update({ id, metadata: { weight: 0.7, subtype: 'specialized' } as object })
|
||||||
|
const entity = await brain.get(id)
|
||||||
|
expect(entity?.weight).toBe(0.7)
|
||||||
|
expect(entity?.subtype).toBe('specialized')
|
||||||
|
expect((entity?.metadata as Record<string, unknown>)?.weight).toBeUndefined()
|
||||||
|
expect((entity?.metadata as Record<string, unknown>)?.subtype).toBeUndefined()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('top-level param wins when both top-level and metadata-patch carry the field', async () => {
|
||||||
|
const id = await brain.add({
|
||||||
|
type: NounType.Concept,
|
||||||
|
subtype: 'general',
|
||||||
|
data: 'z',
|
||||||
|
metadata: { confidence: 0.5 } as object
|
||||||
|
})
|
||||||
|
|
||||||
|
await brain.update({ id, confidence: 0.9, metadata: { confidence: 0.1 } as object })
|
||||||
|
const entity = await brain.get(id)
|
||||||
|
expect(entity?.confidence).toBe(0.9)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('drops system-managed fields from patches without corrupting the entity', async () => {
|
||||||
|
const id = await brain.add({
|
||||||
|
type: NounType.Concept,
|
||||||
|
subtype: 'general',
|
||||||
|
data: 'w',
|
||||||
|
metadata: { keep: 'me' }
|
||||||
|
})
|
||||||
|
const before = await brain.get(id)
|
||||||
|
|
||||||
|
await brain.update({
|
||||||
|
id,
|
||||||
|
metadata: { createdAt: 1, _rev: 999, noun: 'organization', other: 'applied' } as object
|
||||||
|
})
|
||||||
|
const after = await brain.get(id)
|
||||||
|
|
||||||
|
expect(after?.createdAt).toBe(before?.createdAt) // immutable
|
||||||
|
expect(after?.type).toBe('concept') // noun patch ignored
|
||||||
|
expect(after?._rev).toBe((before?._rev ?? 1) + 1) // _rev patch ignored; normal bump applied
|
||||||
|
expect((after?.metadata as Record<string, unknown>)?.other).toBe('applied') // custom fields still merge
|
||||||
|
expect((after?.metadata as Record<string, unknown>)?.keep).toBe('me')
|
||||||
|
expect((after?.metadata as Record<string, unknown>)?._rev).toBeUndefined()
|
||||||
|
expect((after?.metadata as Record<string, unknown>)?.createdAt).toBeUndefined()
|
||||||
|
expect((after?.metadata as Record<string, unknown>)?.noun).toBeUndefined()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('custom (non-reserved) metadata patches are unaffected by the remap', async () => {
|
||||||
|
const id = await brain.add({
|
||||||
|
type: NounType.Concept,
|
||||||
|
subtype: 'general',
|
||||||
|
data: 'v',
|
||||||
|
metadata: { status: 'draft' }
|
||||||
|
})
|
||||||
|
|
||||||
|
await brain.update({ id, metadata: { status: 'reviewed', rating: 4.5 } })
|
||||||
|
const entity = await brain.get(id)
|
||||||
|
expect((entity?.metadata as Record<string, unknown>)?.status).toBe('reviewed')
|
||||||
|
expect((entity?.metadata as Record<string, unknown>)?.rating).toBe(4.5)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('add() — explicit lift, identical contract', () => {
|
||||||
|
it('lifts confidence/weight/subtype out of the bag to top level', async () => {
|
||||||
|
const id = await brain.add({
|
||||||
|
type: NounType.Person,
|
||||||
|
data: 'lift check',
|
||||||
|
metadata: { confidence: 0.8, weight: 0.6, subtype: 'employee', dept: 'eng' } as object
|
||||||
|
})
|
||||||
|
|
||||||
|
const entity = await brain.get(id)
|
||||||
|
expect(entity?.confidence).toBe(0.8)
|
||||||
|
expect(entity?.weight).toBe(0.6)
|
||||||
|
expect(entity?.subtype).toBe('employee')
|
||||||
|
expect(entity?.metadata).toEqual({ dept: 'eng' })
|
||||||
|
})
|
||||||
|
|
||||||
|
it('lifts service (settable at add time) and lets the top-level param win', async () => {
|
||||||
|
const lifted = await brain.add({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'service lift',
|
||||||
|
metadata: { service: 'orders' } as object
|
||||||
|
})
|
||||||
|
expect((await brain.get(lifted))?.service).toBe('orders')
|
||||||
|
|
||||||
|
const topLevelWins = await brain.add({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'service precedence',
|
||||||
|
service: 'billing',
|
||||||
|
metadata: { service: 'orders' } as object
|
||||||
|
})
|
||||||
|
const entity = await brain.get(topLevelWins)
|
||||||
|
expect(entity?.service).toBe('billing')
|
||||||
|
expect((entity?.metadata as Record<string, unknown>)?.service).toBeUndefined()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('a remapped subtype satisfies subtype enforcement like a top-level one', async () => {
|
||||||
|
brain.requireSubtype(NounType.Document)
|
||||||
|
|
||||||
|
// Top-level missing, but the bag carries it — must not throw.
|
||||||
|
const id = await brain.add({
|
||||||
|
type: NounType.Document,
|
||||||
|
data: 'enforcement via remap',
|
||||||
|
metadata: { subtype: 'invoice' } as object
|
||||||
|
})
|
||||||
|
expect((await brain.get(id))?.subtype).toBe('invoice')
|
||||||
|
|
||||||
|
// Neither place carries it — must throw.
|
||||||
|
await expect(
|
||||||
|
brain.add({ type: NounType.Document, data: 'no subtype anywhere' })
|
||||||
|
).rejects.toThrow(/subtype/)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('transact() — same remap on add and update ops', () => {
|
||||||
|
it('normalizes reserved fields in transact add + update ops', async () => {
|
||||||
|
const db1 = await brain.transact([
|
||||||
|
{
|
||||||
|
op: 'add',
|
||||||
|
type: NounType.Concept,
|
||||||
|
subtype: 'general',
|
||||||
|
data: 'tx',
|
||||||
|
metadata: { confidence: 0.7, custom: 'a' } as object
|
||||||
|
}
|
||||||
|
])
|
||||||
|
const id = db1.receipt!.ids[0]
|
||||||
|
|
||||||
|
let entity = await brain.get(id)
|
||||||
|
expect(entity?.confidence).toBe(0.7)
|
||||||
|
expect(entity?.metadata).toEqual({ custom: 'a' })
|
||||||
|
|
||||||
|
await brain.transact([
|
||||||
|
{ op: 'update', id, metadata: { confidence: 0.25, custom: 'b' } as object }
|
||||||
|
])
|
||||||
|
entity = await brain.get(id)
|
||||||
|
expect(entity?.confidence).toBe(0.25)
|
||||||
|
expect(entity?.metadata).toEqual({ custom: 'b' })
|
||||||
|
expect((entity?.metadata as Record<string, unknown>)?.confidence).toBeUndefined()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('historical asOf() reads surface reserved fields ONLY top-level', async () => {
|
||||||
|
const db1 = await brain.transact([
|
||||||
|
{
|
||||||
|
op: 'add',
|
||||||
|
type: NounType.Concept,
|
||||||
|
subtype: 'general',
|
||||||
|
data: 'historical',
|
||||||
|
metadata: { confidence: 0.9, custom: 'past' } as object
|
||||||
|
}
|
||||||
|
])
|
||||||
|
const id = db1.receipt!.ids[0]
|
||||||
|
|
||||||
|
// Move the world forward so generation db1 is historical.
|
||||||
|
await brain.transact([{ op: 'update', id, confidence: 0.1, metadata: { custom: 'now' } }])
|
||||||
|
|
||||||
|
const past = await brain.asOf(db1.generation)
|
||||||
|
const historical = await past.get(id)
|
||||||
|
expect(historical?.confidence).toBe(0.9)
|
||||||
|
expect(historical?.metadata).toEqual({ custom: 'past' })
|
||||||
|
await past.release()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('with() speculative views apply the same normalization', async () => {
|
||||||
|
const base = await brain.now()
|
||||||
|
const speculative = await base.with([
|
||||||
|
{
|
||||||
|
op: 'add',
|
||||||
|
id: 'spec-entity',
|
||||||
|
type: NounType.Concept,
|
||||||
|
subtype: 'general',
|
||||||
|
data: 'spec',
|
||||||
|
metadata: { confidence: 0.65, custom: 'spec' } as object
|
||||||
|
}
|
||||||
|
])
|
||||||
|
|
||||||
|
const entity = await speculative.get('spec-entity')
|
||||||
|
expect(entity?.confidence).toBe(0.65)
|
||||||
|
expect(entity?.metadata).toEqual({ custom: 'spec' })
|
||||||
|
await speculative.release()
|
||||||
|
await base.release()
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('read paths never echo reserved fields inside metadata', () => {
|
||||||
|
it('find() (storage pagination path) returns custom-only metadata with reserved fields top-level', async () => {
|
||||||
|
const id = await brain.add({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'pagination echo check',
|
||||||
|
confidence: 0.8,
|
||||||
|
weight: 0.6,
|
||||||
|
metadata: { dept: 'eng' }
|
||||||
|
})
|
||||||
|
|
||||||
|
// No query/filter → served by the direct storage pagination path
|
||||||
|
// (getNounsWithPagination), which historically echoed the full flat
|
||||||
|
// record (noun/subtype/createdAt/… inside metadata).
|
||||||
|
const results = await brain.find({ limit: 50 })
|
||||||
|
const result = results.find((r) => r.id === id)
|
||||||
|
expect(result).toBeDefined()
|
||||||
|
expect(result?.entity.metadata).toEqual({ dept: 'eng' })
|
||||||
|
expect(result?.entity.type).toBe(NounType.Person)
|
||||||
|
expect(result?.entity.subtype).toBe('employee')
|
||||||
|
expect(result?.entity.confidence).toBe(0.8)
|
||||||
|
expect(result?.entity.weight).toBe(0.6)
|
||||||
|
expect(typeof result?.entity.createdAt).toBe('number')
|
||||||
|
expect(result?.entity._rev).toBe(1)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('getRelations() by target surfaces reserved fields top-level, custom-only metadata', async () => {
|
||||||
|
const a = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'src' })
|
||||||
|
const b = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'tgt' })
|
||||||
|
const relId = await brain.relate({
|
||||||
|
from: a,
|
||||||
|
to: b,
|
||||||
|
type: VerbType.ReportsTo,
|
||||||
|
subtype: 'direct',
|
||||||
|
confidence: 0.9,
|
||||||
|
weight: 0.5,
|
||||||
|
service: 'orders',
|
||||||
|
metadata: { note: 'target path' }
|
||||||
|
})
|
||||||
|
|
||||||
|
const relations = await brain.getRelations({ to: b })
|
||||||
|
const rel = relations.find((r) => r.id === relId)
|
||||||
|
expect(rel).toBeDefined()
|
||||||
|
expect(rel?.metadata).toEqual({ note: 'target path' })
|
||||||
|
expect(rel?.subtype).toBe('direct')
|
||||||
|
expect(rel?.confidence).toBe(0.9)
|
||||||
|
expect(rel?.weight).toBe(0.5)
|
||||||
|
expect(rel?.service).toBe('orders')
|
||||||
|
expect(typeof rel?.createdAt).toBe('number')
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('relationships — relate() / updateRelation() mirror', () => {
|
||||||
|
let a: string
|
||||||
|
let b: string
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
a = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'A' })
|
||||||
|
b = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'B' })
|
||||||
|
})
|
||||||
|
|
||||||
|
it('relate() persists the top-level confidence and service params', async () => {
|
||||||
|
const relId = await brain.relate({
|
||||||
|
from: a,
|
||||||
|
to: b,
|
||||||
|
type: VerbType.ReportsTo,
|
||||||
|
subtype: 'direct',
|
||||||
|
confidence: 0.77,
|
||||||
|
service: 'orders'
|
||||||
|
})
|
||||||
|
|
||||||
|
const relations = await brain.getRelations({ from: a })
|
||||||
|
const rel = relations.find((r) => r.id === relId)
|
||||||
|
expect(rel?.confidence).toBe(0.77)
|
||||||
|
expect(rel?.service).toBe('orders')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('relate() remaps reserved fields out of the metadata bag', async () => {
|
||||||
|
const relId = await brain.relate({
|
||||||
|
from: a,
|
||||||
|
to: b,
|
||||||
|
type: VerbType.RelatedTo,
|
||||||
|
subtype: 'colleague',
|
||||||
|
metadata: { confidence: 0.4, weight: 0.3, role: 'peer' } as object
|
||||||
|
})
|
||||||
|
|
||||||
|
const relations = await brain.getRelations({ from: a })
|
||||||
|
const rel = relations.find((r) => r.id === relId)
|
||||||
|
expect(rel?.confidence).toBe(0.4)
|
||||||
|
expect(rel?.weight).toBe(0.3)
|
||||||
|
expect(rel?.metadata).toEqual({ role: 'peer' })
|
||||||
|
})
|
||||||
|
|
||||||
|
it('relation.metadata never echoes the verb type key', async () => {
|
||||||
|
const relId = await brain.relate({
|
||||||
|
from: a,
|
||||||
|
to: b,
|
||||||
|
type: VerbType.RelatedTo,
|
||||||
|
subtype: 'colleague',
|
||||||
|
metadata: { note: 'no echo' }
|
||||||
|
})
|
||||||
|
|
||||||
|
const relations = await brain.getRelations({ from: a })
|
||||||
|
const rel = relations.find((r) => r.id === relId)
|
||||||
|
expect(rel?.type).toBe(VerbType.RelatedTo)
|
||||||
|
expect((rel?.metadata as Record<string, unknown>)?.verb).toBeUndefined()
|
||||||
|
expect(rel?.metadata).toEqual({ note: 'no echo' })
|
||||||
|
})
|
||||||
|
|
||||||
|
it('updateRelation() remaps the user-mutable trio and preserves service', async () => {
|
||||||
|
const relId = await brain.relate({
|
||||||
|
from: a,
|
||||||
|
to: b,
|
||||||
|
type: VerbType.ReportsTo,
|
||||||
|
subtype: 'direct',
|
||||||
|
service: 'orders',
|
||||||
|
metadata: { keep: 'me' }
|
||||||
|
})
|
||||||
|
|
||||||
|
await brain.updateRelation({
|
||||||
|
id: relId,
|
||||||
|
metadata: { confidence: 0.55, subtype: 'dotted-line', extra: 'applied' } as object
|
||||||
|
})
|
||||||
|
|
||||||
|
const relations = await brain.getRelations({ from: a })
|
||||||
|
const rel = relations.find((r) => r.id === relId)
|
||||||
|
expect(rel?.confidence).toBe(0.55)
|
||||||
|
expect(rel?.subtype).toBe('dotted-line')
|
||||||
|
expect(rel?.service).toBe('orders') // fixed at relate() time, never erased by updates
|
||||||
|
expect(rel?.metadata).toEqual({ keep: 'me', extra: 'applied' })
|
||||||
|
})
|
||||||
|
})
|
||||||
|
})
|
||||||
265
tests/unit/types/reserved-metadata-keys.test-d.ts
Normal file
265
tests/unit/types/reserved-metadata-keys.test-d.ts
Normal file
|
|
@ -0,0 +1,265 @@
|
||||||
|
/**
|
||||||
|
* @module tests/unit/types/reserved-metadata-keys.test-d
|
||||||
|
* @description Compile-time tests for the reserved-field contract (layer 1 of
|
||||||
|
* three — see src/types/reservedFields.ts): a literal reserved key inside any
|
||||||
|
* `metadata` param is a TypeScript error, while the generic `T` ergonomics
|
||||||
|
* stay intact (typed bags, untyped brains, index-signature shapes, and the
|
||||||
|
* documented exemption for consumers who explicitly declare a reserved key in
|
||||||
|
* their own metadata type).
|
||||||
|
*
|
||||||
|
* Runs under vitest typecheck mode (`test.typecheck` in
|
||||||
|
* tests/configs/vitest.unit.config.ts) — these assertions are validated by
|
||||||
|
* `tsc`, never executed. The runtime half of the contract (the write-path
|
||||||
|
* remap for untyped callers) is pinned by
|
||||||
|
* tests/unit/brainy/update-reserved-metadata-remap.test.ts.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { describe, it, assertType } from 'vitest'
|
||||||
|
import type {
|
||||||
|
AddParams,
|
||||||
|
UpdateParams,
|
||||||
|
RelateParams,
|
||||||
|
UpdateRelationParams,
|
||||||
|
TxOperation
|
||||||
|
} from '../../../src/index.js'
|
||||||
|
import { NounType, VerbType } from '../../../src/types/graphTypes.js'
|
||||||
|
|
||||||
|
describe('reserved entity keys in metadata are compile errors', () => {
|
||||||
|
it('AddParams (untyped brain) rejects every reserved key but stays open for custom fields', () => {
|
||||||
|
// Custom fields of any shape remain legal — exactly the pre-8.0 latitude.
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
metadata: { dept: 'eng', level: 3, tags: ['a', 'b'], nested: { ok: true } }
|
||||||
|
})
|
||||||
|
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — 'noun' is reserved (the entity type travels via the top-level 'type' param)
|
||||||
|
metadata: { noun: 'organization' }
|
||||||
|
})
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — 'subtype' is reserved (use the top-level 'subtype' param)
|
||||||
|
metadata: { subtype: 'contractor' }
|
||||||
|
})
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — 'createdAt' is reserved (system-managed)
|
||||||
|
metadata: { createdAt: Date.now() }
|
||||||
|
})
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — 'updatedAt' is reserved (system-managed)
|
||||||
|
metadata: { updatedAt: Date.now() }
|
||||||
|
})
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — 'confidence' is reserved (use the top-level 'confidence' param)
|
||||||
|
metadata: { confidence: 0.8 }
|
||||||
|
})
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — 'weight' is reserved (use the top-level 'weight' param)
|
||||||
|
metadata: { weight: 0.5 }
|
||||||
|
})
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — 'service' is reserved (use the top-level 'service' param)
|
||||||
|
metadata: { service: 'orders' }
|
||||||
|
})
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — 'data' is reserved (use the top-level 'data' param)
|
||||||
|
metadata: { data: 'content' }
|
||||||
|
})
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — 'createdBy' is reserved (use the top-level 'createdBy' param)
|
||||||
|
metadata: { createdBy: { augmentation: 'importer', version: '1.0' } }
|
||||||
|
})
|
||||||
|
assertType<AddParams>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — '_rev' is reserved (system-managed revision counter)
|
||||||
|
metadata: { _rev: 7 }
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
it('AddParams<T> (typed brain) rejects reserved keys alongside the declared shape', () => {
|
||||||
|
interface EmployeeMeta {
|
||||||
|
dept: string
|
||||||
|
level: number
|
||||||
|
}
|
||||||
|
|
||||||
|
assertType<AddParams<EmployeeMeta>>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
metadata: { dept: 'eng', level: 3 }
|
||||||
|
})
|
||||||
|
|
||||||
|
assertType<AddParams<EmployeeMeta>>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
// @ts-expect-error — 'confidence' is reserved even when T declares other fields
|
||||||
|
metadata: { dept: 'eng', level: 3, confidence: 0.8 }
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
it('documented exemptions: T-declared reserved keys and index-signature shapes stay assignable', () => {
|
||||||
|
// A consumer who *explicitly* types a reserved key into their metadata
|
||||||
|
// shape keeps a working (if unwise) type — the guard exempts keyof T.
|
||||||
|
interface LegacyMeta {
|
||||||
|
confidence: number
|
||||||
|
note: string
|
||||||
|
}
|
||||||
|
assertType<AddParams<LegacyMeta>>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
metadata: { confidence: 0.8, note: 'declared by the consumer type' }
|
||||||
|
})
|
||||||
|
|
||||||
|
// Index-signature metadata types (keyof T = string) remain fully open.
|
||||||
|
assertType<AddParams<Record<string, unknown>>>({
|
||||||
|
type: NounType.Person,
|
||||||
|
subtype: 'employee',
|
||||||
|
data: 'x',
|
||||||
|
metadata: { anything: 'goes', confidence: 0.8 }
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
it('UpdateParams patch rejects reserved keys but accepts partial custom patches', () => {
|
||||||
|
interface EmployeeMeta {
|
||||||
|
dept: string
|
||||||
|
level: number
|
||||||
|
}
|
||||||
|
|
||||||
|
// Partial patch of the declared shape is legal.
|
||||||
|
assertType<UpdateParams<EmployeeMeta>>({ id: 'e1', metadata: { dept: 'sales' } })
|
||||||
|
// Untyped patch with custom fields is legal.
|
||||||
|
assertType<UpdateParams>({ id: 'e1', metadata: { status: 'reviewed', rating: 4.5 } })
|
||||||
|
|
||||||
|
// @ts-expect-error — 'confidence' is reserved (use the top-level 'confidence' param)
|
||||||
|
assertType<UpdateParams>({ id: 'e1', metadata: { confidence: 0.33 } })
|
||||||
|
// @ts-expect-error — 'subtype' is reserved (use the top-level 'subtype' param)
|
||||||
|
assertType<UpdateParams>({ id: 'e1', metadata: { subtype: 'specialized' } })
|
||||||
|
// @ts-expect-error — '_rev' is reserved (pass 'ifRev' for optimistic concurrency)
|
||||||
|
assertType<UpdateParams>({ id: 'e1', metadata: { _rev: 3 } })
|
||||||
|
// @ts-expect-error — 'confidence' is reserved even when T declares other fields
|
||||||
|
assertType<UpdateParams<EmployeeMeta>>({ id: 'e1', metadata: { confidence: 0.1 } })
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('reserved relationship keys in metadata are compile errors', () => {
|
||||||
|
it('RelateParams rejects reserved keys but stays open for custom edge fields', () => {
|
||||||
|
assertType<RelateParams>({
|
||||||
|
from: 'a',
|
||||||
|
to: 'b',
|
||||||
|
type: VerbType.ReportsTo,
|
||||||
|
subtype: 'direct',
|
||||||
|
metadata: { role: 'peer', since: 2024 }
|
||||||
|
})
|
||||||
|
|
||||||
|
assertType<RelateParams>({
|
||||||
|
from: 'a',
|
||||||
|
to: 'b',
|
||||||
|
type: VerbType.ReportsTo,
|
||||||
|
subtype: 'direct',
|
||||||
|
// @ts-expect-error — 'verb' is reserved (the relationship type travels via the top-level 'type' param)
|
||||||
|
metadata: { verb: 'relatedTo' }
|
||||||
|
})
|
||||||
|
assertType<RelateParams>({
|
||||||
|
from: 'a',
|
||||||
|
to: 'b',
|
||||||
|
type: VerbType.ReportsTo,
|
||||||
|
subtype: 'direct',
|
||||||
|
// @ts-expect-error — 'confidence' is reserved (use the top-level 'confidence' param)
|
||||||
|
metadata: { confidence: 0.9 }
|
||||||
|
})
|
||||||
|
assertType<RelateParams>({
|
||||||
|
from: 'a',
|
||||||
|
to: 'b',
|
||||||
|
type: VerbType.ReportsTo,
|
||||||
|
subtype: 'direct',
|
||||||
|
// @ts-expect-error — 'weight' is reserved (use the top-level 'weight' param)
|
||||||
|
metadata: { weight: 0.4 }
|
||||||
|
})
|
||||||
|
assertType<RelateParams>({
|
||||||
|
from: 'a',
|
||||||
|
to: 'b',
|
||||||
|
type: VerbType.ReportsTo,
|
||||||
|
subtype: 'direct',
|
||||||
|
// @ts-expect-error — 'service' is reserved (use the top-level 'service' param)
|
||||||
|
metadata: { service: 'orders' }
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
it('UpdateRelationParams patch rejects reserved keys', () => {
|
||||||
|
assertType<UpdateRelationParams>({ id: 'r1', metadata: { note: 'fine' } })
|
||||||
|
|
||||||
|
// @ts-expect-error — 'confidence' is reserved (use the top-level 'confidence' param)
|
||||||
|
assertType<UpdateRelationParams>({ id: 'r1', metadata: { confidence: 0.5 } })
|
||||||
|
// @ts-expect-error — 'subtype' is reserved (use the top-level 'subtype' param)
|
||||||
|
assertType<UpdateRelationParams>({ id: 'r1', metadata: { subtype: 'dotted-line' } })
|
||||||
|
// @ts-expect-error — 'createdAt' is reserved (system-managed)
|
||||||
|
assertType<UpdateRelationParams>({ id: 'r1', metadata: { createdAt: 1 } })
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('transact() operations inherit the same guard', () => {
|
||||||
|
it('TxOperation add/update/relate metadata rejects reserved keys', () => {
|
||||||
|
assertType<TxOperation>({
|
||||||
|
op: 'add',
|
||||||
|
type: NounType.Concept,
|
||||||
|
subtype: 'general',
|
||||||
|
data: 'tx',
|
||||||
|
metadata: { custom: 'a' }
|
||||||
|
})
|
||||||
|
assertType<TxOperation>({
|
||||||
|
op: 'add',
|
||||||
|
type: NounType.Concept,
|
||||||
|
subtype: 'general',
|
||||||
|
data: 'tx',
|
||||||
|
// @ts-expect-error — 'confidence' is reserved on transact add ops too
|
||||||
|
metadata: { confidence: 0.7 }
|
||||||
|
})
|
||||||
|
assertType<TxOperation>({
|
||||||
|
op: 'update',
|
||||||
|
id: 'e1',
|
||||||
|
// @ts-expect-error — 'weight' is reserved on transact update ops too
|
||||||
|
metadata: { weight: 0.2 }
|
||||||
|
})
|
||||||
|
assertType<TxOperation>({
|
||||||
|
op: 'relate',
|
||||||
|
from: 'a',
|
||||||
|
to: 'b',
|
||||||
|
type: VerbType.RelatedTo,
|
||||||
|
subtype: 'colleague',
|
||||||
|
// @ts-expect-error — 'verb' is reserved on transact relate ops too
|
||||||
|
metadata: { verb: 'contains' }
|
||||||
|
})
|
||||||
|
})
|
||||||
|
})
|
||||||
Loading…
Add table
Add a link
Reference in a new issue