soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

feat(8.0): reserved-field contract — one canonical location, typed prevention, unified read/write

Browse source 
Brainy-owned field names (noun/verb, subtype, createdAt, updatedAt,
confidence, weight, service, data, createdBy, _rev) now have exactly one
home — top level — enforced by three layers driven from a single source of
truth, src/types/reservedFields.ts (RESERVED_ENTITY_FIELDS /
RESERVED_RELATION_FIELDS, exported):

1. Compile time — AddParams/UpdateParams/RelateParams/UpdateRelationParams
   metadata (and the transact() ops that extend them) reject a literal
   reserved key as a TypeScript error while keeping generic T ergonomics
   (typed bags, untyped brains, index-signature shapes, and a documented
   exemption for T-declared reserved keys). Pinned by @ts-expect-error
   type tests run under vitest typecheck mode on every unit run.

2. Write time — the 7.x update() remap is ported to 8.0 and extended to
   every write path: add/update/relate/updateRelation, their transact()
   mirrors, and db.with() overlays. User-settable fields lift to their
   dedicated param (top-level wins when both are supplied — closes the 7.x
   trap where update({metadata:{confidence}}) silently no-oped), and
   system-managed fields drop with a one-shot warning naming the right
   path. A remapped subtype satisfies subtype-pairing enforcement exactly
   like a top-level one.

3. Read time — every storage combine goes through one canonical hydration
   helper (hydrateNounWithMetadata / hydrateVerbWithMetadata over
   splitNoun/VerbMetadataRecord), so reserved fields surface ONLY top-level
   and entity/relation.metadata carry ONLY custom fields on live reads,
   batch reads, paginated listings, getRelations by source/target, streamed
   verbs, and historical asOf() materialization alike.

Read-path echoes found and fixed (previously the full stored record —
including the verb type key — leaked inside metadata): noun pagination,
verb pagination, getVerbsBySource/ByTarget (adjacency + shard fallback),
getVerbsBySourceBatch (which also dropped subtype/data), and the
filesystem verb stream. getRelations() results now surface
confidence/updatedAt top-level via verbsToRelations, updateRelation() no
longer erases service/createdBy, relate() persists its top-level
confidence/service params, and the dead convertHNSWVerbToGraphVerb echo
path is deleted. Import paths (CLI extract, deduplicator, coordinators,
neural import) write confidence through the dedicated param instead of the
bag. UpdateRelationParams is now exported from the package root.

Documented for consumers in docs/concepts/consistency-model.md ("Reserved
fields") and RELEASES.md. Regression tests ported from the 7.x fix and
extended to the full 8.0 contract (17 runtime tests + 41 type-level
assertions); full unit suite 1427/1427, db-mvcc integration 24/24.
This commit is contained in:
David Snelling 2026-06-11 13:12:50 -07:00
parent c44678390e
commit 970e08c466
18 changed files with 1688 additions and 452 deletions
Download patch file Download diff file Expand all files Collapse all files

48
src/types/brainy.types.ts
View file

@ -6,6 +6,12 @@
import { StorageAdapter, Vector } from '../coreTypes.js'
import { NounType, VerbType } from './graphTypes.js'
import type {
EntityMetadataInput,
EntityMetadataPatch,
RelationMetadataInput,
RelationMetadataPatch
} from './reservedFields.js'
// ============= Core Types =============
@ -283,8 +289,18 @@ export interface AddParams<T = any> {
* aggregation on the standard-field fast path.
*/
subtype?: string
/** Structured queryable fields — indexed by MetadataIndex, used in `where` filters */
metadata?: T
/**
* Structured queryable fields — indexed by MetadataIndex, used in `where` filters.
*
* Reserved entity fields (`RESERVED_ENTITY_FIELDS` — `noun`, `subtype`, `createdAt`,
* `updatedAt`, `confidence`, `weight`, `service`, `data`, `createdBy`, `_rev`) may NOT
* appear here — they have dedicated top-level params and the type makes a literal
* reserved key a compile error. Untyped (JavaScript) callers that pass one anyway are
* normalized at write time: user-settable fields remap to their top-level param
* (top-level wins when both are supplied), system-managed fields are dropped with a
* one-shot warning.
*/
metadata?: EntityMetadataInput<T>
/** Custom entity ID (auto-generated UUID v4 if not provided) */
id?: string
/** Pre-computed embedding vector (skips auto-embedding when provided) */
@ -314,7 +330,15 @@ export interface UpdateParams<T = any> {
data?: any // New content to re-embed
type?: NounType // Change type
subtype?: string // Change subtype (set to '' or null-equivalent via dedicated unset is future work)
metadata?: Partial<T> // Metadata to update
/**
* Metadata fields to merge (or replace when `merge: false`). Reserved entity
* fields (`RESERVED_ENTITY_FIELDS`) may NOT appear here — `confidence` /
* `weight` / `subtype` have dedicated params on this call, and the rest are
* system-managed. A literal reserved key is a compile error; untyped callers
* are normalized at write time (remap user-settable, drop system-managed
* with a one-shot warning).
*/
metadata?: EntityMetadataPatch<T>
merge?: boolean // Merge or replace metadata (default: true)
vector?: Vector // New pre-computed vector
confidence?: number // Update type classification confidence
@ -355,8 +379,14 @@ export interface RelateParams<T = any> {
weight?: number
/** Content for the relationship (optional — overrides auto-computed vector) */
data?: any
/** Structured queryable fields on the edge */
metadata?: T
/**
* Structured queryable fields on the edge. Reserved relationship fields
* (`RESERVED_RELATION_FIELDS` — `verb`, `subtype`, `createdAt`, `updatedAt`,
* `confidence`, `weight`, `service`, `data`, `createdBy`, `_rev`) may NOT
* appear here — they have dedicated params. A literal reserved key is a
* compile error; untyped callers are normalized at write time.
*/
metadata?: RelationMetadataInput<T>
/** Create reverse edge too (default: false) */
bidirectional?: boolean
/** Multi-tenancy service identifier */
@ -377,7 +407,13 @@ export interface UpdateRelationParams<T = any> {
weight?: number // New weight
confidence?: number // New confidence (0-1)
data?: any // New content
metadata?: Partial<T> // Metadata to update
/**
* Metadata fields to merge (or replace when `merge: false`). Reserved
* relationship fields (`RESERVED_RELATION_FIELDS`) may NOT appear here —
* a literal reserved key is a compile error; untyped callers are
* normalized at write time.
*/
metadata?: RelationMetadataPatch<T>
merge?: boolean // Merge or replace metadata
}

250
src/types/reservedFields.ts Normal file
View file

@ -0,0 +1,250 @@
/**
* @module types/reservedFields
* @description The canonical reserved-field contract — ONE place that defines
* which keys belong to Brainy (top-level entity/relationship fields) and may
* therefore never live inside a `metadata` bag.
*
* Three layers enforce the contract, all driven by the constants below:
*
* 1. **Compile time** — `AddParams.metadata`, `UpdateParams.metadata`,
* `RelateParams.metadata` and `UpdateRelationParams.metadata` are typed so
* a literal reserved key is a TypeScript error (see
* {@link EntityMetadataInput} / {@link RelationMetadataInput}).
* 2. **Write time** — for untyped (JavaScript) callers that smuggle a
* reserved key past the compiler anyway, every write path normalizes the
* bag: user-mutable fields are remapped to their dedicated top-level
* param (top-level wins when both are supplied) and system-managed fields
* are dropped with a one-shot warning naming the correct write path.
* 3. **Read time** — every read path splits the stored flat record through
* {@link splitNounMetadataRecord} / {@link splitVerbMetadataRecord}, so a
* reserved field is surfaced ONLY at top level and `entity.metadata` /
* `relation.metadata` contain ONLY custom fields, always — live reads,
* batch reads, and historical (`asOf`) reads alike.
*
* Documented for consumers in `docs/concepts/consistency-model.md`
* ("Reserved fields").
*/
/**
* @description Entity (noun) field names reserved by Brainy. These keys are
* stored in the flat per-entity metadata record alongside custom fields, but
* they belong to Brainy: every read path extracts them to top-level
* `Entity` fields, and no write path accepts them inside `metadata`.
*
* | Key | Canonical write path |
* |-----|----------------------|
* | `noun` | the `type` param of `add()` / `update()` (stored under the key `noun`) |
* | `subtype` | the `subtype` param |
* | `createdAt` | system-managed — set once at `add()` time |
* | `updatedAt` | system-managed — set on every write |
* | `confidence` | the `confidence` param |
* | `weight` | the `weight` param |
* | `service` | the `service` param of `add()` (immutable afterwards) |
* | `data` | the `data` param |
* | `createdBy` | the `createdBy` param of `add()` (immutable afterwards) |
* | `_rev` | system-managed revision counter — pass `ifRev` to `update()` for CAS |
*
* @example
* import { RESERVED_ENTITY_FIELDS } from '@soulcraft/brainy'
* const isReserved = (key: string) =>
* (RESERVED_ENTITY_FIELDS as readonly string[]).includes(key)
*/
export const RESERVED_ENTITY_FIELDS = [
'noun',
'subtype',
'createdAt',
'updatedAt',
'confidence',
'weight',
'service',
'data',
'createdBy',
'_rev'
] as const
/**
* @description Union of the entity field names reserved by Brainy — the
* element type of {@link RESERVED_ENTITY_FIELDS}.
*/
export type ReservedEntityField = (typeof RESERVED_ENTITY_FIELDS)[number]
/**
* @description Relationship (verb) field names reserved by Brainy — the verb
* mirror of {@link RESERVED_ENTITY_FIELDS}. The stored flat record keys the
* relationship type under `verb` (the public `Relation` field is `type`);
* everything else matches the entity list.
*
* | Key | Canonical write path |
* |-----|----------------------|
* | `verb` | the `type` param of `relate()` / `updateRelation()` (stored under the key `verb`) |
* | `subtype` | the `subtype` param |
* | `createdAt` | system-managed — set once at `relate()` time |
* | `updatedAt` | system-managed — set on every write |
* | `confidence` | the `confidence` param |
* | `weight` | the `weight` param |
* | `service` | the `service` param of `relate()` (immutable afterwards) |
* | `data` | the `data` param |
* | `createdBy` | system-managed |
* | `_rev` | system-managed |
*/
export const RESERVED_RELATION_FIELDS = [
'verb',
'subtype',
'createdAt',
'updatedAt',
'confidence',
'weight',
'service',
'data',
'createdBy',
'_rev'
] as const
/**
* @description Union of the relationship field names reserved by Brainy —
* the element type of {@link RESERVED_RELATION_FIELDS}.
*/
export type ReservedRelationField = (typeof RESERVED_RELATION_FIELDS)[number]
/**
* @description `true` when `T` is exactly `any` (the classic
* `0 extends 1 & T` probe — only `any` absorbs the impossible intersection).
* Used to keep the reserved-key guard active for untyped brains, where a
* plain `T & guard` intersection would collapse to `any` and check nothing.
*/
type IsAny<T> = 0 extends 1 & T ? true : false
/**
* @description Compile-time tripwire: marks every reserved entity key as
* `never` so an object literal carrying one fails to type-check. Keys that
* `T` itself declares (including via an index signature, where
* `keyof T = string`) are exempted — a consumer who *explicitly* types a
* reserved key into their metadata shape keeps a working (if unwise) type,
* and index-signature metadata types remain assignable.
*/
export type NoReservedEntityKeys<T> = {
readonly [K in ReservedEntityField as K extends keyof T ? never : K]?: never
}
/**
* @description Relationship mirror of {@link NoReservedEntityKeys}.
*/
export type NoReservedRelationKeys<T> = {
readonly [K in ReservedRelationField as K extends keyof T ? never : K]?: never
}
/**
* @description The metadata bag shape for untyped brains (`T = any`): an
* open index signature (any custom key, any value — exactly the pre-8.0
* latitude) intersected with the reserved-key guard, whose declared
* `?: never` properties take precedence over the index signature so a
* literal reserved key is still a compile error.
*/
type OpenBag<Guard> = { [key: string]: any } & Guard
/**
* @description The type of `AddParams.metadata`: the consumer's metadata
* shape `T` with reserved entity keys forbidden at compile time. For untyped
* brains (`T = any`) the bag stays open ({@link OpenBag}), so arbitrary
* custom fields remain legal while literal reserved keys still error.
*/
export type EntityMetadataInput<T> = IsAny<T> extends true
? OpenBag<NoReservedEntityKeys<object>>
: T & NoReservedEntityKeys<T>
/**
* @description The type of `UpdateParams.metadata`: a partial patch of the
* consumer's metadata shape with reserved entity keys forbidden at compile
* time. Same `T = any` handling as {@link EntityMetadataInput}.
*/
export type EntityMetadataPatch<T> = IsAny<T> extends true
? OpenBag<NoReservedEntityKeys<object>>
: Partial<T> & NoReservedEntityKeys<T>
/**
* @description The type of `RelateParams.metadata`: the consumer's edge
* metadata shape with reserved relationship keys forbidden at compile time.
*/
export type RelationMetadataInput<T> = IsAny<T> extends true
? OpenBag<NoReservedRelationKeys<object>>
: T & NoReservedRelationKeys<T>
/**
* @description The type of `UpdateRelationParams.metadata`: a partial patch
* of the consumer's edge metadata shape with reserved relationship keys
* forbidden at compile time.
*/
export type RelationMetadataPatch<T> = IsAny<T> extends true
? OpenBag<NoReservedRelationKeys<object>>
: Partial<T> & NoReservedRelationKeys<T>
/**
* @description Result of splitting a stored flat metadata record into its
* reserved (Brainy-owned) and custom (consumer-owned) halves.
*/
export interface SplitMetadataRecord<F extends string> {
/** The reserved fields present in the record, keyed by reserved name. */
reserved: Partial<Record<F, unknown>>
/** Every other key — the consumer's custom metadata, and nothing else. */
custom: Record<string, unknown>
}
const RESERVED_ENTITY_SET: ReadonlySet<string> = new Set(RESERVED_ENTITY_FIELDS)
const RESERVED_RELATION_SET: ReadonlySet<string> = new Set(RESERVED_RELATION_FIELDS)
/**
* @description Shared splitter — partitions a record's keys against a
* reserved-name set. `null`/`undefined` records split to two empty objects.
* @param record - The stored flat metadata record (reserved + custom keys mixed).
* @param reservedSet - The reserved-name set to partition against.
* @returns The `{ reserved, custom }` halves.
*/
function splitRecord<F extends string>(
record: Record<string, unknown> | null | undefined,
reservedSet: ReadonlySet<string>
): SplitMetadataRecord<F> {
const reserved: Record<string, unknown> = {}
const custom: Record<string, unknown> = {}
if (record && typeof record === 'object') {
for (const [key, value] of Object.entries(record)) {
if (reservedSet.has(key)) {
reserved[key] = value
} else {
custom[key] = value
}
}
}
return { reserved: reserved as Partial<Record<F, unknown>>, custom }
}
/**
* @description Split a stored entity (noun) flat metadata record into
* reserved fields and custom metadata — THE canonical read-side split. Every
* entity read path (live `get()`, batch reads, paginated listings, and
* historical `asOf()` materialization) goes through this function, so the
* reserved list can never drift between read paths.
* @param record - The stored flat metadata record.
* @returns `reserved` (Brainy-owned fields) and `custom` (the consumer's metadata bag).
* @example
* const { reserved, custom } = splitNounMetadataRecord(stored)
* // reserved.noun → entity.type, reserved.confidence → entity.confidence, …
* // custom → entity.metadata (custom fields only, always)
*/
export function splitNounMetadataRecord(
record: Record<string, unknown> | null | undefined
): SplitMetadataRecord<ReservedEntityField> {
return splitRecord(record, RESERVED_ENTITY_SET)
}
/**
* @description Split a stored relationship (verb) flat metadata record into
* reserved fields and custom metadata — the verb mirror of
* {@link splitNounMetadataRecord}, used by every relationship read path.
* @param record - The stored flat metadata record.
* @returns `reserved` (Brainy-owned fields) and `custom` (the consumer's metadata bag).
*/
export function splitVerbMetadataRecord(
record: Record<string, unknown> | null | undefined
): SplitMetadataRecord<ReservedRelationField> {
return splitRecord(record, RESERVED_RELATION_SET)
}