David Snelling
54c7c39669
An untyped (JS) caller that smuggles a Brainy-reserved field (confidence, weight, subtype, visibility, service, createdBy, noun/verb, data, createdAt, updatedAt, _rev) inside a write-path metadata bag previously got a silent remap-or-drop — a class of bug where confidence-evolution writes no-oped for weeks before being caught on read-back. 8.0 closes this with no silent failures. - New BrainyConfig.reservedFieldPolicy: 'throw' | 'warn' | 'remap' (default 'throw'). 'throw' rejects the write naming every offending key + its correct write path; 'warn' remaps with a one-shot per-key warning; 'remap' is the legacy silent path. - Central enforceReservedPolicy gate wired into all four remap methods (add, update, relate, updateRelation) so live calls AND their transact()/with() mirrors honor it. Single-source reservedWritePath guidance shared by throw+warn. - 'warn' now warns for EVERY reserved key (closes the gap where only system-managed fields warned). Dead warnDropped* helpers removed. - Import pipeline migrated to route reserved values (confidence/weight/subtype) through dedicated params and strip reserved keys from extractor/customMetadata bags via the canonical split*MetadataRecord helpers — imports no longer trip the default throw. - Tests: new reservedFieldPolicy matrix (throw/warn/remap across every write path + transact); remap-correctness suite reframed as opt-in 'remap'; shared test-factory no longer emits reserved keys in custom metadata.
403 lines
15 KiB
TypeScript
403 lines
15 KiB
TypeScript
/**
|
|
* @module tests/unit/brainy/update-reserved-metadata-remap
|
|
* @description Regression tests for the reserved-field metadata-bag trap,
|
|
* ported from the 7.x fix and extended to the full 8.0 contract.
|
|
*
|
|
* History: `add({metadata: {confidence}})` lifted reserved fields to their
|
|
* canonical top-level location, but `update({metadata: {confidence}})`
|
|
* silently dropped the same shape — the patch value survived the merge and
|
|
* was then clobbered by the preserve-existing spread. A production
|
|
* consumer's confidence-evolution writes no-oped for weeks before being
|
|
* caught by reading values back.
|
|
*
|
|
* These tests pin the LEGACY REMAP behavior, which in 8.0 is opt-in via
|
|
* `reservedFieldPolicy: 'remap'` (the default is `'throw'` — see the policy
|
|
* matrix in tests/unit/brainy/reserved-field-policy.test.ts). The brain in
|
|
* every test below is constructed with `reservedFieldPolicy: 'remap'` so these
|
|
* deep correctness assertions about the remap path stay exercised.
|
|
*
|
|
* Remap contract under test (every write path, entities AND relationships):
|
|
* - user-mutable reserved fields (`confidence`, `weight`, `subtype` — plus
|
|
* `service`/`createdBy` at add()/relate() time) remap from the metadata
|
|
* bag to their dedicated top-level param, with top-level winning when both
|
|
* are present;
|
|
* - system-managed reserved fields (`createdAt`, `_rev`, `noun`/`verb`,
|
|
* `data`, …) are dropped from the bag;
|
|
* - the same normalization applies to `transact()` operations and `with()`
|
|
* speculative views;
|
|
* - reads NEVER echo a reserved field inside `metadata`.
|
|
*
|
|
* TypeScript callers can't write these shapes at all (compile-time guard on
|
|
* the metadata param types — see tests/unit/types/reserved-metadata-keys.test-d.ts);
|
|
* these tests simulate untyped (JavaScript) callers, hence the `as object`
|
|
* widenings on the metadata literals.
|
|
*/
|
|
|
|
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
|
|
import { Brainy } from '../../../src/index.js'
|
|
import { NounType, VerbType } from '../../../src/types/graphTypes.js'
|
|
import { createTestConfig } from '../../helpers/test-factory.js'
|
|
|
|
describe('reserved-field metadata remap (8.0 legacy remap path)', () => {
|
|
let brain: Brainy
|
|
|
|
beforeEach(async () => {
|
|
// The remap path is opt-in in 8.0 (default policy is 'throw').
|
|
brain = new Brainy(createTestConfig({ reservedFieldPolicy: 'remap' }))
|
|
await brain.init()
|
|
})
|
|
|
|
afterEach(async () => {
|
|
await brain.close()
|
|
})
|
|
|
|
describe('update() — the ported 7.x regression', () => {
|
|
it('remaps metadata.confidence to the top-level field (the production repro)', async () => {
|
|
const id = await brain.add({
|
|
type: NounType.Concept,
|
|
subtype: 'general',
|
|
data: 'x',
|
|
metadata: { confidence: 0.8 } as object
|
|
})
|
|
|
|
// Top-level write works (always did)
|
|
await brain.update({ id, confidence: 0.42 })
|
|
let entity = await brain.get(id)
|
|
expect(entity?.confidence).toBe(0.42)
|
|
|
|
// Metadata-patch write — silently dropped pre-fix, remapped now
|
|
await brain.update({ id, metadata: { confidence: 0.33 } as object })
|
|
entity = await brain.get(id)
|
|
expect(entity?.confidence).toBe(0.33)
|
|
// The reserved key must not linger inside the metadata bag
|
|
expect((entity?.metadata as Record<string, unknown>)?.confidence).toBeUndefined()
|
|
})
|
|
|
|
it('remaps metadata.weight and metadata.subtype the same way', async () => {
|
|
const id = await brain.add({
|
|
type: NounType.Concept,
|
|
subtype: 'general',
|
|
data: 'y',
|
|
metadata: {}
|
|
})
|
|
|
|
await brain.update({ id, metadata: { weight: 0.7, subtype: 'specialized' } as object })
|
|
const entity = await brain.get(id)
|
|
expect(entity?.weight).toBe(0.7)
|
|
expect(entity?.subtype).toBe('specialized')
|
|
expect((entity?.metadata as Record<string, unknown>)?.weight).toBeUndefined()
|
|
expect((entity?.metadata as Record<string, unknown>)?.subtype).toBeUndefined()
|
|
})
|
|
|
|
it('top-level param wins when both top-level and metadata-patch carry the field', async () => {
|
|
const id = await brain.add({
|
|
type: NounType.Concept,
|
|
subtype: 'general',
|
|
data: 'z',
|
|
metadata: { confidence: 0.5 } as object
|
|
})
|
|
|
|
await brain.update({ id, confidence: 0.9, metadata: { confidence: 0.1 } as object })
|
|
const entity = await brain.get(id)
|
|
expect(entity?.confidence).toBe(0.9)
|
|
})
|
|
|
|
it('drops system-managed fields from patches without corrupting the entity', async () => {
|
|
const id = await brain.add({
|
|
type: NounType.Concept,
|
|
subtype: 'general',
|
|
data: 'w',
|
|
metadata: { keep: 'me' }
|
|
})
|
|
const before = await brain.get(id)
|
|
|
|
await brain.update({
|
|
id,
|
|
metadata: { createdAt: 1, _rev: 999, noun: 'organization', other: 'applied' } as object
|
|
})
|
|
const after = await brain.get(id)
|
|
|
|
expect(after?.createdAt).toBe(before?.createdAt) // immutable
|
|
expect(after?.type).toBe('concept') // noun patch ignored
|
|
expect(after?._rev).toBe((before?._rev ?? 1) + 1) // _rev patch ignored; normal bump applied
|
|
expect((after?.metadata as Record<string, unknown>)?.other).toBe('applied') // custom fields still merge
|
|
expect((after?.metadata as Record<string, unknown>)?.keep).toBe('me')
|
|
expect((after?.metadata as Record<string, unknown>)?._rev).toBeUndefined()
|
|
expect((after?.metadata as Record<string, unknown>)?.createdAt).toBeUndefined()
|
|
expect((after?.metadata as Record<string, unknown>)?.noun).toBeUndefined()
|
|
})
|
|
|
|
it('custom (non-reserved) metadata patches are unaffected by the remap', async () => {
|
|
const id = await brain.add({
|
|
type: NounType.Concept,
|
|
subtype: 'general',
|
|
data: 'v',
|
|
metadata: { status: 'draft' }
|
|
})
|
|
|
|
await brain.update({ id, metadata: { status: 'reviewed', rating: 4.5 } })
|
|
const entity = await brain.get(id)
|
|
expect((entity?.metadata as Record<string, unknown>)?.status).toBe('reviewed')
|
|
expect((entity?.metadata as Record<string, unknown>)?.rating).toBe(4.5)
|
|
})
|
|
})
|
|
|
|
describe('add() — explicit lift, identical contract', () => {
|
|
it('lifts confidence/weight/subtype out of the bag to top level', async () => {
|
|
const id = await brain.add({
|
|
type: NounType.Person,
|
|
data: 'lift check',
|
|
metadata: { confidence: 0.8, weight: 0.6, subtype: 'employee', dept: 'eng' } as object
|
|
})
|
|
|
|
const entity = await brain.get(id)
|
|
expect(entity?.confidence).toBe(0.8)
|
|
expect(entity?.weight).toBe(0.6)
|
|
expect(entity?.subtype).toBe('employee')
|
|
expect(entity?.metadata).toEqual({ dept: 'eng' })
|
|
})
|
|
|
|
it('lifts service (settable at add time) and lets the top-level param win', async () => {
|
|
const lifted = await brain.add({
|
|
type: NounType.Person,
|
|
subtype: 'employee',
|
|
data: 'service lift',
|
|
metadata: { service: 'orders' } as object
|
|
})
|
|
expect((await brain.get(lifted))?.service).toBe('orders')
|
|
|
|
const topLevelWins = await brain.add({
|
|
type: NounType.Person,
|
|
subtype: 'employee',
|
|
data: 'service precedence',
|
|
service: 'billing',
|
|
metadata: { service: 'orders' } as object
|
|
})
|
|
const entity = await brain.get(topLevelWins)
|
|
expect(entity?.service).toBe('billing')
|
|
expect((entity?.metadata as Record<string, unknown>)?.service).toBeUndefined()
|
|
})
|
|
|
|
it('a remapped subtype satisfies subtype enforcement like a top-level one', async () => {
|
|
brain.requireSubtype(NounType.Document)
|
|
|
|
// Top-level missing, but the bag carries it — must not throw.
|
|
const id = await brain.add({
|
|
type: NounType.Document,
|
|
data: 'enforcement via remap',
|
|
metadata: { subtype: 'invoice' } as object
|
|
})
|
|
expect((await brain.get(id))?.subtype).toBe('invoice')
|
|
|
|
// Neither place carries it — must throw.
|
|
await expect(
|
|
brain.add({ type: NounType.Document, data: 'no subtype anywhere' })
|
|
).rejects.toThrow(/subtype/)
|
|
})
|
|
})
|
|
|
|
describe('transact() — same remap on add and update ops', () => {
|
|
it('normalizes reserved fields in transact add + update ops', async () => {
|
|
const db1 = await brain.transact([
|
|
{
|
|
op: 'add',
|
|
type: NounType.Concept,
|
|
subtype: 'general',
|
|
data: 'tx',
|
|
metadata: { confidence: 0.7, custom: 'a' } as object
|
|
}
|
|
])
|
|
const id = db1.receipt!.ids[0]
|
|
|
|
let entity = await brain.get(id)
|
|
expect(entity?.confidence).toBe(0.7)
|
|
expect(entity?.metadata).toEqual({ custom: 'a' })
|
|
|
|
await brain.transact([
|
|
{ op: 'update', id, metadata: { confidence: 0.25, custom: 'b' } as object }
|
|
])
|
|
entity = await brain.get(id)
|
|
expect(entity?.confidence).toBe(0.25)
|
|
expect(entity?.metadata).toEqual({ custom: 'b' })
|
|
expect((entity?.metadata as Record<string, unknown>)?.confidence).toBeUndefined()
|
|
})
|
|
|
|
it('historical asOf() reads surface reserved fields ONLY top-level', async () => {
|
|
const db1 = await brain.transact([
|
|
{
|
|
op: 'add',
|
|
type: NounType.Concept,
|
|
subtype: 'general',
|
|
data: 'historical',
|
|
metadata: { confidence: 0.9, custom: 'past' } as object
|
|
}
|
|
])
|
|
const id = db1.receipt!.ids[0]
|
|
|
|
// Move the world forward so generation db1 is historical.
|
|
await brain.transact([{ op: 'update', id, confidence: 0.1, metadata: { custom: 'now' } }])
|
|
|
|
const past = await brain.asOf(db1.generation)
|
|
const historical = await past.get(id)
|
|
expect(historical?.confidence).toBe(0.9)
|
|
expect(historical?.metadata).toEqual({ custom: 'past' })
|
|
await past.release()
|
|
})
|
|
|
|
it('with() speculative views apply the same normalization', async () => {
|
|
const base = await brain.now()
|
|
const speculative = await base.with([
|
|
{
|
|
op: 'add',
|
|
id: 'spec-entity',
|
|
type: NounType.Concept,
|
|
subtype: 'general',
|
|
data: 'spec',
|
|
metadata: { confidence: 0.65, custom: 'spec' } as object
|
|
}
|
|
])
|
|
|
|
const entity = await speculative.get('spec-entity')
|
|
expect(entity?.confidence).toBe(0.65)
|
|
// 8.0 id normalization: a natural-key id is mapped to a stable UUID and
|
|
// the caller's original string is preserved under _originalId — surfaced
|
|
// here exactly as the durable transact()/add() paths do.
|
|
expect(entity?.metadata).toEqual({ custom: 'spec', _originalId: 'spec-entity' })
|
|
await speculative.release()
|
|
await base.release()
|
|
})
|
|
})
|
|
|
|
describe('read paths never echo reserved fields inside metadata', () => {
|
|
it('find() (storage pagination path) returns custom-only metadata with reserved fields top-level', async () => {
|
|
const id = await brain.add({
|
|
type: NounType.Person,
|
|
subtype: 'employee',
|
|
data: 'pagination echo check',
|
|
confidence: 0.8,
|
|
weight: 0.6,
|
|
metadata: { dept: 'eng' }
|
|
})
|
|
|
|
// No query/filter → served by the direct storage pagination path
|
|
// (getNounsWithPagination), which historically echoed the full flat
|
|
// record (noun/subtype/createdAt/… inside metadata).
|
|
const results = await brain.find({ limit: 50 })
|
|
const result = results.find((r) => r.id === id)
|
|
expect(result).toBeDefined()
|
|
expect(result?.entity.metadata).toEqual({ dept: 'eng' })
|
|
expect(result?.entity.type).toBe(NounType.Person)
|
|
expect(result?.entity.subtype).toBe('employee')
|
|
expect(result?.entity.confidence).toBe(0.8)
|
|
expect(result?.entity.weight).toBe(0.6)
|
|
expect(typeof result?.entity.createdAt).toBe('number')
|
|
expect(result?.entity._rev).toBe(1)
|
|
})
|
|
|
|
it('related() by target surfaces reserved fields top-level, custom-only metadata', async () => {
|
|
const a = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'src' })
|
|
const b = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'tgt' })
|
|
const relId = await brain.relate({
|
|
from: a,
|
|
to: b,
|
|
type: VerbType.ReportsTo,
|
|
subtype: 'direct',
|
|
confidence: 0.9,
|
|
weight: 0.5,
|
|
service: 'orders',
|
|
metadata: { note: 'target path' }
|
|
})
|
|
|
|
const relations = await brain.related({ to: b })
|
|
const rel = relations.find((r) => r.id === relId)
|
|
expect(rel).toBeDefined()
|
|
expect(rel?.metadata).toEqual({ note: 'target path' })
|
|
expect(rel?.subtype).toBe('direct')
|
|
expect(rel?.confidence).toBe(0.9)
|
|
expect(rel?.weight).toBe(0.5)
|
|
expect(rel?.service).toBe('orders')
|
|
expect(typeof rel?.createdAt).toBe('number')
|
|
})
|
|
})
|
|
|
|
describe('relationships — relate() / updateRelation() mirror', () => {
|
|
let a: string
|
|
let b: string
|
|
|
|
beforeEach(async () => {
|
|
a = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'A' })
|
|
b = await brain.add({ type: NounType.Person, subtype: 'employee', data: 'B' })
|
|
})
|
|
|
|
it('relate() persists the top-level confidence and service params', async () => {
|
|
const relId = await brain.relate({
|
|
from: a,
|
|
to: b,
|
|
type: VerbType.ReportsTo,
|
|
subtype: 'direct',
|
|
confidence: 0.77,
|
|
service: 'orders'
|
|
})
|
|
|
|
const relations = await brain.related({ from: a })
|
|
const rel = relations.find((r) => r.id === relId)
|
|
expect(rel?.confidence).toBe(0.77)
|
|
expect(rel?.service).toBe('orders')
|
|
})
|
|
|
|
it('relate() remaps reserved fields out of the metadata bag', async () => {
|
|
const relId = await brain.relate({
|
|
from: a,
|
|
to: b,
|
|
type: VerbType.RelatedTo,
|
|
subtype: 'colleague',
|
|
metadata: { confidence: 0.4, weight: 0.3, role: 'peer' } as object
|
|
})
|
|
|
|
const relations = await brain.related({ from: a })
|
|
const rel = relations.find((r) => r.id === relId)
|
|
expect(rel?.confidence).toBe(0.4)
|
|
expect(rel?.weight).toBe(0.3)
|
|
expect(rel?.metadata).toEqual({ role: 'peer' })
|
|
})
|
|
|
|
it('relation.metadata never echoes the verb type key', async () => {
|
|
const relId = await brain.relate({
|
|
from: a,
|
|
to: b,
|
|
type: VerbType.RelatedTo,
|
|
subtype: 'colleague',
|
|
metadata: { note: 'no echo' }
|
|
})
|
|
|
|
const relations = await brain.related({ from: a })
|
|
const rel = relations.find((r) => r.id === relId)
|
|
expect(rel?.type).toBe(VerbType.RelatedTo)
|
|
expect((rel?.metadata as Record<string, unknown>)?.verb).toBeUndefined()
|
|
expect(rel?.metadata).toEqual({ note: 'no echo' })
|
|
})
|
|
|
|
it('updateRelation() remaps the user-mutable trio and preserves service', async () => {
|
|
const relId = await brain.relate({
|
|
from: a,
|
|
to: b,
|
|
type: VerbType.ReportsTo,
|
|
subtype: 'direct',
|
|
service: 'orders',
|
|
metadata: { keep: 'me' }
|
|
})
|
|
|
|
await brain.updateRelation({
|
|
id: relId,
|
|
metadata: { confidence: 0.55, subtype: 'dotted-line', extra: 'applied' } as object
|
|
})
|
|
|
|
const relations = await brain.related({ from: a })
|
|
const rel = relations.find((r) => r.id === relId)
|
|
expect(rel?.confidence).toBe(0.55)
|
|
expect(rel?.subtype).toBe('dotted-line')
|
|
expect(rel?.service).toBe('orders') // fixed at relate() time, never erased by updates
|
|
expect(rel?.metadata).toEqual({ keep: 'me', extra: 'applied' })
|
|
})
|
|
})
|
|
})
|