feat(8.0): visibility field (public/internal/system) on nouns + verbs
Adds a reserved, top-level `visibility` field (mirrors the subtype rollout):
'public' (default, surfaced) | 'internal' (developer app-internal — hidden from
default find/count/stats, opt-in via includeInternal) | 'system' (Brainy
plumbing, library-set only).
Fixes a real leak: the VFS root entity counted in getNounCount() and appeared in
find() (a fresh brain reported 1 entity). It is now visibility:'system' →
excluded from every user-facing surface. Developers also get a first-class
hidden-unless-asked tier (e.g. learned internals vs user-exposed data).
- Reserved (RESERVED_ENTITY_FIELDS / RESERVED_RELATION_FIELDS) — spoof-proof from metadata.
- Threaded through add/relate/update/transact; surfaced top-level on reads.
- Default exclusion in counts (baseStorage), find()/related() (hard candidate
filter via excludeVisibility — keeps topK/limit correct), and stats;
includeInternal/includeSystem opt-ins.
- VFS root marked 'system'.
Tests: visibility.test.ts 17/17 (fresh-brain getNounCount()===0, internal hidden
+ opt-in, verb symmetry, top-level surfacing, metadata-spoof rejection). Unit
1431 green; count-synchronization integration now passes (off-by-one fixed).
2026-06-16 15:20:26 -07:00
/ * *
* @module tests / unit / brainy / visibility
* @description Tests for the 8.0 reserved ` visibility ` field — the three - tier
* ( ` public ` | ` internal ` | ` system ` ) gate that controls whether an entity or
* relationship surfaces on default user - facing reads .
*
* Contract under test :
* - Absent === ` 'public' ` : counted and returned everywhere .
* - ` 'internal' ` : hidden from default ` find() ` / ` related() ` / counts / ` stats() ` ,
* but retrievable with ` includeInternal: true ` .
* - ` 'system' ` : Brainy plumbing ( the VFS root ) ; hidden everywhere by default ,
* surfaced only with ` includeSystem: true ` . Not settable through the public
* ` add() ` / ` relate() ` params ( the param type narrows to ` 'public' | 'internal' ` ) .
* - ` visibility ` is a reserved top - level field : surfaced top - level on reads , never
* inside ` metadata ` , and an untyped caller smuggling it through ` metadata ` is
* normalized ( a ` 'public' ` / ` 'internal' ` value is lifted ; ` 'system' ` is dropped ) .
*
* THE key regression : a fresh brain reports ` getNounCount() === 0 ` even though the
* VFS root entity exists — because the root is ` visibility: 'system' ` and excluded .
*
* Runs under the deterministic embedder ( the unit setup sets ` BRAINY_UNIT_TEST ` ) .
* /
import { describe , it , expect , beforeEach , afterEach } from 'vitest'
import { Brainy } from '../../../src/index.js'
import { NounType , VerbType } from '../../../src/types/graphTypes.js'
import { createTestConfig } from '../../helpers/test-factory.js'
describe ( 'visibility (8.0 reserved field)' , ( ) = > {
let brain : Brainy
beforeEach ( async ( ) = > {
brain = new Brainy ( createTestConfig ( ) )
await brain . init ( )
} )
afterEach ( async ( ) = > {
await brain . close ( )
} )
describe ( 'counts exclude system + internal' , ( ) = > {
it ( 'a fresh brain reports getNounCount() === 0 (the VFS root is system → excluded)' , async ( ) = > {
// KEY REGRESSION: the VFS root entity exists after init() but is
// visibility:'system', so it must not show up in the user-facing count.
expect ( await brain . getNounCount ( ) ) . toBe ( 0 )
} )
it ( 'add() with no visibility is counted (public default)' , async ( ) = > {
await brain . add ( { type : NounType . Concept , data : 'public thing' } )
expect ( await brain . getNounCount ( ) ) . toBe ( 1 )
} )
it ( 'add({ visibility: "internal" }) is NOT counted' , async ( ) = > {
await brain . add ( { type : NounType . Concept , data : 'public one' } )
await brain . add ( { type : NounType . Concept , data : 'app-internal' , visibility : 'internal' } )
// Only the public entity counts.
expect ( await brain . getNounCount ( ) ) . toBe ( 1 )
} )
it ( 'flipping visibility via update() moves the entity in/out of the count' , async ( ) = > {
const id = await brain . add ( { type : NounType . Concept , data : 'flip me' } )
expect ( await brain . getNounCount ( ) ) . toBe ( 1 )
await brain . update ( { id , visibility : 'internal' } )
expect ( await brain . getNounCount ( ) ) . toBe ( 0 )
await brain . update ( { id , visibility : 'public' } )
expect ( await brain . getNounCount ( ) ) . toBe ( 1 )
} )
it ( 'stats().entityCount reports only public entities' , async ( ) = > {
await brain . add ( { type : NounType . Concept , data : 'a' } )
await brain . add ( { type : NounType . Concept , data : 'b' , visibility : 'internal' } )
const stats = await brain . stats ( )
expect ( stats . entityCount ) . toBe ( 1 )
} )
} )
describe ( 'find() default-excludes internal + system' , ( ) = > {
it ( 'public entities are returned; internal are hidden by default' , async ( ) = > {
const pubId = await brain . add ( { type : NounType . Concept , data : 'visible' } )
await brain . add ( { type : NounType . Concept , data : 'hidden' , visibility : 'internal' } )
const def = await brain . find ( { type : NounType . Concept , limit : 100 } )
const ids = def . map ( ( r ) = > r . id )
expect ( ids ) . toContain ( pubId )
expect ( def . length ) . toBe ( 1 )
} )
it ( 'find({ includeInternal: true }) also returns internal entities' , async ( ) = > {
const pubId = await brain . add ( { type : NounType . Concept , data : 'visible' } )
const intId = await brain . add ( { type : NounType . Concept , data : 'hidden' , visibility : 'internal' } )
const withInternal = await brain . find ( { type : NounType . Concept , includeInternal : true , limit : 100 } )
const ids = withInternal . map ( ( r ) = > r . id )
expect ( ids ) . toContain ( pubId )
expect ( ids ) . toContain ( intId )
expect ( withInternal . length ) . toBe ( 2 )
} )
it ( 'empty-query find() (no filter) also excludes internal by default' , async ( ) = > {
const pubId = await brain . add ( { type : NounType . Concept , data : 'visible' } )
await brain . add ( { type : NounType . Concept , data : 'hidden' , visibility : 'internal' } )
const all = await brain . find ( { limit : 100 } )
const ids = all . map ( ( r ) = > r . id )
expect ( ids ) . toContain ( pubId )
// Only the one public user entity (VFS root is system → excluded too).
expect ( all . length ) . toBe ( 1 )
} )
it ( 'limit stays exact when internal entities are interleaved (hard candidate filter)' , async ( ) = > {
// Two public + two internal; a default find(limit:2) must return exactly the
// two public ones, not get short-changed by the hidden ones.
await brain . add ( { type : NounType . Concept , data : 'pub-1' } )
await brain . add ( { type : NounType . Concept , data : 'int-1' , visibility : 'internal' } )
await brain . add ( { type : NounType . Concept , data : 'pub-2' } )
await brain . add ( { type : NounType . Concept , data : 'int-2' , visibility : 'internal' } )
const page = await brain . find ( { type : NounType . Concept , limit : 2 } )
expect ( page . length ) . toBe ( 2 )
for ( const r of page ) {
expect ( r . visibility === undefined || r . visibility === 'public' ) . toBe ( true )
}
} )
} )
describe ( 'the VFS root (system) entity' , ( ) = > {
it ( 'never appears in find(), even with includeInternal' , async ( ) = > {
const rootId = '00000000-0000-0000-0000-000000000000'
// Sanity: the root really exists (get() is an explicit by-id read, not a default surface).
expect ( await brain . get ( rootId ) ) . not . toBeNull ( )
const def = await brain . find ( { limit : 1000 } )
expect ( def . map ( ( r ) = > r . id ) ) . not . toContain ( rootId )
const withInternal = await brain . find ( { includeInternal : true , limit : 1000 } )
expect ( withInternal . map ( ( r ) = > r . id ) ) . not . toContain ( rootId )
} )
it ( 'appears only with includeSystem: true' , async ( ) = > {
const rootId = '00000000-0000-0000-0000-000000000000'
const withSystem = await brain . find ( { includeSystem : true , limit : 1000 } )
expect ( withSystem . map ( ( r ) = > r . id ) ) . toContain ( rootId )
} )
it ( 'the root carries visibility "system" when surfaced via get()' , async ( ) = > {
const rootId = '00000000-0000-0000-0000-000000000000'
const root = await brain . get ( rootId )
expect ( root ? . visibility ) . toBe ( 'system' )
} )
} )
describe ( 'verbs (relationships) — symmetric to nouns' , ( ) = > {
it ( 'relate({ visibility: "internal" }) is excluded from getVerbCount() + related() by default, included with includeInternal' , async ( ) = > {
const a = await brain . add ( { type : NounType . Person , data : 'a' } )
const b = await brain . add ( { type : NounType . Person , data : 'b' } )
const c = await brain . add ( { type : NounType . Person , data : 'c' } )
// One public edge, one internal edge from the same source.
await brain . relate ( { from : a , to : b , type : VerbType . RelatedTo } )
await brain . relate ( { from : a , to : c , type : VerbType . RelatedTo , visibility : 'internal' } )
// Counts: only the public edge.
expect ( await brain . getVerbCount ( ) ) . toBe ( 1 )
// related() default: only the public edge.
const def = await brain . related ( { from : a } )
expect ( def . length ) . toBe ( 1 )
expect ( def [ 0 ] . to ) . toBe ( b )
// related({ includeInternal }): both edges.
const withInternal = await brain . related ( { from : a , includeInternal : true } )
expect ( withInternal . length ) . toBe ( 2 )
expect ( withInternal . map ( ( r ) = > r . to ) . sort ( ) ) . toEqual ( [ b , c ] . sort ( ) )
} )
} )
describe ( 'visibility is a reserved top-level field' , ( ) = > {
it ( 'is surfaced top-level on get(), never inside metadata' , async ( ) = > {
const id = await brain . add ( {
type : NounType . Concept ,
data : 'x' ,
visibility : 'internal' ,
metadata : { kind : 'note' }
} )
const entity = await brain . get ( id )
expect ( entity ? . visibility ) . toBe ( 'internal' )
// metadata holds ONLY custom fields, never the reserved visibility key.
expect ( entity ? . metadata ) . toEqual ( { kind : 'note' } )
expect ( ( entity ? . metadata as Record < string , unknown > ) ? . visibility ) . toBeUndefined ( )
} )
it ( 'a public-default entity stores no visibility (absent === public)' , async ( ) = > {
const id = await brain . add ( { type : NounType . Concept , data : 'plain' } )
const entity = await brain . get ( id )
// Absent — not the literal string 'public'. Kept lean on the common path.
expect ( entity ? . visibility ) . toBeUndefined ( )
} )
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
it ( 'metadata.visibility is the USER’ s field (field-addressing law) — stored verbatim, never lifted to the engine tier' , async ( ) = > {
const id = await brain . add ( {
type : NounType . Concept ,
data : 'y' ,
metadata : { visibility : 'internal' , tag : 't' } as object
} )
const entity = await brain . get ( id )
// The user's field lives in the bag, verbatim…
expect ( ( entity ? . metadata as Record < string , unknown > ) ? . visibility ) . toBe ( 'internal' )
expect ( ( entity ? . metadata as Record < string , unknown > ) ? . tag ) . toBe ( 't' )
// …and the ENGINE tier is untouched: absent === public, so the entity
// stays visible on default reads (the engine tier is set only via the
// dedicated visibility param and reads at system.visibility).
expect ( entity ? . visibility ) . toBeUndefined ( )
const visible = await brain . find ( { type : NounType . Concept , limit : 20 } )
expect ( visible . map ( ( r ) = > r . id ) ) . toContain ( id )
feat(8.0): visibility field (public/internal/system) on nouns + verbs
Adds a reserved, top-level `visibility` field (mirrors the subtype rollout):
'public' (default, surfaced) | 'internal' (developer app-internal — hidden from
default find/count/stats, opt-in via includeInternal) | 'system' (Brainy
plumbing, library-set only).
Fixes a real leak: the VFS root entity counted in getNounCount() and appeared in
find() (a fresh brain reported 1 entity). It is now visibility:'system' →
excluded from every user-facing surface. Developers also get a first-class
hidden-unless-asked tier (e.g. learned internals vs user-exposed data).
- Reserved (RESERVED_ENTITY_FIELDS / RESERVED_RELATION_FIELDS) — spoof-proof from metadata.
- Threaded through add/relate/update/transact; surfaced top-level on reads.
- Default exclusion in counts (baseStorage), find()/related() (hard candidate
filter via excludeVisibility — keeps topK/limit correct), and stats;
includeInternal/includeSystem opt-ins.
- VFS root marked 'system'.
Tests: visibility.test.ts 17/17 (fresh-brain getNounCount()===0, internal hidden
+ opt-in, verb symmetry, top-level surfacing, metadata-spoof rejection). Unit
1431 green; count-synchronization integration now passes (off-by-one fixed).
2026-06-16 15:20:26 -07:00
} )
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
it ( 'a user field valued "system" cannot smuggle the Brainy-only tier — it is just user data' , async ( ) = > {
const id = await brain . add ( {
type : NounType . Concept ,
data : 'z' ,
metadata : { visibility : 'system' } as object
} )
const entity = await brain . get ( id )
// Engine tier unaffected → entity stays public (counted, visible);
// the string 'system' is ordinary user data in the bag.
expect ( entity ? . visibility ) . toBeUndefined ( )
expect ( ( entity ? . metadata as Record < string , unknown > ) ? . visibility ) . toBe ( 'system' )
const found = await brain . find ( { type : NounType . Concept , limit : 10 } )
expect ( found . map ( ( r ) = > r . id ) ) . toContain ( id )
feat(8.0): reserved-field enforcement — reservedFieldPolicy defaults to throw
An untyped (JS) caller that smuggles a Brainy-reserved field (confidence,
weight, subtype, visibility, service, createdBy, noun/verb, data, createdAt,
updatedAt, _rev) inside a write-path metadata bag previously got a silent
remap-or-drop — a class of bug where confidence-evolution writes no-oped for
weeks before being caught on read-back. 8.0 closes this with no silent failures.
- New BrainyConfig.reservedFieldPolicy: 'throw' | 'warn' | 'remap' (default 'throw').
'throw' rejects the write naming every offending key + its correct write path;
'warn' remaps with a one-shot per-key warning; 'remap' is the legacy silent path.
- Central enforceReservedPolicy gate wired into all four remap methods (add,
update, relate, updateRelation) so live calls AND their transact()/with()
mirrors honor it. Single-source reservedWritePath guidance shared by throw+warn.
- 'warn' now warns for EVERY reserved key (closes the gap where only
system-managed fields warned). Dead warnDropped* helpers removed.
- Import pipeline migrated to route reserved values (confidence/weight/subtype)
through dedicated params and strip reserved keys from extractor/customMetadata
bags via the canonical split*MetadataRecord helpers — imports no longer trip
the default throw.
- Tests: new reservedFieldPolicy matrix (throw/warn/remap across every write
path + transact); remap-correctness suite reframed as opt-in 'remap'; shared
test-factory no longer emits reserved keys in custom metadata.
2026-06-20 15:37:21 -07:00
} )
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
it ( 'a forged system.visibility key in metadata refuses loudly at the write door' , async ( ) = > {
feat(8.0): reserved-field enforcement — reservedFieldPolicy defaults to throw
An untyped (JS) caller that smuggles a Brainy-reserved field (confidence,
weight, subtype, visibility, service, createdBy, noun/verb, data, createdAt,
updatedAt, _rev) inside a write-path metadata bag previously got a silent
remap-or-drop — a class of bug where confidence-evolution writes no-oped for
weeks before being caught on read-back. 8.0 closes this with no silent failures.
- New BrainyConfig.reservedFieldPolicy: 'throw' | 'warn' | 'remap' (default 'throw').
'throw' rejects the write naming every offending key + its correct write path;
'warn' remaps with a one-shot per-key warning; 'remap' is the legacy silent path.
- Central enforceReservedPolicy gate wired into all four remap methods (add,
update, relate, updateRelation) so live calls AND their transact()/with()
mirrors honor it. Single-source reservedWritePath guidance shared by throw+warn.
- 'warn' now warns for EVERY reserved key (closes the gap where only
system-managed fields warned). Dead warnDropped* helpers removed.
- Import pipeline migrated to route reserved values (confidence/weight/subtype)
through dedicated params and strip reserved keys from extractor/customMetadata
bags via the canonical split*MetadataRecord helpers — imports no longer trip
the default throw.
- Tests: new reservedFieldPolicy matrix (throw/warn/remap across every write
path + transact); remap-correctness suite reframed as opt-in 'remap'; shared
test-factory no longer emits reserved keys in custom metadata.
2026-06-20 15:37:21 -07:00
await expect (
brain . add ( {
type : NounType . Concept ,
data : 'throws' ,
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
metadata : { 'system.visibility' : 'internal' } as object
feat(8.0): reserved-field enforcement — reservedFieldPolicy defaults to throw
An untyped (JS) caller that smuggles a Brainy-reserved field (confidence,
weight, subtype, visibility, service, createdBy, noun/verb, data, createdAt,
updatedAt, _rev) inside a write-path metadata bag previously got a silent
remap-or-drop — a class of bug where confidence-evolution writes no-oped for
weeks before being caught on read-back. 8.0 closes this with no silent failures.
- New BrainyConfig.reservedFieldPolicy: 'throw' | 'warn' | 'remap' (default 'throw').
'throw' rejects the write naming every offending key + its correct write path;
'warn' remaps with a one-shot per-key warning; 'remap' is the legacy silent path.
- Central enforceReservedPolicy gate wired into all four remap methods (add,
update, relate, updateRelation) so live calls AND their transact()/with()
mirrors honor it. Single-source reservedWritePath guidance shared by throw+warn.
- 'warn' now warns for EVERY reserved key (closes the gap where only
system-managed fields warned). Dead warnDropped* helpers removed.
- Import pipeline migrated to route reserved values (confidence/weight/subtype)
through dedicated params and strip reserved keys from extractor/customMetadata
bags via the canonical split*MetadataRecord helpers — imports no longer trip
the default throw.
- Tests: new reservedFieldPolicy matrix (throw/warn/remap across every write
path + transact); remap-correctness suite reframed as opt-in 'remap'; shared
test-factory no longer emits reserved keys in custom metadata.
2026-06-20 15:37:21 -07:00
} )
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
) . rejects . toThrow ( /system\./ )
feat(8.0): visibility field (public/internal/system) on nouns + verbs
Adds a reserved, top-level `visibility` field (mirrors the subtype rollout):
'public' (default, surfaced) | 'internal' (developer app-internal — hidden from
default find/count/stats, opt-in via includeInternal) | 'system' (Brainy
plumbing, library-set only).
Fixes a real leak: the VFS root entity counted in getNounCount() and appeared in
find() (a fresh brain reported 1 entity). It is now visibility:'system' →
excluded from every user-facing surface. Developers also get a first-class
hidden-unless-asked tier (e.g. learned internals vs user-exposed data).
- Reserved (RESERVED_ENTITY_FIELDS / RESERVED_RELATION_FIELDS) — spoof-proof from metadata.
- Threaded through add/relate/update/transact; surfaced top-level on reads.
- Default exclusion in counts (baseStorage), find()/related() (hard candidate
filter via excludeVisibility — keeps topK/limit correct), and stats;
includeInternal/includeSystem opt-ins.
- VFS root marked 'system'.
Tests: visibility.test.ts 17/17 (fresh-brain getNounCount()===0, internal hidden
+ opt-in, verb symmetry, top-level surfacing, metadata-spoof rejection). Unit
1431 green; count-synchronization integration now passes (off-by-one fixed).
2026-06-16 15:20:26 -07:00
} )
} )
} )