feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
/ * *
* Migration System Integration Tests
*
* Verifies :
* - MigrationRunner with in - memory storage
* - brain . migrate ( ) public API ( dry - run and apply )
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
* - Pre - migration snapshots via ` backupTo ` ( persist - before - migrate + restore )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
* - No - op when MIGRATIONS array is empty
* - Warning log when autoMigrate is false
* - Multi - tenant independent migration state
* - Verb and noun transforms
* /
import { describe , it , expect , beforeEach , afterEach , vi } from 'vitest'
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
import * as fs from 'node:fs'
import * as os from 'node:os'
import * as path from 'node:path'
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
import { Brainy } from '../../src/brainy.js'
import { MigrationRunner , MIGRATIONS } from '../../src/migration/index.js'
import type { Migration } from '../../src/migration/index.js'
import { NounType , VerbType } from '../../src/types/graphTypes.js'
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
// THE VIEW CONTRACT (field-addressing law): transforms receive engine fields
// top-level and the USER's bag nested under `metadata` — user-field changes
// go inside the bag. These two helpers keep the one-liner migrations tidy.
const bagOf = ( m : Record < string , unknown > ) : Record < string , unknown > = >
m . metadata as Record < string , unknown >
const withBag = (
m : Record < string , unknown > ,
patch : Record < string , unknown >
) : Record < string , unknown > = > ( { . . . m , metadata : { . . . bagOf ( m ) , . . . patch } } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
// Helper to temporarily inject migrations into the MIGRATIONS array
function withMigrations ( migrations : Migration [ ] , fn : ( ) = > Promise < void > ) : Promise < void > {
const original = MIGRATIONS . splice ( 0 , MIGRATIONS . length )
MIGRATIONS . push ( . . . migrations )
return fn ( ) . finally ( ( ) = > {
MIGRATIONS . splice ( 0 , MIGRATIONS . length )
MIGRATIONS . push ( . . . original )
} )
}
describe ( 'Migration System' , ( ) = > {
let brain : Brainy
beforeEach ( async ( ) = > {
feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1)
Brainy 8.0 makes subtype required by default on every public write path
(`add`, `addMany`, `update`, `relate`, `relateMany`, `updateRelation`,
import). Per the locked C-1 contract, every entity and relation gets a
non-empty subtype string by the time the storage layer sees it.
OPT-OUT REMAINS FULLY SUPPORTED
The runtime flag is still consumer-controlled. Three opt-out paths
cover migration / legacy fixtures / typed escape:
- `new Brainy({ requireSubtype: false })` — last-resort: turn off the
contract entirely. Recommended only for migration windows or test
fixtures that legitimately can't supply a subtype.
- `new Brainy({ requireSubtype: { except: [NounType.Thing, ...] } })` —
per-type allowlist: strict everywhere except the listed types.
- `brain.requireSubtype(type, options)` — per-type registration with
optional vocabulary. Composes with the brain-wide flag.
Default is now `true`. Opt-out is explicit and documented; nothing
silently degrades.
TEST SWEEP
Bulk-applied `requireSubtype: false` to every `new Brainy({...})` call
site across 120 test files. Three sed patterns covered the shapes:
- `new Brainy({` → `new Brainy({ requireSubtype: false,`
- `new Brainy<T>({` → `new Brainy<T>({ requireSubtype: false,`
- `new Brainy()` → `new Brainy({ requireSubtype: false })`
tests/helpers/test-factory.ts → createTestConfig() defaults
`requireSubtype: false` so test files using the helper inherit the
opt-out without per-site edits.
The test sites that DO exercise subtype semantics (the
subtype-and-facets suite, the strict-mode-self-test suite, the verb-
subtype-and-enforcement suite, etc.) already pass real subtypes — they
were the 7.30.x acceptance tests for this contract. Those tests
continue to pass unchanged.
CHANGES
src/brainy.ts
- normalizeConfig() — `requireSubtype` default `false` → `true`.
Comment refreshed to document the three opt-out paths.
tests/* (120 files)
- Bulk-edited brain construction sites. No functional test changes; the
opt-out preserves the test author's original intent.
tests/helpers/test-factory.ts
- createTestConfig() base config gains `requireSubtype: false`.
NO-OP for consumers who were already passing subtype on every write.
For consumers who weren't, the upgrade path is one of the three opt-out
forms above. Migration recipe documented in 8.0 release notes (next
commit).
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1408 / 1409 (same pre-existing race-condition outstanding;
no other regressions from the flip)
2026-06-09 14:58:25 -07:00
brain = new Brainy ( { requireSubtype : false , storage : { type : 'memory' } , silent : true } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
await brain . init ( )
} )
afterEach ( async ( ) = > {
await brain . close ( )
} )
// ─── No-op tests ───────────────────────────────────────────────
describe ( 'No-op with empty MIGRATIONS' , ( ) = > {
it ( 'should have zero pending migrations by default' , async ( ) = > {
// MIGRATIONS is empty by default — init() should add ~0ms overhead
const preview = await brain . migrate ( { dryRun : true } )
expect ( preview ) . toHaveProperty ( 'pendingMigrations' )
const p = preview as any
expect ( p . pendingMigrations ) . toHaveLength ( 0 )
expect ( p . affectedEntities ) . toBe ( 0 )
} )
it ( 'should return no-op result when applying with no migrations' , async ( ) = > {
const result = await brain . migrate ( )
expect ( result ) . toHaveProperty ( 'migrationsApplied' )
const r = result as any
expect ( r . migrationsApplied ) . toHaveLength ( 0 )
expect ( r . entitiesModified ) . toBe ( 0 )
} )
} )
// ─── Dry-run preview ──────────────────────────────────────────
describe ( 'Dry-run preview' , ( ) = > {
it ( 'should preview affected entities without writing' , async ( ) = > {
// Add some entities with a unique marker field
const id1 = await brain . add ( { type : NounType . Concept , data : { name : 'Alpha' } , metadata : { status : 'active' } } )
const id2 = await brain . add ( { type : NounType . Concept , data : { name : 'Beta' } , metadata : { status : 'inactive' } } )
const id3 = await brain . add ( { type : NounType . Concept , data : { name : 'Gamma' } , metadata : { status : 'pending' } } )
const testMigration : Migration = {
id : 'test-1.0.0-add-version' ,
version : '1.0.0' ,
description : 'Add version field to entities with status' ,
applies : 'nouns' ,
transform : ( m ) = > {
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
// Only transform entities that have our specific 'status' USER field
// (user fields live in the nested bag — the view contract).
const bag = m . metadata as Record < string , unknown >
if ( 'status' in bag && ! ( 'version' in bag ) ) {
return { . . . m , metadata : { . . . bag , version : 1 } }
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
return null
}
}
await withMigrations ( [ testMigration ] , async ( ) = > {
const preview = await brain . migrate ( { dryRun : true } )
const p = preview as any
expect ( p . pendingMigrations ) . toHaveLength ( 1 )
expect ( p . pendingMigrations [ 0 ] . id ) . toBe ( 'test-1.0.0-add-version' )
// All 3 entities have 'status' metadata
expect ( p . affectedEntities ) . toBeGreaterThanOrEqual ( 3 )
expect ( p . sampleChanges . length ) . toBeGreaterThan ( 0 )
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
// Samples carry the VIEW shape: user fields inside `.metadata`.
expect ( p . sampleChanges [ 0 ] . after . metadata . version ) . toBe ( 1 )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
// Verify no data was modified (dry-run)
const entity = await brain . get ( id1 )
expect ( entity ? . metadata ? . version ) . toBeUndefined ( )
} )
} )
it ( 'should show correct sample before/after' , async ( ) = > {
await brain . add ( { type : NounType . Concept , data : { name : 'Test' } , metadata : { state : 'draft' } } )
const renameMigration : Migration = {
id : 'test-rename' ,
version : '1.0.0' ,
description : 'Rename state to status' ,
applies : 'nouns' ,
transform : ( m ) = > {
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
const bag = m . metadata as Record < string , unknown >
if ( 'state' in bag ) {
const { state , . . . rest } = bag
return { . . . m , metadata : { . . . rest , status : state } }
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
return null
}
}
await withMigrations ( [ renameMigration ] , async ( ) = > {
const preview = await brain . migrate ( { dryRun : true } )
const p = preview as any
expect ( p . sampleChanges . length ) . toBeGreaterThanOrEqual ( 1 )
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
// Find the sample for our entity (it has the 'state' USER field —
// samples carry the VIEW shape, user fields inside `.metadata`)
const sample = p . sampleChanges . find ( ( s : any ) = > s . before . metadata . state === 'draft' )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
expect ( sample ) . toBeDefined ( )
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
expect ( sample . after . metadata . status ) . toBe ( 'draft' )
expect ( sample . after . metadata . state ) . toBeUndefined ( )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
} )
} )
} )
// ─── Apply migrations ─────────────────────────────────────────
describe ( 'Apply migrations' , ( ) = > {
it ( 'should transform noun metadata and return result' , async ( ) = > {
const id = await brain . add ( {
type : NounType . Concept ,
data : { name : 'Alice' } ,
metadata : { priority : 'low' }
} )
const addFieldMigration : Migration = {
id : 'test-add-field' ,
version : '1.0.0' ,
description : 'Add migrated flag to entities with priority' ,
applies : 'nouns' ,
transform : ( m ) = > {
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
if ( 'priority' in bagOf ( m ) && ! ( 'migrated' in bagOf ( m ) ) ) {
return withBag ( m , { migrated : true } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
return null
}
}
await withMigrations ( [ addFieldMigration ] , async ( ) = > {
const result = await brain . migrate ( )
const r = result as any
expect ( r . migrationsApplied ) . toContain ( 'test-add-field' )
expect ( r . entitiesModified ) . toBeGreaterThanOrEqual ( 1 )
// Verify data was actually transformed
const entity = await brain . get ( id )
expect ( entity ? . metadata ? . migrated ) . toBe ( true )
expect ( entity ? . metadata ? . priority ) . toBe ( 'low' )
} )
} )
it ( 'should skip entities that return null from transform' , async ( ) = > {
await brain . add ( { type : NounType . Concept , data : { name : 'Has status' } , metadata : { status : 'active' } } )
await brain . add ( { type : NounType . Concept , data : { name : 'No status' } , metadata : { priority : 'high' } } )
const conditionalMigration : Migration = {
id : 'test-conditional' ,
version : '1.0.0' ,
description : 'Uppercase status field only when present' ,
applies : 'nouns' ,
transform : ( m ) = > {
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
if ( typeof bagOf ( m ) . status === 'string' ) {
return withBag ( m , { status : ( bagOf ( m ) . status as string ) . toUpperCase ( ) } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
return null
}
}
await withMigrations ( [ conditionalMigration ] , async ( ) = > {
const result = await brain . migrate ( )
const r = result as any
// Only 1 entity has 'status', the other has 'priority'
expect ( r . entitiesModified ) . toBeGreaterThanOrEqual ( 1 )
expect ( r . entitiesProcessed ) . toBeGreaterThanOrEqual ( 2 )
} )
} )
it ( 'should run multiple migrations in order' , async ( ) = > {
await brain . add ( { type : NounType . Concept , data : { name : 'Test' } , metadata : { count : 1 } } )
const migration1 : Migration = {
id : 'test-double' ,
version : '1.0.0' ,
description : 'Double count' ,
applies : 'nouns' ,
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
transform : ( m ) = > typeof bagOf ( m ) . count === 'number' ? withBag ( m , { count : ( bagOf ( m ) . count as number ) * 2 } ) : null
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
const migration2 : Migration = {
id : 'test-add-ten' ,
version : '1.1.0' ,
description : 'Add 10 to count' ,
applies : 'nouns' ,
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
transform : ( m ) = > typeof bagOf ( m ) . count === 'number' ? withBag ( m , { count : ( bagOf ( m ) . count as number ) + 10 } ) : null
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
await withMigrations ( [ migration1 , migration2 ] , async ( ) = > {
const result = await brain . migrate ( )
const r = result as any
expect ( r . migrationsApplied ) . toEqual ( [ 'test-double' , 'test-add-ten' ] )
} )
} )
it ( 'should not re-run completed migrations' , async ( ) = > {
await brain . add ( { type : NounType . Concept , data : { name : 'Test' } , metadata : { v : 1 } } )
const migration : Migration = {
id : 'test-increment' ,
version : '1.0.0' ,
description : 'Increment v' ,
applies : 'nouns' ,
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
transform : ( m ) = > typeof bagOf ( m ) . v === 'number' ? withBag ( m , { v : ( bagOf ( m ) . v as number ) + 1 } ) : null
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
await withMigrations ( [ migration ] , async ( ) = > {
// Run once
const result1 = await brain . migrate ( )
expect ( ( result1 as any ) . migrationsApplied ) . toHaveLength ( 1 )
// Run again — should be no-op
const result2 = await brain . migrate ( )
expect ( ( result2 as any ) . migrationsApplied ) . toHaveLength ( 0 )
expect ( ( result2 as any ) . entitiesModified ) . toBe ( 0 )
} )
} )
} )
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
// ─── Pre-migration snapshot tests ─────────────────────────────
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
describe ( 'Pre-migration snapshots (backupTo)' , ( ) = > {
let backupDir : string
beforeEach ( ( ) = > {
backupDir = path . join ( os . tmpdir ( ) , ` brainy-migration-backup- ${ Date . now ( ) } - ${ Math . random ( ) . toString ( 36 ) . slice ( 2 ) } ` )
} )
afterEach ( ( ) = > {
fs . rmSync ( backupDir , { recursive : true , force : true } )
} )
it ( 'should persist a snapshot before migrating and report its path' , async ( ) = > {
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
await brain . add ( { type : NounType . Concept , data : { name : 'Test' } , metadata : { x : 1 } } )
const migration : Migration = {
id : 'test-backup' ,
version : '2.0.0' ,
description : 'Add y field to entities with x' ,
applies : 'nouns' ,
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
transform : ( m ) = > 'x' in bagOf ( m ) && ! ( 'y' in bagOf ( m ) ) ? withBag ( m , { y : 2 } ) : null
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
await withMigrations ( [ migration ] , async ( ) = > {
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
const result = await brain . migrate ( { backupTo : backupDir } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
const r = result as any
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
expect ( r . backupPath ) . toBe ( backupDir )
expect ( r . migrationsApplied ) . toContain ( 'test-backup' )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
// The snapshot is a self-contained store directory
expect ( fs . existsSync ( backupDir ) ) . toBe ( true )
expect ( fs . readdirSync ( backupDir ) . length ) . toBeGreaterThan ( 0 )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
} )
} )
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
it ( 'should restore the pre-migration state wholesale from the snapshot' , async ( ) = > {
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
const id = await brain . add ( { type : NounType . Concept , data : { name : 'Restore Test' } , metadata : { original : true } } )
const migration : Migration = {
id : 'test-restore' ,
version : '3.0.0' ,
description : 'Replace original with migrated' ,
applies : 'nouns' ,
transform : ( m ) = > {
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
if ( bagOf ( m ) . original === true ) {
return withBag ( m , { original : false , migrated : true } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
return null
}
}
await withMigrations ( [ migration ] , async ( ) = > {
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
const result = await brain . migrate ( { backupTo : backupDir } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
const r = result as any
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
expect ( r . backupPath ) . toBe ( backupDir )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
// Verify migration was applied
const migrated = await brain . get ( id )
expect ( migrated ? . metadata ? . migrated ) . toBe ( true )
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
expect ( migrated ? . metadata ? . original ) . toBe ( false )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
// Restore the snapshot — the store returns to its pre-migration state
await brain . restore ( backupDir , { confirm : true } )
const restored = await brain . get ( id )
expect ( restored ? . metadata ? . original ) . toBe ( true )
expect ( restored ? . metadata ? . migrated ) . toBeUndefined ( )
} )
} )
it ( 'should report backupPath null when no backupTo is supplied' , async ( ) = > {
await brain . add ( { type : NounType . Concept , data : { name : 'NoBackup' } , metadata : { q : 1 } } )
const migration : Migration = {
id : 'test-no-backup' ,
version : '4.0.0' ,
description : 'Add field' ,
applies : 'nouns' ,
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
transform : ( m ) = > 'q' in bagOf ( m ) && ! ( 'r' in bagOf ( m ) ) ? withBag ( m , { r : 2 } ) : null
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
}
await withMigrations ( [ migration ] , async ( ) = > {
const result = await brain . migrate ( )
const r = result as any
expect ( r . backupPath ) . toBeNull ( )
expect ( r . migrationsApplied ) . toContain ( 'test-no-backup' )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
} )
} )
} )
// ─── Warning log tests ────────────────────────────────────────
2026-06-17 13:11:41 -07:00
describe ( 'autoMigrate: false' , ( ) = > {
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
it ( 'should log warning when pending migrations exist' , async ( ) = > {
const consoleSpy = vi . spyOn ( console , 'log' ) . mockImplementation ( ( ) = > { } )
const migration : Migration = {
id : 'test-warning' ,
version : '1.0.0' ,
description : 'Test warning' ,
applies : 'nouns' ,
transform : ( ) = > null
}
await withMigrations ( [ migration ] , async ( ) = > {
2026-06-17 13:11:41 -07:00
// autoMigrate defaults to TRUE in 8.0, which would run the migration
// inline during init() (small dataset) instead of logging. Opt out so
// the deferred-notice path is exercised. silent stays false (default)
// so the notice is actually emitted to console.log.
const warnBrain = new Brainy ( { requireSubtype : false , autoMigrate : false , storage : { type : 'memory' } } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
await warnBrain . init ( )
const migrationLogs = consoleSpy . mock . calls
. filter ( call = > typeof call [ 0 ] === 'string' && call [ 0 ] . includes ( 'pending migration' ) )
expect ( migrationLogs . length ) . toBeGreaterThan ( 0 )
await warnBrain . close ( )
} )
consoleSpy . mockRestore ( )
} )
} )
// ─── autoMigrate: true inline ─────────────────────────────────
describe ( 'autoMigrate: true' , ( ) = > {
it ( 'should auto-migrate small datasets during init' , async ( ) = > {
// Create a brain with data, close it
feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1)
Brainy 8.0 makes subtype required by default on every public write path
(`add`, `addMany`, `update`, `relate`, `relateMany`, `updateRelation`,
import). Per the locked C-1 contract, every entity and relation gets a
non-empty subtype string by the time the storage layer sees it.
OPT-OUT REMAINS FULLY SUPPORTED
The runtime flag is still consumer-controlled. Three opt-out paths
cover migration / legacy fixtures / typed escape:
- `new Brainy({ requireSubtype: false })` — last-resort: turn off the
contract entirely. Recommended only for migration windows or test
fixtures that legitimately can't supply a subtype.
- `new Brainy({ requireSubtype: { except: [NounType.Thing, ...] } })` —
per-type allowlist: strict everywhere except the listed types.
- `brain.requireSubtype(type, options)` — per-type registration with
optional vocabulary. Composes with the brain-wide flag.
Default is now `true`. Opt-out is explicit and documented; nothing
silently degrades.
TEST SWEEP
Bulk-applied `requireSubtype: false` to every `new Brainy({...})` call
site across 120 test files. Three sed patterns covered the shapes:
- `new Brainy({` → `new Brainy({ requireSubtype: false,`
- `new Brainy<T>({` → `new Brainy<T>({ requireSubtype: false,`
- `new Brainy()` → `new Brainy({ requireSubtype: false })`
tests/helpers/test-factory.ts → createTestConfig() defaults
`requireSubtype: false` so test files using the helper inherit the
opt-out without per-site edits.
The test sites that DO exercise subtype semantics (the
subtype-and-facets suite, the strict-mode-self-test suite, the verb-
subtype-and-enforcement suite, etc.) already pass real subtypes — they
were the 7.30.x acceptance tests for this contract. Those tests
continue to pass unchanged.
CHANGES
src/brainy.ts
- normalizeConfig() — `requireSubtype` default `false` → `true`.
Comment refreshed to document the three opt-out paths.
tests/* (120 files)
- Bulk-edited brain construction sites. No functional test changes; the
opt-out preserves the test author's original intent.
tests/helpers/test-factory.ts
- createTestConfig() base config gains `requireSubtype: false`.
NO-OP for consumers who were already passing subtype on every write.
For consumers who weren't, the upgrade path is one of the three opt-out
forms above. Migration recipe documented in 8.0 release notes (next
commit).
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1408 / 1409 (same pre-existing race-condition outstanding;
no other regressions from the flip)
2026-06-09 14:58:25 -07:00
const setupBrain = new Brainy ( { requireSubtype : false , storage : { type : 'memory' } , silent : true } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
await setupBrain . init ( )
await setupBrain . add ( { type : NounType . Concept , data : { name : 'AutoTest' } , metadata : { legacy : true } } )
// We can't easily share memory storage between instances,
// so test the migrate() path directly instead
const migration : Migration = {
id : 'test-auto' ,
version : '1.0.0' ,
description : 'Auto migrate test' ,
applies : 'nouns' ,
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
transform : ( m ) = > 'legacy' in bagOf ( m ) ? withBag ( m , { legacy : false , upgraded : true } ) : null
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
await withMigrations ( [ migration ] , async ( ) = > {
const result = await setupBrain . migrate ( )
const r = result as any
expect ( r . migrationsApplied ) . toContain ( 'test-auto' )
expect ( r . entitiesModified ) . toBeGreaterThanOrEqual ( 1 )
} )
await setupBrain . close ( )
} )
} )
// ─── Progress callback ────────────────────────────────────────
describe ( 'Progress callback' , ( ) = > {
it ( 'should call onProgress during migration' , async ( ) = > {
await brain . add ( { type : NounType . Concept , data : { name : 'A' } , metadata : { x : 1 } } )
await brain . add ( { type : NounType . Concept , data : { name : 'B' } , metadata : { x : 2 } } )
const migration : Migration = {
id : 'test-progress' ,
version : '1.0.0' ,
description : 'Add y to entities with x' ,
applies : 'nouns' ,
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
transform : ( m ) = > 'x' in bagOf ( m ) ? withBag ( m , { y : true } ) : null
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
const progressCalls : any [ ] = [ ]
await withMigrations ( [ migration ] , async ( ) = > {
await brain . migrate ( {
onProgress : ( p ) = > progressCalls . push ( p )
} )
} )
expect ( progressCalls . length ) . toBeGreaterThan ( 0 )
expect ( progressCalls [ 0 ] . migrationId ) . toBe ( 'test-progress' )
expect ( progressCalls [ 0 ] . processed ) . toBeGreaterThan ( 0 )
} )
} )
// ─── Multi-tenant independence ─────────────────────────────────
describe ( 'Multi-tenant independence' , ( ) = > {
it ( 'should have independent migration state per instance' , async ( ) = > {
feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1)
Brainy 8.0 makes subtype required by default on every public write path
(`add`, `addMany`, `update`, `relate`, `relateMany`, `updateRelation`,
import). Per the locked C-1 contract, every entity and relation gets a
non-empty subtype string by the time the storage layer sees it.
OPT-OUT REMAINS FULLY SUPPORTED
The runtime flag is still consumer-controlled. Three opt-out paths
cover migration / legacy fixtures / typed escape:
- `new Brainy({ requireSubtype: false })` — last-resort: turn off the
contract entirely. Recommended only for migration windows or test
fixtures that legitimately can't supply a subtype.
- `new Brainy({ requireSubtype: { except: [NounType.Thing, ...] } })` —
per-type allowlist: strict everywhere except the listed types.
- `brain.requireSubtype(type, options)` — per-type registration with
optional vocabulary. Composes with the brain-wide flag.
Default is now `true`. Opt-out is explicit and documented; nothing
silently degrades.
TEST SWEEP
Bulk-applied `requireSubtype: false` to every `new Brainy({...})` call
site across 120 test files. Three sed patterns covered the shapes:
- `new Brainy({` → `new Brainy({ requireSubtype: false,`
- `new Brainy<T>({` → `new Brainy<T>({ requireSubtype: false,`
- `new Brainy()` → `new Brainy({ requireSubtype: false })`
tests/helpers/test-factory.ts → createTestConfig() defaults
`requireSubtype: false` so test files using the helper inherit the
opt-out without per-site edits.
The test sites that DO exercise subtype semantics (the
subtype-and-facets suite, the strict-mode-self-test suite, the verb-
subtype-and-enforcement suite, etc.) already pass real subtypes — they
were the 7.30.x acceptance tests for this contract. Those tests
continue to pass unchanged.
CHANGES
src/brainy.ts
- normalizeConfig() — `requireSubtype` default `false` → `true`.
Comment refreshed to document the three opt-out paths.
tests/* (120 files)
- Bulk-edited brain construction sites. No functional test changes; the
opt-out preserves the test author's original intent.
tests/helpers/test-factory.ts
- createTestConfig() base config gains `requireSubtype: false`.
NO-OP for consumers who were already passing subtype on every write.
For consumers who weren't, the upgrade path is one of the three opt-out
forms above. Migration recipe documented in 8.0 release notes (next
commit).
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1408 / 1409 (same pre-existing race-condition outstanding;
no other regressions from the flip)
2026-06-09 14:58:25 -07:00
const brain1 = new Brainy ( { requireSubtype : false , storage : { type : 'memory' } , silent : true } )
const brain2 = new Brainy ( { requireSubtype : false , storage : { type : 'memory' } , silent : true } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
await brain1 . init ( )
await brain2 . init ( )
await brain1 . add ( { type : NounType . Concept , data : { name : 'User1 data' } , metadata : { v : 1 } } )
await brain2 . add ( { type : NounType . Concept , data : { name : 'User2 data' } , metadata : { v : 1 } } )
const migration : Migration = {
id : 'test-tenant' ,
version : '1.0.0' ,
description : 'Increment v on entities that have it' ,
applies : 'nouns' ,
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
transform : ( m ) = > typeof bagOf ( m ) . v === 'number' ? withBag ( m , { v : ( bagOf ( m ) . v as number ) + 1 } ) : null
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
await withMigrations ( [ migration ] , async ( ) = > {
// Migrate only brain1
const r1 = await brain1 . migrate ( )
expect ( ( r1 as any ) . entitiesModified ) . toBeGreaterThanOrEqual ( 1 )
// brain2 should still have pending migrations
const preview = await brain2 . migrate ( { dryRun : true } )
expect ( ( preview as any ) . pendingMigrations ) . toHaveLength ( 1 )
expect ( ( preview as any ) . affectedEntities ) . toBeGreaterThanOrEqual ( 1 )
} )
await brain1 . close ( )
await brain2 . close ( )
} )
} )
// ─── Verb migration ───────────────────────────────────────────
describe ( 'Verb migration' , ( ) = > {
it ( 'should transform verb metadata when applies is verbs' , async ( ) = > {
const id1 = await brain . add ( { type : NounType . Concept , data : { name : 'A' } } )
const id2 = await brain . add ( { type : NounType . Concept , data : { name : 'B' } } )
await brain . relate ( { from : id1 , to : id2 , type : VerbType . RelatedTo , metadata : { strength : 'weak' } } )
const migration : Migration = {
id : 'test-verb-migration' ,
version : '1.0.0' ,
description : 'Rename strength to intensity' ,
applies : 'verbs' ,
transform : ( m ) = > {
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
const bag = bagOf ( m )
if ( 'strength' in bag ) {
const { strength , . . . rest } = bag
return { . . . m , metadata : { . . . rest , intensity : strength } }
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
return null
}
}
await withMigrations ( [ migration ] , async ( ) = > {
const result = await brain . migrate ( )
expect ( ( result as any ) . migrationsApplied ) . toContain ( 'test-verb-migration' )
// At least the one verb with 'strength' should be modified
expect ( ( result as any ) . entitiesModified ) . toBeGreaterThanOrEqual ( 1 )
} )
} )
} )
// ─── Both nouns and verbs ─────────────────────────────────────
describe ( 'applies: both' , ( ) = > {
it ( 'should transform both nouns and verbs when applies is both' , async ( ) = > {
const id1 = await brain . add ( { type : NounType . Concept , data : { name : 'A' } , metadata : { tag : 'old' } } )
const id2 = await brain . add ( { type : NounType . Concept , data : { name : 'B' } , metadata : { tag : 'old' } } )
await brain . relate ( { from : id1 , to : id2 , type : VerbType . RelatedTo , metadata : { tag : 'old' } } )
const migration : Migration = {
id : 'test-both' ,
version : '1.0.0' ,
description : 'Update tag from old to new' ,
applies : 'both' ,
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
transform : ( m ) = > bagOf ( m ) . tag === 'old' ? withBag ( m , { tag : 'new' } ) : null
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
await withMigrations ( [ migration ] , async ( ) = > {
const result = await brain . migrate ( )
const r = result as any
// At least 2 nouns + 1 verb should be modified
expect ( r . entitiesModified ) . toBeGreaterThanOrEqual ( 3 )
} )
} )
} )
// ─── MigrationRunner unit-level tests ─────────────────────────
describe ( 'MigrationRunner' , ( ) = > {
it ( 'should report no pending migrations when array is empty' , async ( ) = > {
const runner = new MigrationRunner ( ( brain as any ) . storage )
expect ( await runner . hasPendingMigrations ( ) ) . toBe ( false )
} )
it ( 'should report pending migrations when array has entries' , async ( ) = > {
const migration : Migration = {
id : 'runner-test' ,
version : '1.0.0' ,
description : 'Test' ,
applies : 'nouns' ,
transform : ( ) = > null
}
await withMigrations ( [ migration ] , async ( ) = > {
const runner = new MigrationRunner ( ( brain as any ) . storage )
expect ( await runner . hasPendingMigrations ( ) ) . toBe ( true )
expect ( await runner . pendingCount ( ) ) . toBe ( 1 )
expect ( runner . nextMigrationVersion ( ) ) . toBe ( '1.0.0' )
} )
} )
it ( 'should estimate time correctly' , async ( ) = > {
const migration : Migration = {
id : 'estimate-test' ,
version : '1.0.0' ,
description : 'Test' ,
applies : 'nouns' ,
transform : ( ) = > null
}
await withMigrations ( [ migration ] , async ( ) = > {
const runner = new MigrationRunner ( ( brain as any ) . storage )
const preview = await runner . preview ( )
// With 0 entities, estimatedTime should be '0ms' or '<1s'
expect ( preview . estimatedTime ) . toBeDefined ( )
} )
} )
} )
// ─── Error handling ──────────────────────────────────────────────
describe ( 'Error handling' , ( ) = > {
it ( 'should track transform errors without crashing the whole migration' , async ( ) = > {
// Add 3 entities — one will trigger a throw
await brain . add ( { type : NounType . Concept , data : { name : 'Good1' } , metadata : { value : 1 } } )
await brain . add ( { type : NounType . Concept , data : { name : 'Bad' } , metadata : { value : 'crash-me' } } )
await brain . add ( { type : NounType . Concept , data : { name : 'Good2' } , metadata : { value : 3 } } )
const migration : Migration = {
id : 'test-error-handling' ,
version : '1.0.0' ,
description : 'Transform that throws on non-number values' ,
applies : 'nouns' ,
transform : ( m ) = > {
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
if ( 'value' in bagOf ( m ) ) {
if ( typeof bagOf ( m ) . value !== 'number' ) {
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
throw new Error ( 'value must be a number' )
}
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
return withBag ( m , { value : ( bagOf ( m ) . value as number ) * 10 } )
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
}
return null
}
}
await withMigrations ( [ migration ] , async ( ) = > {
const result = await brain . migrate ( )
const r = result as any
// Migration should still complete for the good entities
expect ( r . migrationsApplied ) . toContain ( 'test-error-handling' )
expect ( r . entitiesModified ) . toBeGreaterThanOrEqual ( 2 )
// Errors should be tracked
expect ( r . errors . length ) . toBeGreaterThanOrEqual ( 1 )
expect ( r . errors [ 0 ] . migrationId ) . toBe ( 'test-error-handling' )
expect ( r . errors [ 0 ] . error ) . toContain ( 'value must be a number' )
expect ( r . errors [ 0 ] . entityId ) . toBeDefined ( )
} )
} )
it ( 'should stop early when maxErrors is exceeded' , async ( ) = > {
// Add many entities that will all fail
for ( let i = 0 ; i < 5 ; i ++ ) {
await brain . add ( { type : NounType . Concept , data : { name : ` Fail ${ i } ` } , metadata : { boom : true } } )
}
const migration : Migration = {
id : 'test-max-errors' ,
version : '1.0.0' ,
description : 'Always throws' ,
applies : 'nouns' ,
transform : ( m ) = > {
feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
2026-08-03 16:59:13 -07:00
if ( 'boom' in bagOf ( m ) ) {
feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
throw new Error ( 'deliberate failure' )
}
return null
}
}
await withMigrations ( [ migration ] , async ( ) = > {
const result = await brain . migrate ( { maxErrors : 2 } )
const r = result as any
// Should have stopped at 2 errors
expect ( r . errors . length ) . toBe ( 2 )
// Not all entities should have been processed
expect ( r . entitiesProcessed ) . toBeLessThanOrEqual ( 5 )
} )
} )
} )
// ─── Validation ─────────────────────────────────────────────────
describe ( 'Validation (MigrationRunner.validateMigrations)' , ( ) = > {
it ( 'should accept empty migrations array' , ( ) = > {
expect ( ( ) = > MigrationRunner . validateMigrations ( [ ] ) ) . not . toThrow ( )
} )
it ( 'should accept valid migrations' , ( ) = > {
const valid : Migration [ ] = [
{ id : 'v1' , version : '1.0.0' , description : 'Test' , applies : 'nouns' , transform : ( ) = > null } ,
{ id : 'v2' , version : '1.1.0' , description : 'Test 2' , applies : 'verbs' , transform : ( ) = > null }
]
expect ( ( ) = > MigrationRunner . validateMigrations ( valid ) ) . not . toThrow ( )
} )
it ( 'should reject duplicate migration IDs' , ( ) = > {
const dupes : Migration [ ] = [
{ id : 'same-id' , version : '1.0.0' , description : 'First' , applies : 'nouns' , transform : ( ) = > null } ,
{ id : 'same-id' , version : '1.1.0' , description : 'Second' , applies : 'nouns' , transform : ( ) = > null }
]
expect ( ( ) = > MigrationRunner . validateMigrations ( dupes ) ) . toThrow ( /Duplicate migration id.*same-id/ )
} )
it ( 'should reject invalid applies value' , ( ) = > {
const bad = [
{ id : 'bad-applies' , version : '1.0.0' , description : 'Bad' , applies : 'widgets' as any , transform : ( ) = > null }
]
expect ( ( ) = > MigrationRunner . validateMigrations ( bad ) ) . toThrow ( /invalid applies value.*widgets/ )
} )
it ( 'should reject non-function transform' , ( ) = > {
const bad = [
{ id : 'bad-transform' , version : '1.0.0' , description : 'Bad' , applies : 'nouns' as const , transform : 'not a function' as any }
]
expect ( ( ) = > MigrationRunner . validateMigrations ( bad ) ) . toThrow ( /non-function transform/ )
} )
it ( 'should reject missing required fields' , ( ) = > {
const noId = [
{ id : '' , version : '1.0.0' , description : 'Bad' , applies : 'nouns' as const , transform : ( ) = > null }
]
expect ( ( ) = > MigrationRunner . validateMigrations ( noId ) ) . toThrow ( /missing or invalid id/ )
const noVersion = [
{ id : 'ok' , version : '' , description : 'Bad' , applies : 'nouns' as const , transform : ( ) = > null }
]
expect ( ( ) = > MigrationRunner . validateMigrations ( noVersion ) ) . toThrow ( /missing or invalid version/ )
const noDescription = [
{ id : 'ok' , version : '1.0.0' , description : '' , applies : 'nouns' as const , transform : ( ) = > null }
]
expect ( ( ) = > MigrationRunner . validateMigrations ( noDescription ) ) . toThrow ( /missing or invalid description/ )
} )
} )
} )