feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law
The write side of the law, ruled 2026-08-03: data is either in main space where developers can use anything, or it is in system.*. - The reserved-name write door DIES: add/update/relate/updateRelation metadata bags accept EVERY name (confidence, type, id, data, level, content, ...) as ordinary user fields — indexed, filterable, sortable, aggregatable, identical to any other field. The remap/enforce/warn machinery, the reservedFieldPolicy config (now a typed init refusal), and the compile-time metadata key bans are all removed. The one write refusal left: keys spelled 'system.*' (namespace forgery), now enforced on all four write doors. - STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag nested verbatim under 'metadata', sealed by a format stamp — by-name storage discrimination is unsound once colliders are admitted. Legacy flat records stay readable forever through the shape-aware splitters (sound for them: the old door refused colliders). Time travel rides the same split (generation store snapshots whole records). - Name-based index exclusions DIE: user frame indexes every name; the excludeFields/indexedFields knobs and their silent-[] holes are gone; bulk-payload protection is value-shape only, uniform across names. - Consumer-sweep findings fixed in the same wave: per-type counts read the frozen 'system.type' column (addToIndex sort, affinity tracking, cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for pre-rebuild reads); resolveHiddenIds addresses 'system.visibility' (bare 'visibility' was a silent no-op under the law — VFS/system entities leaked into default reads). - Fidelity fallout fixed in the owning layers: readEntityFieldAddress reads the bag first (colliders were absent-shadowed by its own guard) and never serves system addresses from the bag; blob history refs read the bag shape-aware; migration transforms now receive ONE normalized view (engine fields + nested bag) regardless of stored era, and stray flat-habit keys refuse with the fix in the message. - THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as gates-green): all ten collider names + plumbing names written as user fields, verified verbatim + queryable across live reads, flush+reopen, a forced epoch rebuild, and asOf time travel; relation mirror; forgery refusals; legacy flat-record compat. 8/8 green. Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance 27/27 (exit 0) · consumer test sweep migrated (10 files).
This commit is contained in:
parent
48a6130a50
commit
24bf6cdbc5
32 changed files with 1355 additions and 1905 deletions
|
|
@ -9,6 +9,67 @@ import type { BaseStorage } from '../storage/baseStorage.js'
|
|||
import type { NounMetadata, VerbMetadata } from '../coreTypes.js'
|
||||
import type { Migration, MigrationState, MigrationPreview, MigrationResult, MigrateOptions, MigrationError } from './types.js'
|
||||
import { MIGRATIONS } from './migrations.js'
|
||||
import {
|
||||
splitNounMetadataRecord,
|
||||
splitVerbMetadataRecord,
|
||||
buildNounMetadataRecord,
|
||||
buildVerbMetadataRecord,
|
||||
RESERVED_ENTITY_FIELDS,
|
||||
RESERVED_RELATION_FIELDS
|
||||
} from '../types/reservedFields.js'
|
||||
|
||||
const RESERVED_NOUN_SET: ReadonlySet<string> = new Set(RESERVED_ENTITY_FIELDS)
|
||||
const RESERVED_VERB_SET: ReadonlySet<string> = new Set(RESERVED_RELATION_FIELDS)
|
||||
|
||||
/**
|
||||
* Normalize a stored record (either era: legacy flat OR v2 nested-bag) into
|
||||
* THE transform view — the one shape every migration transform receives:
|
||||
* engine fields top-level, the user's metadata bag nested under `metadata`.
|
||||
* Transforms never see the storage era; a migration written today works on
|
||||
* a brain of any age.
|
||||
*/
|
||||
function toTransformView(
|
||||
record: Record<string, unknown>,
|
||||
kind: 'noun' | 'verb'
|
||||
): Record<string, unknown> {
|
||||
const { reserved, custom } =
|
||||
kind === 'noun' ? splitNounMetadataRecord(record) : splitVerbMetadataRecord(record)
|
||||
return { ...reserved, metadata: { ...custom } }
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert a transform's returned view back into a stamped v2 stored record.
|
||||
* LOUD CONTRACT: user fields belong inside `.metadata` — a stray top-level
|
||||
* key that is not an engine field is a migration bug under the
|
||||
* field-addressing law (pre-law transforms wrote user fields flat), and it
|
||||
* refuses with the fix in the message rather than silently dropping or
|
||||
* silently storing it as an engine key.
|
||||
*/
|
||||
function fromTransformView(
|
||||
view: Record<string, unknown>,
|
||||
kind: 'noun' | 'verb'
|
||||
): Record<string, unknown> {
|
||||
const reservedSet = kind === 'noun' ? RESERVED_NOUN_SET : RESERVED_VERB_SET
|
||||
const engine: Record<string, unknown> = {}
|
||||
for (const [key, value] of Object.entries(view)) {
|
||||
if (key === 'metadata') continue
|
||||
if (!reservedSet.has(key)) {
|
||||
throw new Error(
|
||||
`migration transform returned a top-level key '${key}' that is not an ` +
|
||||
`engine field — under the field-addressing law user fields live inside ` +
|
||||
`.metadata (return { ...view, metadata: { ...view.metadata, ${key}: … } }).`
|
||||
)
|
||||
}
|
||||
engine[key] = value
|
||||
}
|
||||
const bag =
|
||||
view.metadata && typeof view.metadata === 'object' && !Array.isArray(view.metadata)
|
||||
? (view.metadata as Record<string, unknown>)
|
||||
: {}
|
||||
return kind === 'noun'
|
||||
? buildNounMetadataRecord(engine, bag)
|
||||
: buildVerbMetadataRecord(engine, bag)
|
||||
}
|
||||
|
||||
const MIGRATION_STATE_KEY = '__migration_state__'
|
||||
const PREVIEW_SAMPLE_SIZE = 5
|
||||
|
|
@ -125,14 +186,16 @@ export class MigrationRunner {
|
|||
const entityMeta = metadataBatch.get(entity.id)
|
||||
if (!entityMeta) continue
|
||||
|
||||
const metadata = entityMeta as Record<string, unknown>
|
||||
const result = this.applyTransforms(metadata, nounMigrations)
|
||||
// Transforms see THE view (engine fields + nested user bag),
|
||||
// never the raw storage era.
|
||||
const view = toTransformView(entityMeta as Record<string, unknown>, 'noun')
|
||||
const result = this.applyTransforms(view, nounMigrations)
|
||||
if (result !== null) {
|
||||
affectedEntities++
|
||||
if (sampleChanges.length < PREVIEW_SAMPLE_SIZE) {
|
||||
sampleChanges.push({
|
||||
id: entity.id,
|
||||
before: { ...metadata },
|
||||
before: view,
|
||||
after: result
|
||||
})
|
||||
}
|
||||
|
|
@ -157,14 +220,14 @@ export class MigrationRunner {
|
|||
const verbMeta = await this.storage.getVerbMetadata(verb.id)
|
||||
if (!verbMeta) continue
|
||||
|
||||
const metadata = verbMeta as Record<string, unknown>
|
||||
const result = this.applyTransforms(metadata, verbMigrations)
|
||||
const view = toTransformView(verbMeta as Record<string, unknown>, 'verb')
|
||||
const result = this.applyTransforms(view, verbMigrations)
|
||||
if (result !== null) {
|
||||
affectedEntities++
|
||||
if (sampleChanges.length < PREVIEW_SAMPLE_SIZE) {
|
||||
sampleChanges.push({
|
||||
id: verb.id,
|
||||
before: { ...metadata },
|
||||
before: view,
|
||||
after: result
|
||||
})
|
||||
}
|
||||
|
|
@ -289,9 +352,16 @@ export class MigrationRunner {
|
|||
if (!entityMeta) continue
|
||||
|
||||
try {
|
||||
const transformed = migration.transform(entityMeta as Record<string, unknown>)
|
||||
const transformed = migration.transform(
|
||||
toTransformView(entityMeta as Record<string, unknown>, 'noun')
|
||||
)
|
||||
if (transformed !== null) {
|
||||
await this.storage.saveNounMetadata(entity.id, transformed as NounMetadata)
|
||||
// Re-stamp as a v2 record (also upgrades legacy records touched
|
||||
// by a migration onto the nested-bag shape).
|
||||
await this.storage.saveNounMetadata(
|
||||
entity.id,
|
||||
fromTransformView(transformed, 'noun') as NounMetadata
|
||||
)
|
||||
modified++
|
||||
}
|
||||
} catch (err) {
|
||||
|
|
@ -357,9 +427,14 @@ export class MigrationRunner {
|
|||
if (!metadata) continue
|
||||
|
||||
try {
|
||||
const transformed = migration.transform(metadata as Record<string, unknown>)
|
||||
const transformed = migration.transform(
|
||||
toTransformView(metadata as Record<string, unknown>, 'verb')
|
||||
)
|
||||
if (transformed !== null) {
|
||||
await this.storage.saveVerbMetadata(verb.id, transformed as VerbMetadata)
|
||||
await this.storage.saveVerbMetadata(
|
||||
verb.id,
|
||||
fromTransformView(transformed, 'verb') as VerbMetadata
|
||||
)
|
||||
modified++
|
||||
}
|
||||
} catch (err) {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue