fix(8.0): restore() reloads a native entity-id mapper before graphIndex.rebuild()
A logical snapshot restore could silently lose graph edges on a brain backed by a native provider: the graph adjacency keys on the shared id-mapper's interned ints (sourceInt/targetInt), which are derived + never persisted, and restore() never reloaded the mapper — so a native graph rebuild resolved verb endpoints against a stale/empty mapper and dropped edges. restore() now calls entityIdMapper.rebuild() — a new OPTIONAL method on the EntityIdMapperProvider contract — BEFORE rebuilding the indexes, so a native mapper reloads its int<->uuid from the restored binary KV first and the graph resolves endpoints correctly. The JS mapper deliberately has NO rebuild() and needs none: MetadataIndex.rebuild() re-derives it from the restored entities via append-only getOrAssign, consistently with the bitmaps it builds — forcing a reload there would blank a still-referenced mapping and break find() after a same-instance restore. Contract: graphIndex.rebuild() resolves sourceId/targetId -> ints through the shared mapper itself (brainy does not re-feed resolved endpoints); brainy's only job is to ensure the mapper is reloaded first. Test: relationships survive a snapshot round-trip (db-mvcc.test.ts). 84 generation/temporal/visibility tests green; tsc clean.
This commit is contained in:
parent
3783e61b30
commit
4d0b64f455
4 changed files with 72 additions and 2 deletions
|
|
@ -6408,8 +6408,23 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
await this.storage.restoreFromDirectory(path)
|
await this.storage.restoreFromDirectory(path)
|
||||||
await this.generationStore.reopenAfterRestore(floorGeneration)
|
await this.generationStore.reopenAfterRestore(floorGeneration)
|
||||||
|
|
||||||
// Every in-memory index is now stale — rebuild all three from the
|
// If the entity-id mapper is a NATIVE provider with a `rebuild()`, reload it
|
||||||
// restored canonical records (each rebuild clears its own state first).
|
// from the restored snapshot FIRST. The graph index resolves every verb
|
||||||
|
// endpoint (sourceId/targetId → stable int) through this mapper, so a native
|
||||||
|
// adjacency (keyed on those ints) needs the mapper to reflect the snapshot's
|
||||||
|
// assignments BEFORE graphIndex.rebuild() — otherwise edges resolve to
|
||||||
|
// stale/missing ints and are silently dropped (CTX-BR-RESTORE-REBUILD). The
|
||||||
|
// JS mapper has no `rebuild()` and needs no reload: MetadataIndex.rebuild()
|
||||||
|
// re-derives it from the restored entities via append-only getOrAssign,
|
||||||
|
// consistently with the bitmaps it builds.
|
||||||
|
const idMapper = this.metadataIndex.getIdMapper() as { rebuild?: () => Promise<void> }
|
||||||
|
if (typeof idMapper.rebuild === 'function') {
|
||||||
|
await idMapper.rebuild()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every in-memory index is now stale — rebuild all three from the restored
|
||||||
|
// canonical records (each rebuild clears its own state first). The graph
|
||||||
|
// rebuild resolves endpoints via the (now-reloaded, for native) mapper.
|
||||||
await Promise.all([
|
await Promise.all([
|
||||||
this.metadataIndex.rebuild(),
|
this.metadataIndex.rebuild(),
|
||||||
this.index.rebuild(),
|
this.index.rebuild(),
|
||||||
|
|
|
||||||
|
|
@ -771,6 +771,18 @@ export interface VectorIndexProvider {
|
||||||
*/
|
*/
|
||||||
export interface EntityIdMapperProvider {
|
export interface EntityIdMapperProvider {
|
||||||
init(): Promise<void>
|
init(): Promise<void>
|
||||||
|
/**
|
||||||
|
* @description OPTIONAL: reload the uuid↔int mapping from the CURRENT storage,
|
||||||
|
* discarding in-memory state — called by `brain.restore()` AFTER the storage
|
||||||
|
* has been replaced from a snapshot and BEFORE the graph index rebuilds, so
|
||||||
|
* the graph resolves each verb endpoint through a mapper that reflects the
|
||||||
|
* snapshot's assignments (without it, native adjacency edges — keyed on these
|
||||||
|
* ints — resolve to stale/missing ints and are silently dropped). A native
|
||||||
|
* mapper reloads from its restored binary KV; the JS fallback re-reads its
|
||||||
|
* persisted metadata. Optional so a mapper that already reloads via `init()`
|
||||||
|
* stays compatible — `restore()` falls back to `init()` when this is absent.
|
||||||
|
*/
|
||||||
|
rebuild?(): Promise<void>
|
||||||
getOrAssign(uuid: string): number
|
getOrAssign(uuid: string): number
|
||||||
getUuid(intId: number): string | undefined
|
getUuid(intId: number): string | undefined
|
||||||
getInt(uuid: string): number | undefined
|
getInt(uuid: string): number | undefined
|
||||||
|
|
|
||||||
|
|
@ -344,6 +344,17 @@ export class EntityIdMapper implements EntityIdMapperProvider {
|
||||||
await this.flush()
|
await this.flush()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// No `rebuild()` on the JS mapper by design (CTX-BR-RESTORE-REBUILD): after a
|
||||||
|
// `brain.restore()`, `MetadataIndex.rebuild()` re-derives this mapper from the
|
||||||
|
// restored entities via append-only `getOrAssign` (the ints it picks are
|
||||||
|
// internally consistent with the bitmaps it builds), so the JS path needs no
|
||||||
|
// explicit post-restore reload — and forcing one (blanking + reloading) would
|
||||||
|
// drop a still-referenced mapping when the snapshot omits the mapper file. The
|
||||||
|
// `rebuild()` reload is the NATIVE mapper's concern: cor's binary KV holds the
|
||||||
|
// authoritative int↔uuid the native adjacency is keyed on, so it must reload
|
||||||
|
// from the restored KV before the native graph rebuild. `restore()` therefore
|
||||||
|
// calls `rebuild()` only when the provider implements it (see brainy.ts).
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Get statistics about the mapper
|
* Get statistics about the mapper
|
||||||
*/
|
*/
|
||||||
|
|
|
||||||
|
|
@ -1096,6 +1096,38 @@ describe('8.0 Db API — generational MVCC', () => {
|
||||||
await after.release()
|
await after.release()
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('restore() rebuilds graph adjacency — relationships survive a snapshot round-trip (id-mapper reloaded before graph)', async () => {
|
||||||
|
const { brain } = await openFsBrain()
|
||||||
|
const a = uid('rr-a')
|
||||||
|
const b = uid('rr-b')
|
||||||
|
const c = uid('rr-c')
|
||||||
|
await brain.transact([
|
||||||
|
{ op: 'add', id: a, type: NounType.Person, data: 'a', vector: vec(1), metadata: {} },
|
||||||
|
{ op: 'add', id: b, type: NounType.Person, data: 'b', vector: vec(2), metadata: {} },
|
||||||
|
{ op: 'add', id: c, type: NounType.Document, data: 'c', vector: vec(3), metadata: {} },
|
||||||
|
{ op: 'relate', from: a, to: b, type: VerbType.RelatedTo },
|
||||||
|
{ op: 'relate', from: a, to: c, type: VerbType.References }
|
||||||
|
])
|
||||||
|
expect((await brain.related(a)).map((r) => r.to).sort()).toEqual([b, c].sort())
|
||||||
|
|
||||||
|
const snapDir = path.join(makeTempDir(), 'rel-snapshot')
|
||||||
|
const db = brain.now()
|
||||||
|
await db.persist(snapDir)
|
||||||
|
await db.release()
|
||||||
|
|
||||||
|
// Drop a + its edges live so the restore must genuinely RECONSTRUCT them.
|
||||||
|
await brain.remove(a)
|
||||||
|
expect(await brain.related(a)).toEqual([])
|
||||||
|
|
||||||
|
await brain.restore(snapDir, { confirm: true })
|
||||||
|
|
||||||
|
// a's relationships are back: the graph adjacency rebuilt by resolving each
|
||||||
|
// verb's endpoints (sourceId/targetId → ints) through the id-mapper that
|
||||||
|
// restore() reloaded from the snapshot BEFORE graphIndex.rebuild(). Without
|
||||||
|
// that reload, a native adjacency keyed on those ints loses the edges.
|
||||||
|
expect((await brain.related(a)).map((r) => r.to).sort()).toEqual([b, c].sort())
|
||||||
|
})
|
||||||
|
|
||||||
it('Db values are first-class: Brainy.open() + chained with() overlays compose', async () => {
|
it('Db values are first-class: Brainy.open() + chained with() overlays compose', async () => {
|
||||||
const brain = await Brainy.open({ requireSubtype: false, storage: { type: 'memory' } })
|
const brain = await Brainy.open({ requireSubtype: false, storage: { type: 'memory' } })
|
||||||
brains.push(brain)
|
brains.push(brain)
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue