fix(8.0): restore() reloads a native entity-id mapper before graphIndex.rebuild()
A logical snapshot restore could silently lose graph edges on a brain backed by a native provider: the graph adjacency keys on the shared id-mapper's interned ints (sourceInt/targetInt), which are derived + never persisted, and restore() never reloaded the mapper — so a native graph rebuild resolved verb endpoints against a stale/empty mapper and dropped edges. restore() now calls entityIdMapper.rebuild() — a new OPTIONAL method on the EntityIdMapperProvider contract — BEFORE rebuilding the indexes, so a native mapper reloads its int<->uuid from the restored binary KV first and the graph resolves endpoints correctly. The JS mapper deliberately has NO rebuild() and needs none: MetadataIndex.rebuild() re-derives it from the restored entities via append-only getOrAssign, consistently with the bitmaps it builds — forcing a reload there would blank a still-referenced mapping and break find() after a same-instance restore. Contract: graphIndex.rebuild() resolves sourceId/targetId -> ints through the shared mapper itself (brainy does not re-feed resolved endpoints); brainy's only job is to ensure the mapper is reloaded first. Test: relationships survive a snapshot round-trip (db-mvcc.test.ts). 84 generation/temporal/visibility tests green; tsc clean.
This commit is contained in:
parent
3783e61b30
commit
4d0b64f455
4 changed files with 72 additions and 2 deletions
|
|
@ -6408,8 +6408,23 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
await this.storage.restoreFromDirectory(path)
|
||||
await this.generationStore.reopenAfterRestore(floorGeneration)
|
||||
|
||||
// Every in-memory index is now stale — rebuild all three from the
|
||||
// restored canonical records (each rebuild clears its own state first).
|
||||
// If the entity-id mapper is a NATIVE provider with a `rebuild()`, reload it
|
||||
// from the restored snapshot FIRST. The graph index resolves every verb
|
||||
// endpoint (sourceId/targetId → stable int) through this mapper, so a native
|
||||
// adjacency (keyed on those ints) needs the mapper to reflect the snapshot's
|
||||
// assignments BEFORE graphIndex.rebuild() — otherwise edges resolve to
|
||||
// stale/missing ints and are silently dropped (CTX-BR-RESTORE-REBUILD). The
|
||||
// JS mapper has no `rebuild()` and needs no reload: MetadataIndex.rebuild()
|
||||
// re-derives it from the restored entities via append-only getOrAssign,
|
||||
// consistently with the bitmaps it builds.
|
||||
const idMapper = this.metadataIndex.getIdMapper() as { rebuild?: () => Promise<void> }
|
||||
if (typeof idMapper.rebuild === 'function') {
|
||||
await idMapper.rebuild()
|
||||
}
|
||||
|
||||
// Every in-memory index is now stale — rebuild all three from the restored
|
||||
// canonical records (each rebuild clears its own state first). The graph
|
||||
// rebuild resolves endpoints via the (now-reloaded, for native) mapper.
|
||||
await Promise.all([
|
||||
this.metadataIndex.rebuild(),
|
||||
this.index.rebuild(),
|
||||
|
|
|
|||
|
|
@ -771,6 +771,18 @@ export interface VectorIndexProvider {
|
|||
*/
|
||||
export interface EntityIdMapperProvider {
|
||||
init(): Promise<void>
|
||||
/**
|
||||
* @description OPTIONAL: reload the uuid↔int mapping from the CURRENT storage,
|
||||
* discarding in-memory state — called by `brain.restore()` AFTER the storage
|
||||
* has been replaced from a snapshot and BEFORE the graph index rebuilds, so
|
||||
* the graph resolves each verb endpoint through a mapper that reflects the
|
||||
* snapshot's assignments (without it, native adjacency edges — keyed on these
|
||||
* ints — resolve to stale/missing ints and are silently dropped). A native
|
||||
* mapper reloads from its restored binary KV; the JS fallback re-reads its
|
||||
* persisted metadata. Optional so a mapper that already reloads via `init()`
|
||||
* stays compatible — `restore()` falls back to `init()` when this is absent.
|
||||
*/
|
||||
rebuild?(): Promise<void>
|
||||
getOrAssign(uuid: string): number
|
||||
getUuid(intId: number): string | undefined
|
||||
getInt(uuid: string): number | undefined
|
||||
|
|
|
|||
|
|
@ -344,6 +344,17 @@ export class EntityIdMapper implements EntityIdMapperProvider {
|
|||
await this.flush()
|
||||
}
|
||||
|
||||
// No `rebuild()` on the JS mapper by design (CTX-BR-RESTORE-REBUILD): after a
|
||||
// `brain.restore()`, `MetadataIndex.rebuild()` re-derives this mapper from the
|
||||
// restored entities via append-only `getOrAssign` (the ints it picks are
|
||||
// internally consistent with the bitmaps it builds), so the JS path needs no
|
||||
// explicit post-restore reload — and forcing one (blanking + reloading) would
|
||||
// drop a still-referenced mapping when the snapshot omits the mapper file. The
|
||||
// `rebuild()` reload is the NATIVE mapper's concern: cor's binary KV holds the
|
||||
// authoritative int↔uuid the native adjacency is keyed on, so it must reload
|
||||
// from the restored KV before the native graph rebuild. `restore()` therefore
|
||||
// calls `rebuild()` only when the provider implements it (see brainy.ts).
|
||||
|
||||
/**
|
||||
* Get statistics about the mapper
|
||||
*/
|
||||
|
|
|
|||
|
|
@ -1096,6 +1096,38 @@ describe('8.0 Db API — generational MVCC', () => {
|
|||
await after.release()
|
||||
})
|
||||
|
||||
it('restore() rebuilds graph adjacency — relationships survive a snapshot round-trip (id-mapper reloaded before graph)', async () => {
|
||||
const { brain } = await openFsBrain()
|
||||
const a = uid('rr-a')
|
||||
const b = uid('rr-b')
|
||||
const c = uid('rr-c')
|
||||
await brain.transact([
|
||||
{ op: 'add', id: a, type: NounType.Person, data: 'a', vector: vec(1), metadata: {} },
|
||||
{ op: 'add', id: b, type: NounType.Person, data: 'b', vector: vec(2), metadata: {} },
|
||||
{ op: 'add', id: c, type: NounType.Document, data: 'c', vector: vec(3), metadata: {} },
|
||||
{ op: 'relate', from: a, to: b, type: VerbType.RelatedTo },
|
||||
{ op: 'relate', from: a, to: c, type: VerbType.References }
|
||||
])
|
||||
expect((await brain.related(a)).map((r) => r.to).sort()).toEqual([b, c].sort())
|
||||
|
||||
const snapDir = path.join(makeTempDir(), 'rel-snapshot')
|
||||
const db = brain.now()
|
||||
await db.persist(snapDir)
|
||||
await db.release()
|
||||
|
||||
// Drop a + its edges live so the restore must genuinely RECONSTRUCT them.
|
||||
await brain.remove(a)
|
||||
expect(await brain.related(a)).toEqual([])
|
||||
|
||||
await brain.restore(snapDir, { confirm: true })
|
||||
|
||||
// a's relationships are back: the graph adjacency rebuilt by resolving each
|
||||
// verb's endpoints (sourceId/targetId → ints) through the id-mapper that
|
||||
// restore() reloaded from the snapshot BEFORE graphIndex.rebuild(). Without
|
||||
// that reload, a native adjacency keyed on those ints loses the edges.
|
||||
expect((await brain.related(a)).map((r) => r.to).sort()).toEqual([b, c].sort())
|
||||
})
|
||||
|
||||
it('Db values are first-class: Brainy.open() + chained with() overlays compose', async () => {
|
||||
const brain = await Brainy.open({ requireSubtype: false, storage: { type: 'memory' } })
|
||||
brains.push(brain)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue