feat: stable EntityIdMapper — rebuild() no longer renumbers UUID→int
Previously metadataIndex.rebuild() called idMapper.clear() which reset nextId to 1 and renumbered every UUID by re-insertion order. Any consumer that had persisted int-keyed data against the old map was silently invalidated — and 2.4.0's vector mmap store (#20), graph link compression (#21), and column-store JS↔native interchange all need persisted int indices that survive a rebuild. Remove the unconditional clear() in rebuild(). The rebuild already re-iterates every entity via idMapper.getOrAssign(uuid), which returns the existing int unchanged for known UUIDs. Stale UUID→int entries for entities no longer in storage persist as harmless memory overhead; a dedicated prune step can be added if it ever matters. clearAllIndexData() — the explicit nuclear recovery path — keeps its existing idMapper.clear() call (renumbering is intentional there), and now logs a prodLog.warn making it explicit that any persisted int-keyed data is invalidated and must be rebuilt from canonical sources. Strengthened the EntityIdMapper class JSDoc to document the stability guarantee as a contract — append-only getOrAssign, monotonic nextId, remove() leaves permanent holes, rebuild() never renumbers, only clear() does. Added tests/regression/entity-id-mapper-stability.test.ts pinning down the five-point contract: (1) single-rebuild stability; (2) many-rebuild stability; (3) post-rebuild adds get fresh monotonic ints; (4) removes leave permanent holes — new entities never recycle; (5) clearAllIndexData() explicitly renumbers (the documented destructive path). Foundation for 2.4.0 #2-#4. Full test suite (62 files, 1417 tests) green.
This commit is contained in:
parent
df7d739008
commit
b2408cb5a6
3 changed files with 209 additions and 10 deletions
|
|
@ -2635,13 +2635,20 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
this.verbCountsByTypeFixed.fill(0)
|
||||
this.typeFieldAffinity.clear()
|
||||
|
||||
// Clear EntityIdMapper
|
||||
// Clear EntityIdMapper. This is the explicit destructive path: the caller
|
||||
// asked for nuclear recovery of a corrupted index, so renumbering UUIDs is
|
||||
// intentional. Persisted int-keyed data (vector-mmap slots, graph
|
||||
// link-compression encodings) is invalidated by this op — the warning
|
||||
// below makes that explicit. Rebuild on its own does NOT clear the mapper.
|
||||
await this.idMapper.clear()
|
||||
|
||||
// Clear chunk manager cache
|
||||
this.chunkManager.clearCache()
|
||||
|
||||
prodLog.info(`✅ Cleared ${deletedCount} field indexes and all in-memory state`)
|
||||
prodLog.warn('⚠️ EntityIdMapper was cleared — any persisted int-keyed data ' +
|
||||
'(vector mmap slots, graph link-compression encodings, etc.) is now stale ' +
|
||||
'and must be rebuilt from canonical sources.')
|
||||
prodLog.info('⚠️ Run brain.index.rebuild() to recreate the index from entity data')
|
||||
}
|
||||
|
||||
|
|
@ -3032,8 +3039,19 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
prodLog.info(`Cleared ${existingFields.length} field indexes from storage`)
|
||||
}
|
||||
|
||||
// Clear EntityIdMapper to start fresh
|
||||
await this.idMapper.clear()
|
||||
// EntityIdMapper is intentionally NOT cleared here. Rebuild re-iterates
|
||||
// every entity in storage and calls idMapper.getOrAssign(uuid), which
|
||||
// returns the existing int for known UUIDs (no renumbering). This is the
|
||||
// foundational stability guarantee — vector-mmap slot indices, graph
|
||||
// link-compression encodings, and any other persisted int-keyed data
|
||||
// remain valid across a rebuild. Previously this line reset nextId to 1
|
||||
// and renumbered every UUID by re-insertion order, silently breaking
|
||||
// any consumer that had persisted int-keyed data against the old map.
|
||||
// Stale entries for UUIDs no longer in storage persist (harmless memory
|
||||
// overhead); a dedicated prune step can be added if it ever matters.
|
||||
// The destructive wipe is still available via clearAllIndexData() →
|
||||
// idMapper.clear(), which is the explicit "recovery" path with the
|
||||
// appropriate warning about invalidating persisted int-keyed data.
|
||||
|
||||
// Clear chunk manager cache
|
||||
this.chunkManager.clearCache()
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue