fix: metadata index corruption after restart — 6-point integrity fix

Root cause: metadata index failed to reconstruct after deploy restart
because the field registry file (__metadata_field_registry__) was lost
during an interrupted flush. init() silently assumed the workspace was
empty even though 5000+ entities existed on disk.

Fix 1 (root cause): init() now probes storage for entities when field
registry is missing. If entities exist, triggers rebuild instead of
silently skipping. Never trusts a missing registry as "empty."

Fix 2 (safety net): after rebuildIndexesIfNeeded(), verifies metadata
index entry count matches storage entity count. Forces second rebuild
if mismatch detected.

Fix 3 (prevention): flush() now always saves field registry and
EntityIdMapper, even when no dirty fields exist. These tiny files are
the critical link that init() needs to discover persisted indices.

Fix 5 (safe rebuild): rebuild() no longer deletes the field registry
file before rewriting. If rebuild fails partway, the registry survives
for the next init() to discover and re-trigger rebuild.

Fix 6 (collision guard): EntityIdMapper init() warns when mapper file
is missing but entities exist on disk, preventing silent ID collisions
from nextId starting at 1.
This commit is contained in:
David Snelling 2026-03-24 12:57:11 -07:00
parent e6c5eb6d8b
commit 9d035aceee
3 changed files with 80 additions and 31 deletions

View file

@ -62,6 +62,22 @@ export class EntityIdMapper {
// Rebuild maps from serialized data
this.uuidToInt = new Map(Object.entries(data.uuidToInt).map(([k, v]) => [k, Number(v)]))
this.intToUuid = new Map(Object.entries(data.intToUuid).map(([k, v]) => [Number(k), v]))
} else {
// Guard: mapper file missing but entities may exist on disk.
// If we start from nextId=1 with existing entities, roaring bitmap
// queries will use wrong integer IDs → silent data corruption.
// Probe storage to detect this case and log a warning.
try {
const probe = await this.storage.getNouns({ pagination: { limit: 1, offset: 0 } })
if ((probe.totalCount ?? 0) > 0 || probe.items.length > 0) {
console.warn(
`[EntityIdMapper] Mapper file missing but entities exist on disk. ` +
`IDs will be rebuilt during metadata index reconstruction.`
)
}
} catch {
// Storage not ready
}
}
} catch (error) {
// First time initialization - maps are empty, nextId = 1