soulcraft/brainy
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

fix: metadata index corruption after restart — 6-point integrity fix

Browse source 
Root cause: metadata index failed to reconstruct after deploy restart
because the field registry file (__metadata_field_registry__) was lost
during an interrupted flush. init() silently assumed the workspace was
empty even though 5000+ entities existed on disk.

Fix 1 (root cause): init() now probes storage for entities when field
registry is missing. If entities exist, triggers rebuild instead of
silently skipping. Never trusts a missing registry as "empty."

Fix 2 (safety net): after rebuildIndexesIfNeeded(), verifies metadata
index entry count matches storage entity count. Forces second rebuild
if mismatch detected.

Fix 3 (prevention): flush() now always saves field registry and
EntityIdMapper, even when no dirty fields exist. These tiny files are
the critical link that init() needs to discover persisted indices.

Fix 5 (safe rebuild): rebuild() no longer deletes the field registry
file before rewriting. If rebuild fails partway, the registry survives
for the next init() to discover and re-trigger rebuild.

Fix 6 (collision guard): EntityIdMapper init() warns when mapper file
is missing but entities exist on disk, preventing silent ID collisions
from nextId starting at 1.
This commit is contained in:
David Snelling 2026-03-24 12:57:11 -07:00
parent e6c5eb6d8b
commit 9d035aceee
3 changed files with 80 additions and 31 deletions
Download patch file Download diff file Expand all files Collapse all files

21
src/brainy.ts
View file

@ -7170,16 +7170,29 @@ export class Brainy<T = any> implements BrainyInterface<T> {
])
const rebuildDuration = Date.now() - rebuildStartTime
const metadataCountAfter = (await this.metadataIndex.getStats()).totalEntries
if (!this.config.silent) {
console.log(
`✅ All indexes rebuilt in ${rebuildDuration}ms:\n` +
` - Metadata: ${await this.metadataIndex.getStats().then(s => s.totalEntries)} entries\n` +
`All indexes rebuilt in ${rebuildDuration}ms:\n` +
` - Metadata: ${metadataCountAfter} entries\n` +
` - HNSW Vector: ${this.index.size()} nodes\n` +
` - Graph Adjacency: ${await this.graphIndex.size()} relationships\n` +
` 💡 Indexes loaded from persisted storage (no recomputation)`
` - Graph Adjacency: ${await this.graphIndex.size()} relationships`
)
}
// Consistency verification: metadata index must match storage entity count.
// If mismatch, the rebuild missed entities — force a second attempt.
if (metadataCountAfter === 0 && totalCount > 0) {
console.error(
`[Brainy] CRITICAL: Metadata index has 0 entries but storage has ${totalCount} entities. ` +
`Forcing second rebuild.`
)
await this.metadataIndex.rebuild()
const secondAttempt = (await this.metadataIndex.getStats()).totalEntries
console.log(`[Brainy] Second rebuild result: ${secondAttempt} entries`)
}
} catch (error) {
console.warn('Warning: Could not rebuild indexes:', error)
// Don't throw - allow system to start even if rebuild fails