soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

fix: metadata index not cleaned up after delete/deleteMany

Browse source 
Three bugs caused deleted entities to persist in the metadata index:

1. idMapper never cleaned up — EntityIdMapper accumulated UUID→int mappings
   permanently. idMapper.getAllIntIds() is used as the universe for ne and
   exists:false operators, so deleted entities returned in those queries
   indefinitely. Fix: removeFromIndex() now calls idMapper.remove(id) and
   idMapper.flush() after all bitmap operations complete (must be last because
   removeFromChunk() reads idMapper.getInt(id) internally).

2. Optional fields indexed as __NULL__ but never unindexed — entityForIndexing
   in add() included confidence, weight, and createdBy as explicit keys even
   when undefined. Object.entries() preserves undefined-valued keys so
   extractIndexableFields() indexed them as '__NULL__' bitmap entries.
   storageMetadata omitted those keys via conditional spreading, so
   removeFromIndex() passed a structure without those keys and never cleaned
   them up. Fix: entityForIndexing now uses conditional spreading for
   confidence, weight, and createdBy matching storageMetadata exactly.

3. result.successful updated before transaction commits — deleteMany() pushed
   ids to result.successful inside the transaction builder, before
   transaction.execute() ran. A rollback would leave result.successful
   containing ids that were never actually deleted. Fix: queued ids are held
   in a local chunkQueued array and moved to result.successful only after
   executeTransaction() resolves without throwing.

Adds regression test suite (14 tests) covering delete() and deleteMany() for
type-index cleanup, ne operator, exists:false operator, optional-field indexing,
and partial deletion correctness.

Reported by wickworks team.
This commit is contained in:
David Snelling 2026-02-18 15:33:56 -08:00
parent cc286ba2fc
commit 1a628daa83
3 changed files with 365 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

10
src/utils/metadataIndex.ts
View file

@ -1667,6 +1667,12 @@ export class MetadataIndexManager {
// Flush all dirty chunks and sparse indices accumulated during remove
await this.flushDirtyMetadata()
// Clean up ID mapper — must happen AFTER bitmap removal since removeFromChunk
// calls idMapper.getInt(id) internally. Skipping this leaves deleted IDs in the
// idMapper universe, causing ne/exists:false queries to return deleted entities.
this.idMapper.remove(id)
await this.idMapper.flush()
} else {
// Remove from all indexes (slower, requires scanning all field indexes)
// This should be rare - prefer providing metadata when removing
@ -1697,6 +1703,10 @@ export class MetadataIndexManager {
// Flush all dirty chunks and sparse indices accumulated during scan-remove
await this.flushDirtyMetadata()
// Clean up ID mapper — must happen AFTER bitmap removal (same reason as fast path above)
this.idMapper.remove(id)
await this.idMapper.flush()
}
}