Three bugs caused deleted entities to persist in the metadata index:
1. idMapper never cleaned up — EntityIdMapper accumulated UUID→int mappings
permanently. idMapper.getAllIntIds() is used as the universe for ne and
exists:false operators, so deleted entities returned in those queries
indefinitely. Fix: removeFromIndex() now calls idMapper.remove(id) and
idMapper.flush() after all bitmap operations complete (must be last because
removeFromChunk() reads idMapper.getInt(id) internally).
2. Optional fields indexed as __NULL__ but never unindexed — entityForIndexing
in add() included confidence, weight, and createdBy as explicit keys even
when undefined. Object.entries() preserves undefined-valued keys so
extractIndexableFields() indexed them as '__NULL__' bitmap entries.
storageMetadata omitted those keys via conditional spreading, so
removeFromIndex() passed a structure without those keys and never cleaned
them up. Fix: entityForIndexing now uses conditional spreading for
confidence, weight, and createdBy matching storageMetadata exactly.
3. result.successful updated before transaction commits — deleteMany() pushed
ids to result.successful inside the transaction builder, before
transaction.execute() ran. A rollback would leave result.successful
containing ids that were never actually deleted. Fix: queued ids are held
in a local chunkQueued array and moved to result.successful only after
executeTransaction() resolves without throwing.
Adds regression test suite (14 tests) covering delete() and deleteMany() for
type-index cleanup, ne operator, exists:false operator, optional-field indexing,
and partial deletion correctness.
Reported by wickworks team.
David Snelling
1a628daa83