Replace JavaScript Sets with hardware-accelerated RoaringBitmap32 for metadata indexes. Key improvements: - 1.4x average speedup, up to 3.3x on 10K entities - 90% memory reduction (40 bytes/UUID → 4 bytes/int) - Hardware-accelerated multi-field intersection via SIMD (AVX2/SSE4.2) - EntityIdMapper for bidirectional UUID ↔ integer mapping - Portable serialization format Benchmark results (1,000 queries): - 10K entities: 3.74ms → 1.14ms (3.3x faster, 90% memory savings) - 100K entities: 2.60ms → 1.78ms (1.5x faster, 88% memory savings) Implementation: - Add EntityIdMapper class for UUID/int mapping with persistence - Modify ChunkData to use Map<string, RoaringBitmap32> - Add getIdsForMultipleFields() for fast bitmap intersection - Include comprehensive tests (25 tests passing) - Add performance benchmark comparing Set vs Roaring Technical details: - roaring@2.4.0 dependency - Maintains backward compatibility - All queries still return UUID strings - Automatic persistence via storage adapter |
||
|---|---|---|
| .. | ||
| augmentations | ||
| brainy | ||
| neural | ||
| utils | ||
| brainy-core.unit.test.ts | ||
| type-matching.unit.test.ts | ||