The JS fallback EntityIdMapper caps at u32::MAX to match the metadata
index's Roaring32 bitmap width. Before this guard, a brain past 4.29 B
entities would silently widen `nextId` into JS's safe-integer range,
truncating ints inside the bitmaps and zeroing query results — the
same silent under-count cortex 3.0's Piece 10 just closed on the
native path. Brainy 8.0 surfaces the overflow loudly instead.
When `nextId` would exceed `U32_ENTITY_ID_MAX`, `getOrAssign` throws
`EntityIdSpaceExceeded` with a message pointing at the cortex 3.0
binary mapper's `idSpace: 'u64'` mode as the migration path (mmap-
backed extendible-hash KV, persists the full u64 range losslessly).
The existing-UUID lookup path bypasses the guard — only fresh
allocations can overflow.
Surfaces via `@soulcraft/brainy/internals`:
- `EntityIdMapper` (already exported, behavior gated)
- `EntityIdSpaceExceeded` (new)
- `U32_ENTITY_ID_MAX` (new constant, = 0xFFFF_FFFF)
This is the lockstep half of cortex 3.0 / Piece 10 / Step 15. Cortex's
`NativeBinaryEntityIdMapperWrapper` ships the U64 binary mapper that
takes over above this ceiling; brainy ships the bright-line failure
that points consumers at it.
New test: tests/unit/utils/entity-id-mapper-u32-ceiling.test.ts (6
assertions covering the constant, the error shape, normal
allocations, the boundary case at exactly u32::MAX, the overflow
throw, and the existing-uuid lookup bypass).
Cortex and other first-party plugins need a shared contract for reading
fields off HNSWNounWithMetadata. Exporting from /internals keeps a single
source of truth and prevents the shape contract from drifting between
packages.
Shutdown/close/flush now properly flushes all 4 components in parallel:
metadataIndex, graphIndex, HNSW dirty nodes, and storage counts. Previously
only counts were flushed, causing native provider data loss on restart.
Also:
- Wire roaring, msgpack, entityIdMapper provider consumption from plugins
- Fix allOf filter O(n²) intersection → O(n) Set-based
- Fix ne/exists negation filter to use Set-based exclusion
- Add setMsgpackImplementation() swap in SSTable for native msgpack
- Add setRoaringImplementation() swap for native CRoaring bitmaps
- Add getAllIntIds() to EntityIdMapper for bitmap operations
- Remove TypeAwareHNSWIndex from default index creation path
- Export memory detection utilities from internals
- Clean up 26 permanently-skipped dead tests
- Wire PluginRegistry into Brainy init() with provider resolution for distance,
metadataIndex, graphIndex, embeddings, roaring, msgpack, and storage adapters
- Add setupStorage() factory that resolves storage:* providers from plugins before
falling back to built-in createStorage()
- Export internals API (setGlobalCache, UnifiedCache, EntityIdMapper, etc.) for
cortex plugin consumption
- Add plugin.test.ts verifying registration, activation, and provider resolution
- Deprecate browser support (OPFS, Web Workers, WASM embeddings) with warnings
in preparation for v8.0 server-only release
- FileSystemStorage: fix setupStorage resolution for mmap-filesystem provider
