CI / Integration + conformance (Node 22) (push) Waiting to run
CI / Bun (latest) (push) Waiting to run
get(), relate() and update() each carried a "very large metadata" case that
parked an array of 1000 (or 100) elements in the metadata bag and asserted it
came back. The indexable-array bound refuses that shape at the write door now
— an array field mints one posting per element, so an unbounded array is an
unbounded write — and the three cases were failing on the refusal they should
have been pinning.
Each is rewritten to the law that replaced it, in two halves:
- a large SCALAR payload still round-trips whole through the door: a
10,000-character string, 100 sibling fields, a ten-deep nest walked to the
bottom, and an array sitting exactly ON the bound, checked first element
to last;
- an array one element OVER the bound refuses with MetadataArrayTooLargeError
carrying the field, the length and the bound, on the error object AND in
the message. update()'s refusal additionally proves the row is unchanged,
and relate()'s that no relation was written — refused means not written,
not written-then-skipped.
Every length is derived from the imported MAX_INDEXED_ARRAY_LENGTH; none is
typed as a number. That is what made the old cases fragile: 100 read as "over
the bound" and 1000 as "large", and both meanings changed under them when the
constant moved. These follow the constant instead.
Shutdown/close/flush now properly flushes all 4 components in parallel:
metadataIndex, graphIndex, HNSW dirty nodes, and storage counts. Previously
only counts were flushed, causing native provider data loss on restart.
Also:
- Wire roaring, msgpack, entityIdMapper provider consumption from plugins
- Fix allOf filter O(n²) intersection → O(n) Set-based
- Fix ne/exists negation filter to use Set-based exclusion
- Add setMsgpackImplementation() swap in SSTable for native msgpack
- Add setRoaringImplementation() swap for native CRoaring bitmaps
- Add getAllIntIds() to EntityIdMapper for bitmap operations
- Remove TypeAwareHNSWIndex from default index creation path
- Export memory detection utilities from internals
- Clean up 26 permanently-skipped dead tests
Critical Bug Fixes (v3.43.2):
Bug #1 - Import Infinite Loop:
- Fix placeholder entity infinite loop in ImportCoordinator
- Use exact matching instead of fuzzy .includes() for entity names
- Search entities array (not rows) for existing placeholders
- Add duplicate relationship prevention in brain.relate()
Bug #2 - Index Rebuild File Discovery:
- Fix fileSystemStorage to scan sharded subdirectories
- Update getAllNodes() to use getAllShardedFiles()
- Update getAllEdges() to use getAllShardedFiles()
- Update getNodesByNounType() to use getAllShardedFiles()
- Fix getStorageStatus() to use O(1) persisted counts
Additional Improvements:
- Add brain.flush() API for explicit index persistence
- Make GraphAdjacencyIndex.flush() public
- Add auto-flush at end of import pipeline
- Update duplicate relationship test to expect deduplication
Files Modified:
- src/storage/adapters/fileSystemStorage.ts
- src/import/ImportCoordinator.ts
- src/brainy.ts
- src/graph/graphAdjacencyIndex.ts
- tests/unit/brainy/relate.test.ts
Fixed critical bugs affecting test suite:
**Clustering (2 tests fixed)**
- Fixed entity.type field reference bug in _getItemsByField()
- Changed entity.noun to entity.type (correct Entity interface field)
- Now includes ALL entities in domain clustering with 'unknown' fallback
**Relationship Metadata (5 tests fixed)**
- Fixed metadata retrieval in memoryStorage.ts getVerbs()
- Changed metadata.data to metadata.metadata for user's custom metadata
- User metadata now correctly returned in GraphVerb.metadata field
**Delete Relationship Cleanup (2 tests fixed)**
- Added deleteVerbMetadata() method to BaseStorage
- Fixed deleteVerb_internal() in memoryStorage to delete verb metadata
- Relationships now properly cleaned up when entities are deleted
**Validation (1 test fixed)**
- Removed overly restrictive self-referential relationship check
- Self-relationships now allowed (valid in graph systems)
Test results: 27 failures → 17 failures (37% improvement)
All 467 tests now enabled (0 skipped)
- Implement self-configuring validation that adapts to system resources
- Add validation for all CRUD operations (add, update, delete, find, relate)
- Auto-configure limits based on available memory (1GB = 10K limit, 8GB = 80K)
- Monitor and auto-tune performance based on query response times
- Fix multiple type filtering with proper anyOf structure
- Enhance type safety by requiring NounType/VerbType enums
- Fix tests to validate correct behavior (no fake implementations)
- Add comprehensive VALIDATION.md documentation
- Update API_REFERENCE.md with validation rules and examples
- Clarify metadata update behavior (null keeps existing, {} clears)
BREAKING CHANGE: getFieldsForType() now requires NounType enum instead of string
Co-Authored-By: Claude <noreply@anthropic.com>
