fix: resolve import hang and index rebuild data loss bugs

Critical Bug Fixes (v3.43.2):

Bug #1 - Import Infinite Loop:
- Fix placeholder entity infinite loop in ImportCoordinator
- Use exact matching instead of fuzzy .includes() for entity names
- Search entities array (not rows) for existing placeholders
- Add duplicate relationship prevention in brain.relate()

Bug #2 - Index Rebuild File Discovery:
- Fix fileSystemStorage to scan sharded subdirectories
- Update getAllNodes() to use getAllShardedFiles()
- Update getAllEdges() to use getAllShardedFiles()
- Update getNodesByNounType() to use getAllShardedFiles()
- Fix getStorageStatus() to use O(1) persisted counts

Additional Improvements:
- Add brain.flush() API for explicit index persistence
- Make GraphAdjacencyIndex.flush() public
- Add auto-flush at end of import pipeline
- Update duplicate relationship test to expect deduplication

Files Modified:
- src/storage/adapters/fileSystemStorage.ts
- src/import/ImportCoordinator.ts
- src/brainy.ts
- src/graph/graphAdjacencyIndex.ts
- tests/unit/brainy/relate.test.ts
This commit is contained in:
David Snelling 2025-10-14 13:06:32 -07:00
parent 165def11a9
commit dcf20ffa1d
6 changed files with 184 additions and 83 deletions

View file

@ -223,26 +223,29 @@ describe('Brainy.relate()', () => {
describe('edge cases', () => {
it('should handle duplicate relationships', async () => {
// Act - Create same relationship twice
await brain.relate({
const id1 = await brain.relate({
from: entity1Id,
to: entity2Id,
type: 'worksWith',
weight: 0.5
})
await brain.relate({
const id2 = await brain.relate({
from: entity1Id,
to: entity2Id,
type: 'worksWith',
weight: 0.8
})
// Assert - Both relationships should exist
// Assert - Should prevent duplicates (v3.43.2 bug fix)
// Second call should return existing relationship ID instead of creating duplicate
expect(id1).toBe(id2)
const relations = await brain.getRelations({ from: entity1Id })
const matches = relations.filter(r =>
const matches = relations.filter(r =>
r.to === entity2Id && r.type === 'worksWith'
)
expect(matches.length).toBe(2)
expect(matches.length).toBe(1) // Only one relationship should exist
})
it('should handle very long metadata', async () => {