fix: flush all native providers on shutdown to prevent data loss

Shutdown/close/flush now properly flushes all 4 components in parallel:
metadataIndex, graphIndex, HNSW dirty nodes, and storage counts. Previously
only counts were flushed, causing native provider data loss on restart.

Also:
- Wire roaring, msgpack, entityIdMapper provider consumption from plugins
- Fix allOf filter O(n²) intersection → O(n) Set-based
- Fix ne/exists negation filter to use Set-based exclusion
- Add setMsgpackImplementation() swap in SSTable for native msgpack
- Add setRoaringImplementation() swap for native CRoaring bitmaps
- Add getAllIntIds() to EntityIdMapper for bitmap operations
- Remove TypeAwareHNSWIndex from default index creation path
- Export memory detection utilities from internals
- Clean up 26 permanently-skipped dead tests
This commit is contained in:
David Snelling 2026-02-01 16:23:49 -08:00
parent b87426409d
commit 773c5171c3
24 changed files with 297 additions and 1268 deletions

View file

@ -270,38 +270,6 @@ describe('VirtualFileSystem - Production Tests', () => {
expect(paths).toContain('/search-test/login.html')
})
// TODO: Investigate "Source entity not found" error in relate() - likely cache/timing issue
it.skip('should filter by metadata', async () => {
// Skip in unit test mode (mocked embeddings don't match file content)
if ((globalThis as any).__BRAINY_UNIT_TEST__) {
console.log('⏭️ Skipping metadata filter test in unit mode')
return
}
const results = await vfs.search('', {
where: {
path: { $startsWith: '/search-test/' },
mimeType: 'application/json'
}
})
expect(results).toHaveLength(1)
expect(results[0].path).toBe('/search-test/config.json')
})
// TODO: Investigate "Source entity not found" error in relate() - likely cache/timing issue
it.skip('should find similar files', async () => {
// Find files similar to auth.js
const similar = await vfs.findSimilar('/search-test/auth.js', {
limit: 3
})
// login.html should be in the results (both about authentication)
// Note: The first result might be auth.js itself (most similar to itself)
expect(similar.length).toBeGreaterThan(0)
const paths = similar.map(r => r.path)
expect(paths).toContain('/search-test/login.html')
})
})
describe('Extended Attributes', () => {