fix: flush all native providers on shutdown to prevent data loss

Shutdown/close/flush now properly flushes all 4 components in parallel:
metadataIndex, graphIndex, HNSW dirty nodes, and storage counts. Previously
only counts were flushed, causing native provider data loss on restart.

Also:
- Wire roaring, msgpack, entityIdMapper provider consumption from plugins
- Fix allOf filter O(n²) intersection → O(n) Set-based
- Fix ne/exists negation filter to use Set-based exclusion
- Add setMsgpackImplementation() swap in SSTable for native msgpack
- Add setRoaringImplementation() swap for native CRoaring bitmaps
- Add getAllIntIds() to EntityIdMapper for bitmap operations
- Remove TypeAwareHNSWIndex from default index creation path
- Export memory detection utilities from internals
- Clean up 26 permanently-skipped dead tests

tests/unit/import/preserve-source-fix.test.ts

@@ -111,31 +111,4 @@ describe('Import preserveSource Fix (v5.1.2)', () => {
     fs.unlinkSync(jsonFilePath)
   })
 
-  it.skip('should work with large text files', async () => {
-    // Create a large CSV (above inline threshold of 100KB)
-    const rows = []
-    for (let i = 0; i < 5000; i++) {
-      rows.push(`row${i},value${i},data${i}`)
-    }
-    const largeCsvContent = 'col1,col2,col3\n' + rows.join('\n')
-
-    const largeFilePath = path.join(tempDir, 'large.csv')
-    fs.writeFileSync(largeFilePath, largeCsvContent)
-
-    // Import large file
-    await brain.import(largeFilePath, {
-      format: 'csv',
-      vfsPath: '/large-file',
-      preserveSource: true,
-      enableNeuralExtraction: false,
-      createEntities: true
-    })
-
-    // Verify file was preserved
-    const content = await brain.vfs.readFile('/large-file/_source.csv')
-    expect(content.toString('utf-8')).toBe(largeCsvContent)
-
-    // Cleanup
-    fs.unlinkSync(largeFilePath)
-  })
 })