David Snelling
02c80a045b
- Remove O(n²) deduplication from import path for 12-24x faster imports - Implement BackgroundDeduplicator with 3-tier strategy (ID/Name/Similarity) - Sequential tier processing reduces entity set after each pass - Auto-schedules 5 minutes after imports (debounced, zero config) - Import-scoped deduplication prevents cross-contamination GraphAdjacencyIndex improvements: - Fix concurrent rebuild race condition with promise-based locking - Fix removeVerb() by filtering deleted IDs in query methods - Replace console.* with prodLog for silent mode compatibility Performance impact: - Import speed: O(n²) → O(n) complexity - 400 entities: 24 min → 2 min (12x faster) - 1000 entities: >2 hours → 5 min (24x faster) - Background dedup uses existing indexes (TypeAware HNSW, MetadataIndexManager) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
38 lines
982 B
TypeScript
38 lines
982 B
TypeScript
/**
|
|
* Unified Import System
|
|
*
|
|
* Single entry point for importing any file format into Brainy with:
|
|
* - Auto-detection of formats
|
|
* - Dual storage (VFS + Knowledge Graph)
|
|
* - Shared entities across imports (deduplication)
|
|
* - Simple, powerful API
|
|
*/
|
|
|
|
export { ImportCoordinator } from './ImportCoordinator.js'
|
|
export { FormatDetector, SupportedFormat, DetectionResult } from './FormatDetector.js'
|
|
export { EntityDeduplicator } from './EntityDeduplicator.js'
|
|
export { BackgroundDeduplicator } from './BackgroundDeduplicator.js'
|
|
export { ImportHistory } from './ImportHistory.js'
|
|
|
|
export type {
|
|
ImportSource,
|
|
ImportOptions,
|
|
ImportProgress,
|
|
ImportResult
|
|
} from './ImportCoordinator.js'
|
|
|
|
export type {
|
|
EntityCandidate,
|
|
DuplicateMatch,
|
|
EntityDeduplicationOptions,
|
|
MergeResult
|
|
} from './EntityDeduplicator.js'
|
|
|
|
export type {
|
|
DeduplicationStats
|
|
} from './BackgroundDeduplicator.js'
|
|
|
|
export type {
|
|
ImportHistoryEntry,
|
|
RollbackResult
|
|
} from './ImportHistory.js'
|