Commit graph

3 commits

Author SHA1 Message Date
4fcef7b8ed fix: import dedup off-switch honesty + brain-owned lifecycle for the background pass
The post-import background deduplication pass (a merge-DELETE writer,
debounced ~5 minutes after import) had four lifecycle defects:

- enableDeduplication: false did not gate the background schedule — an
  import that explicitly opted out could still have entities auto-removed
  minutes later. The flag now gates both the inline and background passes.
- Each import() constructed its own coordinator-owned deduplicator, so the
  debounce never spanned imports (N imports = N delete timers). The brain
  now owns a single lazy instance (getBackgroundDeduplicator).
- close() never cancelled pending dedup; a delete pass could fire against
  a closed brain. close() now cancels it first.
- The 5-minute timer held the process open (exit-hang class); now unref'd.

Four regression tests pin the contract (background-dedup-lifecycle);
import guides document that false disables both passes.
2026-07-18 10:40:45 -07:00
1f7e365a4e chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
02c80a045b perf: optimize imports with background deduplication (12-24x speedup)
- Remove O(n²) deduplication from import path for 12-24x faster imports
- Implement BackgroundDeduplicator with 3-tier strategy (ID/Name/Similarity)
- Sequential tier processing reduces entity set after each pass
- Auto-schedules 5 minutes after imports (debounced, zero config)
- Import-scoped deduplication prevents cross-contamination

GraphAdjacencyIndex improvements:
- Fix concurrent rebuild race condition with promise-based locking
- Fix removeVerb() by filtering deleted IDs in query methods
- Replace console.* with prodLog for silent mode compatibility

Performance impact:
- Import speed: O(n²) → O(n) complexity
- 400 entities: 24 min → 2 min (12x faster)
- 1000 entities: >2 hours → 5 min (24x faster)
- Background dedup uses existing indexes (TypeAware HNSW, MetadataIndexManager)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-11 14:10:14 -08:00