Commit graph

4 commits

Author SHA1 Message Date
cb16a39a0c chore(8.0): Phase A + B — purge all @deprecated APIs + cacheManager dead branches
PHASE A — every @deprecated marker resolved (~25 removed)

src/coreTypes.ts
- GraphVerb: dropped the "@deprecated Will be replaced by HNSWVerbWithMetadata"
  note. GraphVerb IS the canonical contract — every public API path speaks
  it. Removed the `source` and `target` legacy alias fields (renamed `from`
  / `to` callers years ago; no consumers remain).
- StorageAdapter: dropped the "@deprecated Use getNouns() with filter" notes
  from `getNounsByNounType`, `getVerbsBySource`, `getVerbsByTarget`,
  `getVerbsByType`. They were never deprecated in spirit — they're useful
  non-paginated convenience wrappers over the paginated `getNouns()` /
  `getVerbs()` surface. Refreshed JSDoc to explain the role.

src/types/graphTypes.ts
- Mirrored the GraphVerb cleanup: dropped `source` + `target` legacy aliases.
  sourceId + targetId are the canonical fields.

src/import/ImportCoordinator.ts
- Deleted the entire DeprecatedImportOptions interface block (130 LOC). It
  was a v3 → v4 migration tool using the `?: never` trick to force
  compile errors on dropped options. Five major versions in, the
  forced-error gate is no longer pulling its weight.

src/triple/TripleIntelligence.ts
- Deleted `TripleIntelligenceEngine = any` alias. No consumers; superseded
  by `TripleIntelligenceSystem`.

src/storage/cow/binaryDataCodec.ts
- Deleted `wrapBinaryData()`. The COW dispatch layer in `baseStorage.ts`
  routes by key-prefix convention; the old guess-by-JSON-parse codec was
  fragile (compressed bytes can accidentally parse as JSON) and unused.

src/storage/baseStorage.ts
- Refreshed JSDoc on `convertHNSWVerbToGraphVerb()` — the method is alive
  and used internally; the deprecation note was stale.

src/embeddings/wasm/AssetLoader.ts → DELETED
- File was @deprecated since model weights moved into the Candle WASM
  bundle. No consumers. Removed from `embeddings/wasm/index.ts` exports.

src/embeddings/wasm/types.ts
- Dropped @deprecated tags on `TokenizerConfig` + `TokenizedInput` — still
  used by `WordPieceTokenizer` (auxiliary tokenization). Deleted
  `InferenceConfig` (truly dead). Updated `embeddings/wasm/index.ts`
  exports.

src/utils/metadataIndex.ts
- Deleted `getIdsForCriteria()` — pure alias for `getIdsForFilter()`, no
  consumers.

src/interfaces/IIndex.ts
- Removed RebuildOptions.lazy (deprecated and unused; lazy mode is auto-
  selected by available-memory detection).

src/hnsw/hnswIndex.ts
- Removed `getNouns()` (returned a full Map; deprecated in favor of
  pagination years ago and no consumers in src/ or tests/).

PHASE B — cacheManager dead StorageType branches

src/storage/cacheManager.ts
- Collapsed the `isRemoteStorage` flag and its 15 dead conditional branches
  spanning calculateOptimalCacheSize() and calculateOptimalBatchSize().
  After dropping cloud adapters in step 7, `coldStorageType` is never S3
  or REMOTE_API; the branches were dead. Cache sizing and batch sizing now
  honor the filesystem-only reality with simpler heuristics.
- Collapsed `detectWarmStorageType()` + `detectColdStorageType()` from
  ~40 LOC of environment-+-availability branching to 2-line returns of
  `StorageType.FILESYSTEM`. Brainy 8.0 ships filesystem + memory only.

NOT YET — Phases C-G in follow-up commits

C: storageAutoConfig.ts + zeroConfig + extensibleConfig + sharedConfigManager
D: TODO/FIXME sweep across src/
E: skipped tests + the parallel-test race condition
F: docs deep clean (BATCHING, augmentations, READMEs)
G: browser support drop (the last 2 @deprecated)

VERIFICATION

- npx tsc --noEmit: clean
- npm test: 1408 / 1409 (same pre-existing race-condition outstanding)
2026-06-09 15:33:56 -07:00
364360d447 fix: exclude __words__ keyword index from corruption detection and getStats()
The __words__ keyword index stores 50-5000 entries per entity (one per
word), which inflated avg entries/entity well above the corruption
threshold of 100. This caused:

1. validateConsistency() to falsely detect corruption on every startup,
   triggering unnecessary clearAllIndexData() + rebuild() cycles
2. getStats() to log false "Metadata index may be corrupted" warnings
   and report inflated totalEntries/totalIds stats

Both methods now skip __words__ when counting, so stats and health
checks reflect metadata fields only (noun, type, createdAt, etc.).
Keyword search is unaffected since the __words__ field index itself
is not modified.
2026-01-27 15:38:21 -08:00
46c6af3f21 feat: implement always-adaptive caching with getCacheStats monitoring
Replaces lazy mode concept with always-adaptive caching strategy:

- Rename getLazyModeStats() → getCacheStats() with enhanced metrics
- Change lazyModeEnabled boolean → cachingStrategy enum ('preloaded' | 'on-demand')
- Update preloading threshold from 30% to 80% for better cache utilization
- Add comprehensive production monitoring and diagnostics
- Add memory detection for containers (Docker/K8s cgroups v1/v2)
- Add adaptive memory sizing from 2GB to 128GB+ systems

Breaking changes: None (backward compatible, deprecated lazy option ignored)

New APIs:
- getCacheStats(): Comprehensive cache performance statistics
- cachingStrategy field: Transparent strategy reporting
- Enhanced fairness metrics and memory pressure monitoring

Documentation:
- Add migration guide for v3.36.0
- Add operations/capacity-planning.md for enterprise deployments
- Update all examples and troubleshooting guides
- Rename monitor-lazy-mode.ts → monitor-cache-performance.ts
2025-10-10 14:09:30 -07:00
6a4d1aeb2b feat: implement HNSW index rebuild and unified index interface
Fixes critical bugs causing data loss after container restarts:
- Bug #1: GraphAdjacencyIndex rebuild now properly called
- Bug #2: Improved early return logic (checks actual storage data)
- Bug #4: HNSW index now has production-grade rebuild mechanism

New features:
- Production-grade HNSW rebuild() with O(N) restoration algorithm
- Unified IIndex interface for consistent lifecycle management
- Parallel index rebuilds (HNSW, Graph, Metadata in parallel)
- HNSW persistence methods across all 5 storage adapters
- Comprehensive integration tests with 9 test scenarios

Performance improvements:
- 20 entities: 8ms rebuild time
- Handles millions of entities via cursor-based pagination
- O(N) restoration vs O(N log N) rebuilding from scratch

All changes are production-ready with no mocks, stubs, or TODOs.
2025-10-10 11:15:17 -07:00