92d9420a5c
fix: eliminate cloud storage write amplification and rate limiting
...
brain.add() was generating 26-40 immediate cloud writes per call, causing
HTTP 429 rate limit errors and high latency on GCS/S3/R2/Azure. Three-layer
fix: (1) deferred metadata writes with dirty-marking, (2) MetadataWriteBuffer
for write coalescing, (3) retry/backoff on all cloud storage adapters.
2026-01-31 09:09:36 -08:00
364360d447
fix: exclude __words__ keyword index from corruption detection and getStats()
...
The __words__ keyword index stores 50-5000 entries per entity (one per
word), which inflated avg entries/entity well above the corruption
threshold of 100. This caused:
1. validateConsistency() to falsely detect corruption on every startup,
triggering unnecessary clearAllIndexData() + rebuild() cycles
2. getStats() to log false "Metadata index may be corrupted" warnings
and report inflated totalEntries/totalIds stats
Both methods now skip __words__ when counting, so stats and health
checks reflect metadata fields only (noun, type, createdAt, etc.).
Keyword search is unaffected since the __words__ field index itself
is not modified.
2026-01-27 15:38:21 -08:00
da7d2ed29d
feat: migrate embeddings to Candle WASM + remove semantic type inference
...
Major architectural changes:
1. EMBEDDINGS ENGINE (ONNX → Candle WASM):
- Replace ONNX Runtime with Rust Candle compiled to WASM
- Embedded model in WASM binary (no external downloads)
- Quantized Q8 precision with <50MB memory footprint
- Zero-download, offline-first operation
- Same embedding quality (all-MiniLM-L6-v2)
2. REMOVE SEMANTIC TYPE INFERENCE:
- Delete embeddedKeywordEmbeddings.ts (14MB of pre-computed embeddings)
- Remove typeAwareQueryPlanner.ts and semanticTypeInference.ts
- Remove VerbExactMatchSignal (uses keyword embeddings)
- Update SmartRelationshipExtractor to 3 signals (55%/30%/15% weights)
API CHANGES (requires v7.0.0):
- Removed: inferTypes(), inferNouns(), inferVerbs(), inferIntent()
- Removed: getSemanticTypeInference(), SemanticTypeInference class
- Removed: TypeInference, SemanticTypeInferenceOptions types
Users can still use natural language queries in find() - they just
need to specify type explicitly for type-optimized searches.
PACKAGE SIZE IMPACT:
- Compressed: 90.1 MB → 86.2 MB (-4.3%)
- Uncompressed: 114.4 MB → 100.3 MB (-12%)
- ~448K lines of code removed
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 12:52:34 -08:00
00aae8023c
chore(release): 4.0.0
...
Major release: Enterprise-scale cost optimization and performance features
Features:
- Cloud storage lifecycle management (GCS Autoclass, AWS Intelligent-Tiering, Azure)
- Batch operations (1000x faster deletions: 533 entities/sec vs 0.5/sec)
- FileSystem compression (60-80% space savings with gzip)
- OPFS quota monitoring for browser storage
- Enhanced CLI system (47 commands, 9 storage management commands)
Cost Impact:
- Up to 96% storage cost savings
- $138,000/year → $5,940/year @ 500TB scale
Breaking Changes: NONE
- 100% backward compatible
- All new features are opt-in
- No migration required
2025-10-17 14:48:34 -07:00
46c6af3f21
feat: implement always-adaptive caching with getCacheStats monitoring
...
Replaces lazy mode concept with always-adaptive caching strategy:
- Rename getLazyModeStats() → getCacheStats() with enhanced metrics
- Change lazyModeEnabled boolean → cachingStrategy enum ('preloaded' | 'on-demand')
- Update preloading threshold from 30% to 80% for better cache utilization
- Add comprehensive production monitoring and diagnostics
- Add memory detection for containers (Docker/K8s cgroups v1/v2)
- Add adaptive memory sizing from 2GB to 128GB+ systems
Breaking changes: None (backward compatible, deprecated lazy option ignored)
New APIs:
- getCacheStats(): Comprehensive cache performance statistics
- cachingStrategy field: Transparent strategy reporting
- Enhanced fairness metrics and memory pressure monitoring
Documentation:
- Add migration guide for v3.36.0
- Add operations/capacity-planning.md for enterprise deployments
- Update all examples and troubleshooting guides
- Rename monitor-lazy-mode.ts → monitor-cache-performance.ts
2025-10-10 14:09:30 -07:00
