David Snelling
abc17397b1
BREAKING CHANGE: Removed getAllNouns() and getAllVerbs() from StorageAdapter interface These methods could cause expensive full scans on cloud storage (S3/R2) leading to high costs and performance issues. Replaced with safe paginated methods. Changes: - Remove getAllNouns/getAllVerbs from StorageAdapter interface and implementations - Add internal optimization methods for intelligent preloading when safe - Fix OPFS storage file naming consistency (.json extension) - Fix S3 high-volume mode detection thresholds (was too aggressive) - Fix TypeScript compilation errors with async methods - Update all tests to use paginated methods Performance: - Add smart dataset size detection for automatic optimization - Maintain all internal performance optimizations through safe preloading - Only preload data in read-only mode or when dataset is small (<10k entities) Fixes: - Fix intelligent verb scoring tests metadata structure - Fix S3 storage getVerbsBySource/Target/Type methods - Fix memory usage in search operations using pagination Docs: - Add comprehensive storage architecture documentation - Document known bash redirection issue - Update README with architecture doc link All affected tests passing |
||
|---|---|---|
| .. | ||
| mocks | ||
| api-integration.test.ts | ||
| auto-configuration.test.ts | ||
| base-hnsw-test.ts | ||
| brainy-chat.test.ts | ||
| core.test.ts | ||
| database-operations.test.ts | ||
| dimension-standardization.test.ts | ||
| distributed-caching.test.ts | ||
| distributed-config-migration.test.ts | ||
| distributed.test.ts | ||
| edge-cases.test.ts | ||
| emergency-high-volume-test.js | ||
| environment.browser.test.ts | ||
| environment.node.test.ts | ||
| error-handling.test.ts | ||
| filter-discovery.test.ts | ||
| filter-test.ts | ||
| frozen-flag.test.ts | ||
| high-volume-test.ts | ||
| intelligent-verb-scoring.test.ts | ||
| json-search-test.js | ||
| metadata-filter-debug.test.ts | ||
| metadata-filter-environments.test.ts | ||
| metadata-filter.test.ts | ||
| metadata-performance.test.ts | ||
| multi-environment.test.ts | ||
| opfs-storage.test.ts | ||
| package-install.test.ts | ||
| package-size-breakdown.test.ts | ||
| package-size-limit.test.ts | ||
| pagination.test.ts | ||
| performance-improvements.test.ts | ||
| performance.test.ts | ||
| s3-storage.test.ts | ||
| service-statistics.test.ts | ||
| setup.ts | ||
| simple-metadata-test.ts | ||
| specialized-scenarios.test.ts | ||
| statistics-storage.test.ts | ||
| statistics.test.ts | ||
| storage-adapter-coverage.test.ts | ||
| storage-adapters.test.ts | ||
| test-matrix.md | ||
| throttling-metrics.test.ts | ||
| type-utils.test.ts | ||
| vector-operations.test.ts | ||
| verify-custom-models.js | ||
| verify-model-loading.js | ||
| verify-model-priority-simple.js | ||
| write-only-direct-reads.test.ts | ||