David Snelling
abc17397b1
BREAKING CHANGE: Removed getAllNouns() and getAllVerbs() from StorageAdapter interface These methods could cause expensive full scans on cloud storage (S3/R2) leading to high costs and performance issues. Replaced with safe paginated methods. Changes: - Remove getAllNouns/getAllVerbs from StorageAdapter interface and implementations - Add internal optimization methods for intelligent preloading when safe - Fix OPFS storage file naming consistency (.json extension) - Fix S3 high-volume mode detection thresholds (was too aggressive) - Fix TypeScript compilation errors with async methods - Update all tests to use paginated methods Performance: - Add smart dataset size detection for automatic optimization - Maintain all internal performance optimizations through safe preloading - Only preload data in read-only mode or when dataset is small (<10k entities) Fixes: - Fix intelligent verb scoring tests metadata structure - Fix S3 storage getVerbsBySource/Target/Type methods - Fix memory usage in search operations using pagination Docs: - Add comprehensive storage architecture documentation - Document known bash redirection issue - Update README with architecture doc link All affected tests passing |
||
|---|---|---|
| .. | ||
| COMPATIBILITY.md | ||
| CONCURRENCY_ANALYSIS.md | ||
| ENVIRONMENT_TESTING.md | ||
| METADATA_HANDLING.md | ||
| model-bundling-analysis.md | ||
| REALTIME_UPDATES.md | ||
| SCALING_STRATEGY.md | ||
| STATISTICS.md | ||
| STORAGE_AND_RETRIEVAL_ARCHITECTURE.md | ||
| STORAGE_CONCURRENCY_ANALYSIS.md | ||
| STORAGE_TESTING.md | ||
| TECHNICAL_GUIDES.md | ||
| TESTING.md | ||
| THREADING.md | ||
| THROTTLING_METRICS.md | ||
| USE_MODEL_LOADING_EXPLANATION.md | ||
| VECTOR_DIMENSION_STANDARDIZATION.md | ||
| VITEST_IMPROVEMENTS.md | ||