David Snelling
fb80808f44
BREAKING CHANGE: Removed getAllNouns() and getAllVerbs() from StorageAdapter interface These methods could cause expensive full scans on cloud storage (S3/R2) leading to high costs and performance issues. Replaced with safe paginated methods. Changes: - Remove getAllNouns/getAllVerbs from StorageAdapter interface and implementations - Add internal optimization methods for intelligent preloading when safe - Fix OPFS storage file naming consistency (.json extension) - Fix S3 high-volume mode detection thresholds (was too aggressive) - Fix TypeScript compilation errors with async methods - Update all tests to use paginated methods Performance: - Add smart dataset size detection for automatic optimization - Maintain all internal performance optimizations through safe preloading - Only preload data in read-only mode or when dataset is small (<10k entities) Fixes: - Fix intelligent verb scoring tests metadata structure - Fix S3 storage getVerbsBySource/Target/Type methods - Fix memory usage in search operations using pagination Docs: - Add comprehensive storage architecture documentation - Document known bash redirection issue - Update README with architecture doc link All affected tests passing |
||
|---|---|---|
| .. | ||
| augmentations | ||
| chat | ||
| cli/commands | ||
| connectors | ||
| cortex | ||
| distributed | ||
| errors | ||
| examples | ||
| hnsw | ||
| mcp | ||
| shared | ||
| storage | ||
| types | ||
| universal | ||
| utils | ||
| augmentationFactory.ts | ||
| augmentationPipeline.ts | ||
| augmentationRegistry.ts | ||
| augmentationRegistryLoader.ts | ||
| brainyData.ts | ||
| browserFramework.minimal.ts | ||
| browserFramework.ts | ||
| coreTypes.ts | ||
| demo.ts | ||
| index.ts | ||
| pipeline.ts | ||
| sequentialPipeline.ts | ||
| setup.ts | ||
| unified.ts | ||
| worker.js | ||
| worker.ts | ||