brainy

soulcraft/brainy

Fork 0

Commit graph

Author	SHA1	Message	Date
David Snelling	46c6af3f21	feat: implement always-adaptive caching with getCacheStats monitoring Replaces lazy mode concept with always-adaptive caching strategy: - Rename getLazyModeStats() → getCacheStats() with enhanced metrics - Change lazyModeEnabled boolean → cachingStrategy enum ('preloaded' \| 'on-demand') - Update preloading threshold from 30% to 80% for better cache utilization - Add comprehensive production monitoring and diagnostics - Add memory detection for containers (Docker/K8s cgroups v1/v2) - Add adaptive memory sizing from 2GB to 128GB+ systems Breaking changes: None (backward compatible, deprecated lazy option ignored) New APIs: - getCacheStats(): Comprehensive cache performance statistics - cachingStrategy field: Transparent strategy reporting - Enhanced fairness metrics and memory pressure monitoring Documentation: - Add migration guide for v3.36.0 - Add operations/capacity-planning.md for enterprise deployments - Update all examples and troubleshooting guides - Rename monitor-lazy-mode.ts → monitor-cache-performance.ts	2025-10-10 14:09:30 -07:00
David Snelling	1966c39f24	fix: move metadata routing to base class, fix GCS/S3 system key crashes Critical fix for GCS/S3 native storage adapters crashing on metadata index keys. Problem: - GCS and S3 adapters crashed with "Invalid UUID format" errors - System keys like __metadata_field_index__status are NOT UUIDs - Adapters incorrectly tried to shard all metadata keys as UUIDs Solution: - Move sharding/routing logic from adapters to BaseStorage class - Add analyzeKey() method to detect system keys vs entity UUIDs - System keys route to _system/ directory (no sharding) - Entity UUIDs route to sharded directories (256 shards) - All adapters now implement 4 primitive operations: * writeObjectToPath(path, data) * readObjectFromPath(path) * deleteObjectFromPath(path) * listObjectsUnderPath(prefix) Benefits: - Impossible for future adapters to repeat this mistake - Zero breaking changes, full backward compatibility - No data migration required - Cleaner architecture with better separation of concerns Updated adapters: GcsStorage, S3CompatibleStorage, OPFSStorage, FileSystemStorage, MemoryStorage Added: docs/architecture/data-storage-architecture.md Updated: README.md with architecture docs link	2025-10-09 13:10:06 -07:00

Author

SHA1

Message

Date

David Snelling

46c6af3f21

feat: implement always-adaptive caching with getCacheStats monitoring

Replaces lazy mode concept with always-adaptive caching strategy:

- Rename getLazyModeStats() → getCacheStats() with enhanced metrics
- Change lazyModeEnabled boolean → cachingStrategy enum ('preloaded' | 'on-demand')
- Update preloading threshold from 30% to 80% for better cache utilization
- Add comprehensive production monitoring and diagnostics
- Add memory detection for containers (Docker/K8s cgroups v1/v2)
- Add adaptive memory sizing from 2GB to 128GB+ systems

Breaking changes: None (backward compatible, deprecated lazy option ignored)

New APIs:
- getCacheStats(): Comprehensive cache performance statistics
- cachingStrategy field: Transparent strategy reporting
- Enhanced fairness metrics and memory pressure monitoring

Documentation:
- Add migration guide for v3.36.0
- Add operations/capacity-planning.md for enterprise deployments
- Update all examples and troubleshooting guides
- Rename monitor-lazy-mode.ts → monitor-cache-performance.ts

2025-10-10 14:09:30 -07:00

David Snelling

1966c39f24

fix: move metadata routing to base class, fix GCS/S3 system key crashes

Critical fix for GCS/S3 native storage adapters crashing on metadata index keys.

Problem:
- GCS and S3 adapters crashed with "Invalid UUID format" errors
- System keys like __metadata_field_index__status are NOT UUIDs
- Adapters incorrectly tried to shard all metadata keys as UUIDs

Solution:
- Move sharding/routing logic from adapters to BaseStorage class
- Add analyzeKey() method to detect system keys vs entity UUIDs
- System keys route to _system/ directory (no sharding)
- Entity UUIDs route to sharded directories (256 shards)
- All adapters now implement 4 primitive operations:
  * writeObjectToPath(path, data)
  * readObjectFromPath(path)
  * deleteObjectFromPath(path)
  * listObjectsUnderPath(prefix)

Benefits:
- Impossible for future adapters to repeat this mistake
- Zero breaking changes, full backward compatibility
- No data migration required
- Cleaner architecture with better separation of concerns

Updated adapters: GcsStorage, S3CompatibleStorage, OPFSStorage,
FileSystemStorage, MemoryStorage

Added: docs/architecture/data-storage-architecture.md
Updated: README.md with architecture docs link

2025-10-09 13:10:06 -07:00

2 commits