David Snelling
3e8b9aacc8
Major architectural improvements and critical bug fixes: ## COW Always-On Architecture - Removed cowEnabled flag from BaseStorage (COW cannot be disabled) - Eliminated marker file system (checkClearMarker, createClearMarker) - Simplified all code paths to assume COW is always enabled - COW automatically re-initializes after clear() operations ## Critical Bug Fix: Cloud Storage clear() - Fixed GCS clear() using correct paths (branches/ instead of entities/nouns/) - Fixed S3Compatible clear() path structure - Fixed R2 clear() implementation - Fixed Azure, FileSystem, OPFS, Memory clear() COW flag handling - clear() now deletes: branches/, _cow/, _system/ - Result: Cloud buckets can now be fully cleared (previously impossible) ## Container Memory Detection - Auto-detect Docker/K8s/Cloud Run memory limits (cgroup v1/v2) - Smart memory allocation (75% graph data, 25% query operations) - Environment variable support (CLOUD_RUN_MEMORY, MEMORY_LIMIT) - Production-grade containerized deployment support ## CommitLog streamHistory Feature - Added streamable commit history with pagination - Efficient memory usage for large commit histories - Support for branch filtering and time ranges ## Comprehensive Storage Documentation - Complete v5.11.0 file structure reference - Detailed path construction algorithms - 8 common storage scenarios with examples - Type-first storage, sharding, COW architecture explained - Public docs: docs/architecture/data-storage-architecture.md (1063 lines) ## Files Modified (14 files) - All 8 storage adapters (GCS, S3, R2, Azure, FS, OPFS, Memory, Historical) - BaseStorage core architecture - CommitLog with streaming - Brainy memory configuration - Parameter validation with container detection - Storage architecture documentation ## Breaking Changes NONE - COW was already enabled by default. This removes the ability to disable it. ## Migration No action required. Upgrade and clear() will work correctly on cloud storage. ## Impact - Users can now clear cloud storage buckets completely - No more corrupted buckets after clear() operations - Container deployments automatically optimize memory allocation - COW is mandatory and always enabled (safer, simpler) v5.11.0 - Production ready |
||
|---|---|---|
| .. | ||
| all-apis-comprehensive.test.ts | ||
| api-parameter-validation.test.ts | ||
| azure-storage.test.ts | ||
| batchImportWithRelations.test.ts | ||
| brainy-complete.integration.test.ts | ||
| brainy-core.integration.test.ts | ||
| brainy-phase1c-integration.test.ts | ||
| clear-persistence.test.ts | ||
| commit-ref-bug.test.ts | ||
| commit-state-capture.test.ts | ||
| count-synchronization.test.ts | ||
| cow-commit-storage.test.ts | ||
| cow-full-integration.test.ts | ||
| empty-tree-bug.test.ts | ||
| entity-confidence-weight.test.ts | ||
| find-unified-integration.test.ts | ||
| fork-persistence.test.ts | ||
| gcs-native-storage.test.ts | ||
| gcs-persistence-fix.test.ts | ||
| get-relations-fix.test.ts | ||
| history-ref-resolution-bug.test.ts | ||
| hnsw-rebuild.test.ts | ||
| image-import.test.ts | ||
| initial-commit-null-parent.test.ts | ||
| memory-enhancements-v5.11.0.test.ts | ||
| metadata-vector-exclusion.test.ts | ||
| phase3TypeFirstQuery.integration.test.ts | ||
| readAfterWrite.test.ts | ||
| relationship-intelligence.test.ts | ||
| remaining-apis.test.ts | ||
| s3-storage.test.ts | ||
| smart-import.test.ts | ||
| typeAwareHNSW.integration.test.ts | ||
| typeInference.hybrid.integration.test.ts | ||
| typeInference.integration.test.ts | ||
| versioning.test.ts | ||
| vfs-and-graph-entities.test.ts | ||
| vfs-api-wiring.test.ts | ||
| vfs-debug.test.ts | ||
| vfs-historical-reads.test.ts | ||
| vfs-import-verification.test.ts | ||
| vfs-knowledge-separation.test.ts | ||