soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions
brainy/src/utils
History
David Snelling 3e8b9aacc8 feat: COW always-on architecture + cloud storage clear() fix (v5.11.0) 
Major architectural improvements and critical bug fixes:

## COW Always-On Architecture
- Removed cowEnabled flag from BaseStorage (COW cannot be disabled)
- Eliminated marker file system (checkClearMarker, createClearMarker)
- Simplified all code paths to assume COW is always enabled
- COW automatically re-initializes after clear() operations

## Critical Bug Fix: Cloud Storage clear()
- Fixed GCS clear() using correct paths (branches/ instead of entities/nouns/)
- Fixed S3Compatible clear() path structure
- Fixed R2 clear() implementation
- Fixed Azure, FileSystem, OPFS, Memory clear() COW flag handling
- clear() now deletes: branches/, _cow/, _system/
- Result: Cloud buckets can now be fully cleared (previously impossible)

## Container Memory Detection
- Auto-detect Docker/K8s/Cloud Run memory limits (cgroup v1/v2)
- Smart memory allocation (75% graph data, 25% query operations)
- Environment variable support (CLOUD_RUN_MEMORY, MEMORY_LIMIT)
- Production-grade containerized deployment support

## CommitLog streamHistory Feature
- Added streamable commit history with pagination
- Efficient memory usage for large commit histories
- Support for branch filtering and time ranges

## Comprehensive Storage Documentation
- Complete v5.11.0 file structure reference
- Detailed path construction algorithms
- 8 common storage scenarios with examples
- Type-first storage, sharding, COW architecture explained
- Public docs: docs/architecture/data-storage-architecture.md (1063 lines)

## Files Modified (14 files)
- All 8 storage adapters (GCS, S3, R2, Azure, FS, OPFS, Memory, Historical)
- BaseStorage core architecture
- CommitLog with streaming
- Brainy memory configuration
- Parameter validation with container detection
- Storage architecture documentation

## Breaking Changes
NONE - COW was already enabled by default. This removes the ability to disable it.

## Migration
No action required. Upgrade and clear() will work correctly on cloud storage.

## Impact
- Users can now clear cloud storage buckets completely
- No more corrupted buckets after clear() operations
- Container deployments automatically optimize memory allocation
- COW is mandatory and always enabled (safer, simpler)

v5.11.0 - Production ready
2025-11-18 13:44:02 -08:00
..
adaptiveBackpressure.ts fix: prevent circuit breaker activation and data loss during bulk imports 2025-10-30 08:54:04 -07:00
adaptiveSocketManager.ts feat: add node: protocol to all Node.js built-in imports for bundler compatibility 2025-09-17 14:20:21 -07:00
autoConfiguration.ts feat: add node: protocol to all Node.js built-in imports for bundler compatibility 2025-09-17 14:20:21 -07:00
BoundedRegistry.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
brainyTypes.ts feat: implement comprehensive type safety system with BrainyTypes API 2025-09-01 09:37:36 -07:00
crypto.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
deletedItemsIndex.ts feat: comprehensive metadata namespace architecture and cleanup system 2025-08-27 15:38:48 -07:00
distance.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
embedding.ts fix: correct Node.js version references from 24 to 22 in comments and code 2025-10-20 11:43:31 -07:00
enhancedLogger.ts feat: Brainy 3.0 - Triple Intelligence Release 2025-09-15 11:06:16 -07:00
ensureDeleted.ts feat: comprehensive metadata namespace architecture and cleanup system 2025-08-27 15:38:48 -07:00
entityIdMapper.ts fix: v4.8.1 critical bug fixes for update() and clustering 2025-10-27 17:01:37 -07:00
environment.ts fix: correct Node.js version references from 24 to 22 in comments and code 2025-10-20 11:43:31 -07:00
fieldNameTracking.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
fieldTypeInference.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
import-progress-tracker.ts feat: comprehensive import progress tracking for all 7 formats 2025-10-24 14:45:46 -07:00
index.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
intelligentTypeMapper.ts feat: Stage 3 CANONICAL taxonomy with 169 types (v5.5.0) 2025-11-06 09:02:23 -08:00
jsonProcessing.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
logger.ts feat: add complete silent mode for TUI applications 2025-09-16 13:18:49 -07:00
memoryDetection.ts feat: implement always-adaptive caching with getCacheStats monitoring 2025-10-10 14:09:30 -07:00
metadataFilter.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
metadataIndex.ts docs: label all performance claims as MEASURED vs PROJECTED (NO FAKE CODE compliance) 2025-11-14 08:26:45 -08:00
metadataIndexCache.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
metadataIndexChunking.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
metadataNamespace.ts feat: comprehensive metadata namespace architecture and cleanup system 2025-08-27 15:38:48 -07:00
mutex.ts feat: complete VFS with Knowledge Layer integration 2025-09-25 10:47:44 -07:00
nodeVersionCheck.ts fix: resolve browser compatibility by avoiding Node.js process access in nodeVersionCheck 2025-09-17 16:59:59 -07:00
operationUtils.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
paramValidation.ts feat: COW always-on architecture + cloud storage clear() fix (v5.11.0) 2025-11-18 13:44:02 -08:00
performanceMonitor.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
periodicCleanup.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
rateLimiter.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
rebuildCounts.ts fix: resolve critical 378x pagination infinite loop bug (v5.7.11) 2025-11-13 14:20:19 -08:00
requestCoalescer.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
requestDeduplicator.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
searchCache.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
statisticsCollector.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
structuredLogger.ts feat: add browser environment compatibility support 2025-09-17 15:48:02 -07:00
textEncoding.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
typeUtils.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
typeValidation.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
unifiedCache.ts perf: more aggressive cache fairness to prevent thrashing 2025-10-13 11:50:53 -07:00
version.ts fix: update hardcoded version references to use dynamic version 2025-09-26 13:41:16 -07:00
workerUtils.ts fix: correct Node.js version references from 24 to 22 in comments and code 2025-10-20 11:43:31 -07:00
writeBuffer.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00