brainy/docs/architecture
David Snelling 1966c39f24 fix: move metadata routing to base class, fix GCS/S3 system key crashes
Critical fix for GCS/S3 native storage adapters crashing on metadata index keys.

Problem:
- GCS and S3 adapters crashed with "Invalid UUID format" errors
- System keys like __metadata_field_index__status are NOT UUIDs
- Adapters incorrectly tried to shard all metadata keys as UUIDs

Solution:
- Move sharding/routing logic from adapters to BaseStorage class
- Add analyzeKey() method to detect system keys vs entity UUIDs
- System keys route to _system/ directory (no sharding)
- Entity UUIDs route to sharded directories (256 shards)
- All adapters now implement 4 primitive operations:
  * writeObjectToPath(path, data)
  * readObjectFromPath(path)
  * deleteObjectFromPath(path)
  * listObjectsUnderPath(prefix)

Benefits:
- Impossible for future adapters to repeat this mistake
- Zero breaking changes, full backward compatibility
- No data migration required
- Cleaner architecture with better separation of concerns

Updated adapters: GcsStorage, S3CompatibleStorage, OPFSStorage,
FileSystemStorage, MemoryStorage

Added: docs/architecture/data-storage-architecture.md
Updated: README.md with architecture docs link
2025-10-09 13:10:06 -07:00
..
API_SURFACE_DESIGN.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00
augmentation-system-audit.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00
augmentations-actual.md feat: remove legacy ImportManager, standardize getStats() API 2025-10-09 11:40:31 -07:00
augmentations.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00
CLUSTERING_ALGORITHMS_ANALYSIS.md docs: add deprecation warnings for addNoun and addVerb methods 2025-09-17 10:48:41 -07:00
data-storage-architecture.md fix: move metadata routing to base class, fix GCS/S3 system key crashes 2025-10-09 13:10:06 -07:00
distributed-storage.md docs: add comprehensive scaling and storage architecture documentation 2025-09-08 14:49:25 -07:00
METADATA_ARCHITECTURE.md feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
noun-verb-taxonomy.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00
overview.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00
PERFORMANCE_ANALYSIS.md feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
storage-architecture.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00
triple-intelligence.md 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
zero-config.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00