David Snelling
1966c39f24
Critical fix for GCS/S3 native storage adapters crashing on metadata index keys. Problem: - GCS and S3 adapters crashed with "Invalid UUID format" errors - System keys like __metadata_field_index__status are NOT UUIDs - Adapters incorrectly tried to shard all metadata keys as UUIDs Solution: - Move sharding/routing logic from adapters to BaseStorage class - Add analyzeKey() method to detect system keys vs entity UUIDs - System keys route to _system/ directory (no sharding) - Entity UUIDs route to sharded directories (256 shards) - All adapters now implement 4 primitive operations: * writeObjectToPath(path, data) * readObjectFromPath(path) * deleteObjectFromPath(path) * listObjectsUnderPath(prefix) Benefits: - Impossible for future adapters to repeat this mistake - Zero breaking changes, full backward compatibility - No data migration required - Cleaner architecture with better separation of concerns Updated adapters: GcsStorage, S3CompatibleStorage, OPFSStorage, FileSystemStorage, MemoryStorage Added: docs/architecture/data-storage-architecture.md Updated: README.md with architecture docs link |
||
|---|---|---|
| .. | ||
| api | ||
| augmentations | ||
| cli | ||
| config | ||
| cortex | ||
| critical | ||
| distributed | ||
| embeddings | ||
| errors | ||
| graph | ||
| hnsw | ||
| import | ||
| importers | ||
| mcp | ||
| neural | ||
| patterns | ||
| scripts | ||
| shared | ||
| storage | ||
| streaming | ||
| triple | ||
| types | ||
| universal | ||
| utils | ||
| vfs | ||
| augmentationManager.ts | ||
| augmentationPipeline.ts | ||
| brainy.ts | ||
| coreTypes.ts | ||
| cortex.ts | ||
| index.ts | ||
| pipeline.ts | ||
| setup.ts | ||
| unified.ts | ||
| worker.ts | ||