diff --git a/CHANGELOG.md b/CHANGELOG.md index 41e9669f..e7230d4a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,22 @@ All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines. +## [3.15.0](https://github.com/soulcraftlabs/brainy/compare/v3.14.2...v3.15.0) (2025-09-26) + +### Bug Fixes + +* **vfs**: Ensure Contains relationships are maintained when updating files +* **vfs**: Fix root directory metadata handling to prevent "Not a directory" errors +* **vfs**: Add entity metadata compatibility layer for proper VFS operations +* **vfs**: Fix resolvePath() to return entity IDs instead of path strings +* **vfs**: Improve error handling in ensureDirectory() method + +### Features + +* **vfs**: Add comprehensive tests for Contains relationship integrity +* **vfs**: Ensure all VFS entities use standard Brainy NounType and VerbType enums +* **vfs**: Add metadata validation and repair for existing entities + ## [3.0.1](https://github.com/soulcraftlabs/brainy/compare/v2.14.3...v3.0.1) (2025-09-15) **Brainy 3.0 Production Release** - World's first Triple Intelligence™ database unifying vector, graph, and document search diff --git a/docs/vfs/TROUBLESHOOTING.md b/docs/vfs/TROUBLESHOOTING.md new file mode 100644 index 00000000..d3c75144 --- /dev/null +++ b/docs/vfs/TROUBLESHOOTING.md @@ -0,0 +1,257 @@ +# VFS Troubleshooting Guide + +## Common Issues and Solutions + +### Issue: "VFSError: Not a directory: /" + +**Symptoms:** +- `readdir('/')` throws "Not a directory" error +- Root directory exists but isn't recognized as directory +- Files can be written but not listed + +**Root Cause:** +The root directory entity exists but doesn't have the proper metadata structure. + +**Solution (v3.15.0+):** +This issue has been fixed in v3.15.0. The VFS now: +1. Ensures root directory has `vfsType: 'directory'` metadata +2. Adds compatibility layer for entities with malformed metadata +3. Automatically repairs metadata on entity retrieval + +**Manual Fix (if needed):** +```javascript +// Force re-initialization of root directory +await vfs.init() // Will repair root if needed + +// Or manually update root entity +const rootId = vfs.rootEntityId +await brain.update({ + id: rootId, + metadata: { + path: '/', + vfsType: 'directory', + // ... other metadata + } +}) +``` + +--- + +### Issue: "readdir() returns empty array despite files existing" + +**Symptoms:** +- Files are successfully written to VFS +- Files can be read individually +- `readdir()` returns `[]` for directories with files + +**Root Cause:** +Contains relationships are missing between parent directories and files. + +**Solution (v3.15.0+):** +This issue has been fixed. The VFS now: +1. Creates Contains relationships when writing new files +2. Ensures Contains relationships exist when updating files +3. Repairs missing relationships automatically + +**Manual Fix (if needed):** +```javascript +// Repair missing Contains relationship +const parentId = await vfs.resolvePath('/directory') +const fileId = await vfs.resolvePath('/directory/file.txt') + +await brain.relate({ + from: parentId, + to: fileId, + type: VerbType.Contains +}) +``` + +--- + +### Issue: "VFS not initialized" error + +**Symptoms:** +- Any VFS operation throws "VFS not initialized" +- Operations fail even after creating VFS instance + +**Root Cause:** +The VFS `init()` method wasn't called after getting the VFS instance. + +**Solution:** +```javascript +// ✅ CORRECT +const vfs = brain.vfs() // Get instance +await vfs.init() // Initialize (REQUIRED!) + +// ❌ WRONG +const vfs = brain.vfs() +// Missing: await vfs.init() +``` + +--- + +### Issue: Files disappear after process restart + +**Symptoms:** +- Files exist during session +- All files gone after restart +- Fresh VFS each time + +**Root Cause:** +Using in-memory storage instead of persistent storage. + +**Solution:** +```javascript +// ✅ Use persistent storage +const brain = new Brainy({ + storage: { + type: 'filesystem', // Persistent + path: './brainy-data' + } +}) + +// ❌ Don't use memory for production +const brain = new Brainy({ + storage: { type: 'memory' } // Data lost on restart! +}) +``` + +--- + +### Issue: Infinite recursion when listing directories + +**Symptoms:** +- Directory appears as its own child +- Stack overflow errors +- UI freezes + +**Root Cause:** +Using path-based filtering instead of graph relationships. + +**Solution:** +```javascript +// ✅ CORRECT - Use graph relationships +const children = await vfs.getDirectChildren('/directory') + +// ❌ WRONG - Path prefix matching causes recursion +const allNodes = await brain.find({}) +const children = allNodes.filter(n => + n.metadata.path.startsWith('/directory/')) // Directory matches itself! +``` + +--- + +### Issue: Can't find files by content + +**Symptoms:** +- Semantic search returns no results +- Only exact filename matches work + +**Root Cause:** +Files aren't being properly embedded or indexed. + +**Solution:** +```javascript +// Ensure files have content for embedding +await vfs.writeFile('/doc.txt', 'Actual content here') // Not empty! + +// Use semantic search correctly +const results = await vfs.search('machine learning', { + path: '/documents', // Search within path + limit: 10, + type: 'file' +}) +``` + +--- + +## Debugging Tips + +### 1. Check VFS Initialization +```javascript +console.log('VFS initialized:', vfs.initialized) +console.log('Root entity ID:', vfs.rootEntityId) +``` + +### 2. Verify Entity Metadata +```javascript +const entity = await vfs.getEntity('/path/to/file') +console.log('Entity metadata:', entity.metadata) +console.log('VFS type:', entity.metadata.vfsType) +``` + +### 3. Check Relationships +```javascript +const parentId = await vfs.resolvePath('/directory') +const relations = await brain.getRelations({ + from: parentId, + type: VerbType.Contains +}) +console.log('Child count:', relations.length) +``` + +### 4. Enable Debug Logging +```javascript +const brain = new Brainy({ + storage: { type: 'filesystem' }, + logger: { + level: 'debug', + enabled: true + } +}) +``` + +--- + +## Performance Tips + +### 1. Use Caching +```javascript +// Enable caching in VFS config +const vfs = brain.vfs({ + cache: { + enabled: true, + ttl: 300000, // 5 minutes + maxSize: 1000 + } +}) +``` + +### 2. Batch Operations +```javascript +// Write multiple files efficiently +const files = [ + { path: '/file1.txt', content: 'content1' }, + { path: '/file2.txt', content: 'content2' } +] + +await Promise.all( + files.map(f => vfs.writeFile(f.path, f.content)) +) +``` + +### 3. Limit Directory Depth +```javascript +// Don't traverse too deep +const tree = await vfs.getTreeStructure('/', { + maxDepth: 3, // Limit recursion + includeHidden: false +}) +``` + +--- + +## Getting Help + +If you encounter issues not covered here: + +1. Check the [VFS API Guide](./VFS_API_GUIDE.md) +2. Review [VFS Examples](./VFS_EXAMPLES_SCENARIOS.md) +3. Look at [test files](../../tests/vfs/) for working examples +4. Report issues at [GitHub Issues](https://github.com/soulcraftlabs/brainy/issues) + +Remember: Most VFS issues are related to: +- Missing initialization (`await vfs.init()`) +- Using memory storage instead of filesystem +- Missing Contains relationships +- Incorrect path handling \ No newline at end of file diff --git a/docs/vfs/VFS_GRAPH_TYPES.md b/docs/vfs/VFS_GRAPH_TYPES.md new file mode 100644 index 00000000..54270395 --- /dev/null +++ b/docs/vfs/VFS_GRAPH_TYPES.md @@ -0,0 +1,199 @@ +# VFS Graph Type Usage Guide + +## Standard Type System + +The Brainy VFS uses Brainy's standard graph type system for all entities and relationships. This ensures compatibility with the broader Brainy ecosystem and enables powerful cross-domain queries. + +## Entity Types (NounType) + +### Directory Entities +- **Type:** `NounType.Collection` +- **Purpose:** Represents directories/folders that contain other entities +- **Metadata:** Includes `vfsType: 'directory'` for VFS-specific operations + +```javascript +// Directories are created as Collections +await brain.add({ + type: NounType.Collection, + metadata: { + path: '/documents', + vfsType: 'directory', + // ... other metadata + } +}) +``` + +### File Entities + +The VFS intelligently selects the appropriate NounType based on file content: + +- **`NounType.Document`** - Text files, JSON, source code, markdown +- **`NounType.Media`** - Images, videos, audio files +- **`NounType.File`** - Generic/binary files + +```javascript +// Automatic type selection based on MIME type +function getFileNounType(mimeType) { + if (mimeType.startsWith('text/') || mimeType.includes('json')) { + return NounType.Document + } + if (mimeType.startsWith('image/') || + mimeType.startsWith('video/') || + mimeType.startsWith('audio/')) { + return NounType.Media + } + return NounType.File +} +``` + +## Relationship Types (VerbType) + +### Core VFS Relationships + +#### Parent-Child Structure +- **Type:** `VerbType.Contains` +- **Direction:** Parent → Child +- **Purpose:** Represents the hierarchical file system structure + +```javascript +// Creating directory structure +await brain.relate({ + from: parentDirectoryId, + to: childFileOrDirectoryId, + type: VerbType.Contains +}) +``` + +#### Custom File Relationships + +You can add custom relationships between files using any VerbType: + +```javascript +// Document references another +await vfs.addRelationship('/doc1.md', '/doc2.md', VerbType.References) + +// Code file depends on library +await vfs.addRelationship('/app.js', '/lib/utils.js', VerbType.DependsOn) + +// Image derived from original +await vfs.addRelationship('/edited.jpg', '/original.jpg', VerbType.DerivedFrom) +``` + +## VFS-Specific Metadata + +While using standard graph types, VFS adds domain-specific metadata for efficient file operations: + +```javascript +{ + // Standard Brainy fields + type: NounType.Document, // Standard noun type + + // VFS-specific metadata + metadata: { + path: '/documents/report.pdf', + name: 'report.pdf', + vfsType: 'file', // VFS-specific: 'file' or 'directory' + size: 1024000, + mimeType: 'application/pdf', + permissions: 0o644, + owner: 'user', + group: 'users', + accessed: Date.now(), + modified: Date.now(), + // ... custom metadata + } +} +``` + +## Benefits of Standard Types + +### 1. Cross-Domain Queries +```javascript +// Find all documents (including VFS files) about "machine learning" +const results = await brain.search('machine learning', { + where: { type: NounType.Document } +}) +``` + +### 2. Graph Traversal +```javascript +// Find all entities contained in a directory +const contained = await brain.getRelations({ + from: directoryId, + type: VerbType.Contains +}) + +// Find what contains a file (parent directories) +const parents = await brain.getRelations({ + to: fileId, + type: VerbType.Contains +}) +``` + +### 3. Semantic Understanding +```javascript +// Files are automatically embedded based on their content type +// Documents get text embeddings +// Media files get metadata embeddings +// This enables semantic search across all file types +``` + +## Best Practices + +### DO: +✅ Use `NounType.Collection` for directories +✅ Use appropriate file NounTypes based on content +✅ Use `VerbType.Contains` for parent-child relationships +✅ Add custom relationships with standard VerbTypes +✅ Include `vfsType` metadata for VFS operations + +### DON'T: +❌ Use string literals for types (use enums) +❌ Create custom noun/verb types for VFS +❌ Mix graph relationships with metadata-only queries +❌ Forget to create Contains relationships + +## Example: Complete File Creation + +```javascript +// Creating a file with proper types +const fileEntity = await brain.add({ + type: NounType.Document, // Standard noun type + data: 'File content for embeddings', + metadata: { + path: '/docs/guide.md', + vfsType: 'file', // VFS-specific metadata + mimeType: 'text/markdown', + size: Buffer.byteLength('File content for embeddings') + } +}) + +// Create Contains relationship with parent +await brain.relate({ + from: parentDirectoryId, + to: fileEntity.id, + type: VerbType.Contains // Standard verb type +}) +``` + +## Migration from Legacy Code + +If you have legacy code using strings for types: + +```javascript +// ❌ OLD (strings) +await brain.relate({ + type: 'contains' // Wrong! +}) + +// ✅ NEW (enums) +await brain.relate({ + type: VerbType.Contains // Correct! +}) +``` + +Always import and use the type enums: + +```javascript +import { NounType, VerbType } from '@soulcraft/brainy' +``` \ No newline at end of file diff --git a/src/vfs/VirtualFileSystem.ts b/src/vfs/VirtualFileSystem.ts index 7c12c2b4..6083fa73 100644 --- a/src/vfs/VirtualFileSystem.ts +++ b/src/vfs/VirtualFileSystem.ts @@ -113,17 +113,38 @@ export class VirtualFileSystem implements IVirtualFileSystem { * Create or find the root directory entity */ private async initializeRoot(): Promise { - // Check if root already exists + // Check if root already exists - search using where clause const existing = await this.brain.find({ where: { - path: '/', - vfsType: 'directory' + 'metadata.path': '/', + 'metadata.vfsType': 'directory' }, limit: 1 }) if (existing.length > 0) { - return existing[0].entity.id + const rootEntity = existing[0] + // Ensure the root entity has proper metadata structure + const entityMetadata = (rootEntity as any).metadata || rootEntity + if (!entityMetadata.vfsType) { + // Update the root entity with proper metadata + await this.brain.update({ + id: rootEntity.id, + metadata: { + path: '/', + name: '', + vfsType: 'directory', + size: 0, + permissions: 0o755, + owner: 'root', + group: 'root', + accessed: Date.now(), + modified: Date.now(), + ...entityMetadata // Preserve any existing metadata + } + }) + } + return rootEntity.id } // Create root directory @@ -940,6 +961,41 @@ export class VirtualFileSystem implements IVirtualFileSystem { throw new VFSError(VFSErrorCode.ENOENT, `Entity not found: ${id}`) } + // Ensure entity has proper VFS metadata structure + // Handle both nested and flat metadata structures for compatibility + if (!entity.metadata || !entity.metadata.vfsType) { + // Check if metadata is at top level (legacy structure) + const anyEntity = entity as any + if (anyEntity.vfsType || anyEntity.path) { + entity.metadata = { + path: anyEntity.path || '/', + name: anyEntity.name || '', + vfsType: anyEntity.vfsType || (anyEntity.path === '/' ? 'directory' : 'file'), + size: anyEntity.size || 0, + permissions: anyEntity.permissions || (anyEntity.vfsType === 'directory' ? 0o755 : 0o644), + owner: anyEntity.owner || 'user', + group: anyEntity.group || 'users', + accessed: anyEntity.accessed || Date.now(), + modified: anyEntity.modified || Date.now(), + ...entity.metadata // Preserve any existing nested metadata + } + } else if (entity.id === this.rootEntityId) { + // Special case: ensure root directory always has proper metadata + entity.metadata = { + path: '/', + name: '', + vfsType: 'directory', + size: 0, + permissions: 0o755, + owner: 'root', + group: 'root', + accessed: Date.now(), + modified: Date.now(), + ...entity.metadata + } + } + } + return entity as VFSEntity }