docs: fix VFS documentation NO FAKE CODE violations
Removed 9 undocumented feature sections from VFS docs (version history, distributed filesystem, AI auto-organization, etc.). Added status labels (Production/Beta/Experimental) to all features, updated performance claims with MEASURED vs PROJECTED labels, and created ROADMAP.md for planned features. Fixed storage adapter list to show only built-in adapters. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
db23836b32
commit
3a4e49a564
4 changed files with 380 additions and 233 deletions
|
|
@ -163,28 +163,32 @@ const parisPhotos = await vfs.search('', {
|
|||
})
|
||||
```
|
||||
|
||||
### 🎯 Smart Collections
|
||||
### 🎯 Semantic Path Access
|
||||
|
||||
Virtual directories based on queries:
|
||||
Access files through semantic dimensions (see [Semantic VFS](./SEMANTIC_VFS.md)):
|
||||
|
||||
```javascript
|
||||
// Create a smart folder that auto-updates
|
||||
await vfs.createVirtualDirectory('/smart/recent-docs', {
|
||||
query: 'type:document modified:last-7-days'
|
||||
// Query-based path access (current functionality)
|
||||
const authFiles = await vfs.search('', {
|
||||
where: { concepts: { contains: 'authentication' }}
|
||||
})
|
||||
|
||||
// Create a collection based on similarity
|
||||
await vfs.createVirtualDirectory('/smart/like-this', {
|
||||
similar: '/examples/good-code.js',
|
||||
// Find files by custom metadata
|
||||
const recent = await vfs.search('', {
|
||||
where: {
|
||||
type: 'document',
|
||||
modified: { greaterThan: Date.now() - 7*24*60*60*1000 }
|
||||
}
|
||||
})
|
||||
|
||||
// Find similar files
|
||||
const similar = await vfs.findSimilar('/examples/good-code.js', {
|
||||
threshold: 0.7
|
||||
})
|
||||
|
||||
// Tag-based collections
|
||||
await vfs.createVirtualDirectory('/smart/important', {
|
||||
where: { tags: 'important' }
|
||||
})
|
||||
```
|
||||
|
||||
> **Note:** Virtual directories (persistent query-based folders) are planned for v2.0. See [ROADMAP](./ROADMAP.md).
|
||||
|
||||
## Real-World Examples
|
||||
|
||||
### 📚 Knowledge Management
|
||||
|
|
@ -314,150 +318,28 @@ const urgent = await vfs.search('', {
|
|||
|
||||
## Advanced Features
|
||||
|
||||
### 🔄 Version History
|
||||
> **Note:** See [VFS ROADMAP](./ROADMAP.md) for planned advanced features like version history, distributed filesystem, and more.
|
||||
|
||||
```javascript
|
||||
// Enable versioning for a file
|
||||
await vfs.enableVersioning('/important/contract.pdf')
|
||||
## Integration Possibilities
|
||||
|
||||
// Write updates - automatically creates versions
|
||||
await vfs.writeFile('/important/contract.pdf', newVersion)
|
||||
VFS can be integrated with existing applications. See [VFS ROADMAP](./ROADMAP.md) for planned integrations like Express.js middleware, VSCode extensions, and more.
|
||||
|
||||
// Get version history
|
||||
const versions = await vfs.getVersions('/important/contract.pdf')
|
||||
// Returns: [{version: 1, date: ..., size: ...}, {version: 2, ...}]
|
||||
**Current approach:** Use VFS directly via API for custom integrations.
|
||||
|
||||
// Restore a previous version
|
||||
await vfs.restoreVersion('/important/contract.pdf', 1)
|
||||
|
||||
// Compare versions
|
||||
const diff = await vfs.diffVersions('/important/contract.pdf', 1, 2)
|
||||
```
|
||||
|
||||
### 🌐 Distributed Filesystem
|
||||
|
||||
```javascript
|
||||
// Mount remote Brainy instances
|
||||
await vfs.mount('/remote/server2', {
|
||||
host: 'brainy.server2.com',
|
||||
credentials: { ... }
|
||||
})
|
||||
|
||||
// Federated search across all mounted systems
|
||||
const results = await vfs.search('project documentation', {
|
||||
distributed: true
|
||||
})
|
||||
|
||||
// Sync directories across instances
|
||||
await vfs.sync('/projects', '/remote/server2/backup/projects')
|
||||
```
|
||||
|
||||
### 🤖 AI-Powered Automation
|
||||
|
||||
```javascript
|
||||
// Auto-organize downloads folder
|
||||
await vfs.autoOrganize('/downloads', {
|
||||
rules: [
|
||||
{ pattern: '*.pdf', destination: '/documents' },
|
||||
{ pattern: '*.{jpg,png}', destination: '/images' },
|
||||
{ semantic: 'code files', destination: '/code' }
|
||||
]
|
||||
})
|
||||
|
||||
// Smart deduplication
|
||||
const duplicates = await vfs.detectDuplicates('/photos')
|
||||
await vfs.deduplicateFiles(duplicates, {
|
||||
strategy: 'keep-highest-quality'
|
||||
})
|
||||
|
||||
// Content-aware compression
|
||||
await vfs.optimizeStorage('/archives', {
|
||||
compress: true,
|
||||
deduplicate: true,
|
||||
indexContent: true
|
||||
})
|
||||
```
|
||||
|
||||
### 🔐 Security & Permissions
|
||||
|
||||
```javascript
|
||||
// Set access control
|
||||
await vfs.setACL('/private', {
|
||||
owner: 'user123',
|
||||
permissions: {
|
||||
owner: 'rwx',
|
||||
group: 'r-x',
|
||||
others: '---'
|
||||
}
|
||||
})
|
||||
|
||||
// Encryption at rest
|
||||
await vfs.encrypt('/sensitive', {
|
||||
algorithm: 'AES-256',
|
||||
key: encryptionKey
|
||||
})
|
||||
|
||||
// Audit trail
|
||||
const audit = await vfs.getAuditLog('/financial/reports')
|
||||
// Returns: who accessed what and when
|
||||
```
|
||||
|
||||
## Integration Examples
|
||||
|
||||
### Node.js fs Compatibility
|
||||
|
||||
```javascript
|
||||
// Drop-in replacement for fs module
|
||||
import { promises as fs } from '@soulcraft/brainy/vfs/fs'
|
||||
|
||||
// Works with existing code!
|
||||
const data = await fs.readFile('/config.json', 'utf8')
|
||||
const config = JSON.parse(data)
|
||||
|
||||
await fs.writeFile('/output.txt', 'Hello VFS!')
|
||||
const stats = await fs.stat('/output.txt')
|
||||
```
|
||||
|
||||
### Express.js Static Files
|
||||
|
||||
```javascript
|
||||
import express from 'express'
|
||||
import { createStaticMiddleware } from '@soulcraft/brainy/vfs/express'
|
||||
|
||||
const app = express()
|
||||
|
||||
// Serve files from VFS
|
||||
app.use('/static', createStaticMiddleware('/public', {
|
||||
intelligentCaching: true, // Cache based on access patterns
|
||||
autoCompress: true // Compress on the fly
|
||||
}))
|
||||
```
|
||||
|
||||
### VSCode Extension
|
||||
|
||||
```javascript
|
||||
// Open VFS in VSCode
|
||||
import { workspace } from 'vscode'
|
||||
import { VFSProvider } from '@soulcraft/brainy/vfs/vscode'
|
||||
|
||||
// Register VFS as a filesystem provider
|
||||
workspace.registerFileSystemProvider('brainy', new VFSProvider(), {
|
||||
isCaseSensitive: true,
|
||||
isReadonly: false
|
||||
})
|
||||
|
||||
// Now you can open: brainy:///projects/my-app
|
||||
```
|
||||
|
||||
## Performance
|
||||
## Performance Characteristics
|
||||
|
||||
Brainy VFS is designed for speed and scale:
|
||||
|
||||
- **Sub-10ms latency** for basic operations
|
||||
- **Intelligent caching** reduces repeated reads to <1ms
|
||||
- **Vector search** returns results in <100ms for millions of files
|
||||
- **Streaming support** for files of any size
|
||||
- **Distributed sharding** for billions of files
|
||||
**Tested at 1K-10K file scale:**
|
||||
- **Sub-10ms latency** for basic operations (measured)
|
||||
- **Intelligent caching** reduces repeated reads to <5ms (measured)
|
||||
|
||||
**PROJECTED at larger scales (not yet tested):**
|
||||
- **Vector search** <100ms for millions of files (projected)
|
||||
- **Streaming support** for files of any size (architecture supports, see [limitations in ROADMAP](./ROADMAP.md))
|
||||
- **Distributed sharding** for billions of files (architecture supports, not tested at scale)
|
||||
|
||||
See tests in `tests/vfs/` for actual measured performance.
|
||||
|
||||
## Triple Intelligence Power 🧠⚡
|
||||
|
||||
|
|
@ -736,54 +618,11 @@ const assigned = await vfs.search('', {
|
|||
})
|
||||
```
|
||||
|
||||
### Monitoring & Operations
|
||||
### Monitoring & Operations (Planned)
|
||||
|
||||
#### **Production Metrics**
|
||||
> **Note:** Production monitoring features are planned for v1.1. See [ROADMAP](./ROADMAP.md).
|
||||
|
||||
```javascript
|
||||
// Get VFS statistics
|
||||
const stats = vfs.getStatistics()
|
||||
console.log(stats)
|
||||
// {
|
||||
// totalFiles: 1234567,
|
||||
// totalDirectories: 45678,
|
||||
// totalSize: 123456789000,
|
||||
// cacheHitRate: 0.95,
|
||||
// avgResponseTime: 12,
|
||||
// activeConnections: 234
|
||||
// }
|
||||
|
||||
// Monitor hot paths
|
||||
const hotPaths = vfs.getHotPaths()
|
||||
// Paths accessed >100 times/minute
|
||||
|
||||
// Check health
|
||||
const health = await vfs.healthCheck()
|
||||
// {
|
||||
// status: 'healthy',
|
||||
// latency: { p50: 10, p99: 100 },
|
||||
// errors: { rate: 0.001 }
|
||||
// }
|
||||
```
|
||||
|
||||
#### **Backup & Recovery**
|
||||
|
||||
```javascript
|
||||
// Incremental backup
|
||||
const changes = await vfs.getChangesSince(lastBackupTime)
|
||||
for (const change of changes) {
|
||||
await backupSystem.store(change)
|
||||
}
|
||||
|
||||
// Point-in-time recovery
|
||||
await vfs.restoreToTime(timestamp)
|
||||
|
||||
// Verify integrity
|
||||
const corrupted = await vfs.verifyIntegrity()
|
||||
if (corrupted.length > 0) {
|
||||
await vfs.repair(corrupted)
|
||||
}
|
||||
```
|
||||
> **Note:** Backup & Recovery features are planned for v1.2. See [ROADMAP](./ROADMAP.md).
|
||||
|
||||
### Deployment Options
|
||||
|
||||
|
|
@ -827,12 +666,15 @@ await vfs.init({
|
|||
})
|
||||
```
|
||||
|
||||
## Coming Soon
|
||||
## Roadmap & Future Features
|
||||
|
||||
- **Version 1.1**: Full streaming support, FUSE driver
|
||||
- **Version 1.2**: Distributed transactions, global replication
|
||||
- **Version 1.3**: Time-travel queries, branching
|
||||
- **Version 2.0**: Quantum-resistant encryption, neural interfaces
|
||||
See [VFS ROADMAP](./ROADMAP.md) for planned features including:
|
||||
- Enhanced streaming support (v1.1)
|
||||
- Version history (v1.2)
|
||||
- Distributed filesystem (v1.2)
|
||||
- AI-powered automation (v2.0)
|
||||
- FUSE driver (v2.0 research)
|
||||
- And more community-requested features
|
||||
|
||||
## Contributing
|
||||
|
||||
|
|
|
|||
257
docs/vfs/ROADMAP.md
Normal file
257
docs/vfs/ROADMAP.md
Normal file
|
|
@ -0,0 +1,257 @@
|
|||
# VFS Roadmap - Planned Features
|
||||
|
||||
**Status:** These features are planned but not yet implemented.
|
||||
**Current Version:** See main [VFS README](./README.md) for implemented features.
|
||||
|
||||
---
|
||||
|
||||
## v1.1 (Next Release)
|
||||
|
||||
### Enhanced Streaming Support
|
||||
Improve VFS streaming to support true chunk-by-chunk reads for large files without loading entire content into memory.
|
||||
|
||||
**Status:** Planned
|
||||
**Effort:** 3-4 weeks
|
||||
|
||||
### VFS Security Integration
|
||||
Integrate Brainy's SecurityAPI with VFS file operations.
|
||||
|
||||
```typescript
|
||||
// Planned API (not yet implemented)
|
||||
await vfs.encrypt('/sensitive-data.json', { algorithm: 'AES-256' })
|
||||
await vfs.setACL('/project', { user: 'alice', permissions: 'rw-' })
|
||||
const auditLog = await vfs.getAuditLog('/project', { since: lastWeek })
|
||||
```
|
||||
|
||||
**Status:** Planned
|
||||
**Note:** SecurityAPI exists (`src/api/SecurityAPI.ts`) but not integrated with VFS operations.
|
||||
|
||||
### Atomic Operations Layer
|
||||
Add transaction support for compound VFS operations.
|
||||
|
||||
```typescript
|
||||
// Planned API (not yet implemented)
|
||||
await vfs.transaction(async (tx) => {
|
||||
await tx.move('/src/file.txt', '/dest/file.txt')
|
||||
await tx.writeFile('/dest/metadata.json', metadata)
|
||||
// Both succeed or both fail
|
||||
})
|
||||
```
|
||||
|
||||
**Status:** Planned
|
||||
**Effort:** 1-2 weeks
|
||||
|
||||
---
|
||||
|
||||
## v1.2
|
||||
|
||||
### Version History
|
||||
Track file versions with history and restoration capabilities.
|
||||
|
||||
```typescript
|
||||
// Planned API (not yet implemented)
|
||||
await vfs.enableVersioning('/important-doc.md')
|
||||
const versions = await vfs.getVersions('/important-doc.md')
|
||||
await vfs.restoreVersion('/important-doc.md', versions[2].id)
|
||||
const diff = await vfs.diffVersions('/important-doc.md', v1, v2)
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Automatic versioning on file changes
|
||||
- Version history with metadata
|
||||
- Point-in-time restoration
|
||||
- Version comparison/diff
|
||||
|
||||
**Status:** Planned
|
||||
**Effort:** 4-5 weeks
|
||||
|
||||
### Distributed Filesystem
|
||||
Mount remote Brainy instances for federated file access.
|
||||
|
||||
```typescript
|
||||
// Planned API (not yet implemented)
|
||||
await vfs.mount('/remote-team', {
|
||||
type: 'brainy-remote',
|
||||
url: 'https://team-brainy.example.com',
|
||||
credentials: {...}
|
||||
})
|
||||
|
||||
// Federated search across mounted instances
|
||||
const results = await vfs.search('project docs', { includeMounted: true })
|
||||
|
||||
// Sync directories
|
||||
await vfs.sync('/local/docs', '/remote-team/docs')
|
||||
```
|
||||
|
||||
**Status:** Planned
|
||||
**Effort:** 6-8 weeks
|
||||
|
||||
### Backup & Recovery API
|
||||
Built-in backup and recovery operations.
|
||||
|
||||
```typescript
|
||||
// Planned API (not yet implemented)
|
||||
const changes = await vfs.getChangesSince(lastBackupTime)
|
||||
await vfs.createSnapshot('/backup-snapshot-2024')
|
||||
await vfs.restoreToTime('/project', timestamp)
|
||||
await vfs.verifyIntegrity('/project')
|
||||
const issues = await vfs.repair('/project')
|
||||
```
|
||||
|
||||
**Status:** Planned
|
||||
**Effort:** 3-4 weeks
|
||||
|
||||
---
|
||||
|
||||
## v2.0
|
||||
|
||||
### AI-Powered Auto-Organization
|
||||
Intelligent file organization based on content and usage patterns.
|
||||
|
||||
```typescript
|
||||
// Planned API (not yet implemented)
|
||||
await vfs.autoOrganize('/downloads', {
|
||||
strategy: 'by-content-type',
|
||||
createFolders: true
|
||||
})
|
||||
|
||||
const duplicates = await vfs.detectDuplicates('/photos')
|
||||
await vfs.deduplicateFiles(duplicates, { keep: 'highest-quality' })
|
||||
|
||||
await vfs.optimizeStorage('/project', {
|
||||
compress: ['*.log', '*.txt'],
|
||||
archive: { olderThan: '90d' }
|
||||
})
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Content-based organization
|
||||
- Smart deduplication
|
||||
- Content-aware compression
|
||||
- Automatic archival
|
||||
|
||||
**Status:** Planned - Research phase
|
||||
**Effort:** 8-10 weeks
|
||||
|
||||
### Smart Collections / Virtual Directories
|
||||
Persistent query-based virtual directories that auto-update.
|
||||
|
||||
```typescript
|
||||
// Planned API (not yet implemented)
|
||||
await vfs.createVirtualDirectory('/auth-related', {
|
||||
query: { concepts: { contains: 'authentication' }},
|
||||
autoUpdate: true
|
||||
})
|
||||
|
||||
// Directory automatically updates as matching files are added/modified
|
||||
```
|
||||
|
||||
**Note:** Current VFS supports query-based *access* (e.g., `/by-concept/auth`) but not persistent virtual directories.
|
||||
|
||||
**Status:** Planned
|
||||
**Effort:** 4-5 weeks
|
||||
|
||||
### FUSE Driver
|
||||
Mount VFS as a native filesystem on Linux/Mac/Windows.
|
||||
|
||||
```typescript
|
||||
// Planned (research phase)
|
||||
import { mountVFS } from '@soulcraft/brainy/vfs/fuse'
|
||||
|
||||
await mountVFS(vfs, {
|
||||
mountPoint: '/mnt/brainy',
|
||||
options: { allowOther: true }
|
||||
})
|
||||
```
|
||||
|
||||
**Challenges:**
|
||||
- FUSE requires synchronous operations (VFS is async-first)
|
||||
- Kernel-level integration complexity
|
||||
- Cross-platform support (FUSE/Dokan/WinFsp)
|
||||
|
||||
**Status:** Research phase
|
||||
**Effort:** 10-12 weeks + significant testing
|
||||
|
||||
---
|
||||
|
||||
## Community Contributions Wanted
|
||||
|
||||
These features would benefit from community contributions. If you're interested in building any of these, please open an issue!
|
||||
|
||||
### Express.js Static Middleware
|
||||
```typescript
|
||||
// Wanted: Community contribution
|
||||
import { createStaticMiddleware } from '@soulcraft/brainy/vfs/express'
|
||||
|
||||
app.use('/files', createStaticMiddleware(vfs, {
|
||||
index: ['index.html', 'index.md'],
|
||||
etag: true,
|
||||
cacheControl: 'max-age=3600'
|
||||
}))
|
||||
```
|
||||
|
||||
### VSCode Extension
|
||||
```typescript
|
||||
// Wanted: Community contribution
|
||||
import { VFSProvider } from '@soulcraft/brainy/vfs/vscode'
|
||||
|
||||
const provider = new VFSProvider(vfs)
|
||||
vscode.workspace.registerFileSystemProvider('brainy', provider)
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Browse VFS in VSCode explorer
|
||||
- Semantic search from command palette
|
||||
- Concept highlighting
|
||||
- Relationship visualization
|
||||
|
||||
### Webpack Plugin
|
||||
Semantic-aware webpack builds with dependency graph from VFS relationships.
|
||||
|
||||
### Vite Plugin
|
||||
Vite integration with VFS for semantic module resolution.
|
||||
|
||||
---
|
||||
|
||||
## Far Future (v5.0+)
|
||||
|
||||
### Automatic Node Discovery
|
||||
Zero-config multi-node setup with automatic discovery via UDP broadcast, Kubernetes DNS, or cloud provider APIs.
|
||||
|
||||
### Automatic Failover
|
||||
Health monitoring and automatic failover in distributed deployments.
|
||||
|
||||
### Cloud Provider Auto-Detection
|
||||
```typescript
|
||||
// Far future concept
|
||||
const brain = new Brainy({
|
||||
storage: 'cloud://brainy-data' // Auto-detects AWS/GCP/Azure
|
||||
})
|
||||
```
|
||||
|
||||
### Hot/Cold Storage Tiering
|
||||
Automatic data movement between hot (SSD/local) and cold (S3/archive) storage based on access patterns.
|
||||
|
||||
---
|
||||
|
||||
## How to Track Progress
|
||||
|
||||
- **GitHub Issues:** Track feature development
|
||||
- **Discussions:** Discuss feature design and requirements
|
||||
- **Pull Requests:** Submit contributions
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
Interested in implementing a planned feature? Here's how to get started:
|
||||
|
||||
1. **Open an issue** discussing the feature you want to implement
|
||||
2. **Review the design** - we'll help with architecture decisions
|
||||
3. **Submit a PR** with implementation + tests
|
||||
4. **Celebrate!** Your contribution helps everyone 🎉
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2025-10-29
|
||||
**VFS Version:** 4.9.0
|
||||
|
|
@ -81,13 +81,15 @@ const experiments = await vfs.readdir('/by-tag/experimental')
|
|||
|
||||
## Supported Semantic Dimensions
|
||||
|
||||
### 1. Traditional Path (Hierarchical)
|
||||
### 1. Traditional Path (Hierarchical) ✅ **Production**
|
||||
```typescript
|
||||
await vfs.readFile('/src/auth/login.ts')
|
||||
// Works exactly like a normal filesystem
|
||||
```
|
||||
|
||||
### 2. By Concept (Semantic)
|
||||
**Status:** ✅ Fully implemented and tested
|
||||
|
||||
### 2. By Concept (Semantic) ⚠️ **Beta**
|
||||
```typescript
|
||||
await vfs.readdir('/by-concept/authentication')
|
||||
// Returns all files about authentication
|
||||
|
|
@ -98,7 +100,9 @@ await vfs.readFile('/by-concept/authentication/login.ts')
|
|||
|
||||
**How it works:** Uses `brain.extractConcepts()` with NeuralEntityExtractor to extract concepts from file content using embeddings and the NounType taxonomy. Indexes concept names for O(log n) queries. See [Neural Extraction API](./NEURAL_EXTRACTION.md) for details.
|
||||
|
||||
### 3. By Author (Ownership)
|
||||
**Status:** ⚠️ Beta - Requires NeuralEntityExtractor setup, tested at <1K file scale
|
||||
|
||||
### 3. By Author (Ownership) ✅ **Production**
|
||||
```typescript
|
||||
await vfs.readdir('/by-author/alice')
|
||||
// All files owned/modified by alice
|
||||
|
|
@ -109,7 +113,9 @@ await vfs.stat('/by-author/alice/config.ts')
|
|||
|
||||
**How it works:** Tracks owner metadata on every file. Indexed by MetadataIndexManager.
|
||||
|
||||
### 4. By Time (Temporal)
|
||||
**Status:** ✅ Fully implemented and tested at 10K file scale
|
||||
|
||||
### 4. By Time (Temporal) ✅ **Production**
|
||||
```typescript
|
||||
await vfs.readdir('/as-of/2024-03-15')
|
||||
// Files modified on March 15, 2024
|
||||
|
|
@ -120,7 +126,9 @@ await vfs.readFile('/as-of/2024-03-15/src/auth.ts')
|
|||
|
||||
**How it works:** Tracks `modified` timestamp. Uses B-tree range queries (`greaterEqual`/`lessEqual`) for O(log n) performance.
|
||||
|
||||
### 5. By Relationship (Graph)
|
||||
**Status:** ✅ Fully implemented and tested at 10K file scale
|
||||
|
||||
### 5. By Relationship (Graph) ✅ **Production**
|
||||
```typescript
|
||||
await vfs.readdir('/related-to/src/auth.ts/depth-2')
|
||||
// Files within 2 relationship hops
|
||||
|
|
@ -131,7 +139,9 @@ await vfs.readdir('/related-to/src/auth.ts/depth-2/types-contains,references')
|
|||
|
||||
**How it works:** Uses GraphAdjacencyIndex for O(1) graph traversal. Supports depth limits and relationship type filtering.
|
||||
|
||||
### 6. By Similarity (Vector)
|
||||
**Status:** ✅ Fully implemented and tested at 10K node scale
|
||||
|
||||
### 6. By Similarity (Vector) ✅ **Production**
|
||||
```typescript
|
||||
await vfs.readdir('/similar-to/src/auth.ts/threshold-0.8')
|
||||
// Files with 80%+ similarity to auth.ts
|
||||
|
|
@ -142,7 +152,9 @@ await vfs.similar('/src/auth.ts', { threshold: 0.9, limit: 10 })
|
|||
|
||||
**How it works:** Uses HNSW vector index for O(log n) nearest neighbor search. Based on content embeddings.
|
||||
|
||||
### 7. By Tag (Classification)
|
||||
**Status:** ✅ Fully implemented and tested at 100K vector scale
|
||||
|
||||
### 7. By Tag (Classification) ✅ **Production**
|
||||
```typescript
|
||||
await vfs.readdir('/by-tag/security')
|
||||
// All security-tagged files
|
||||
|
|
@ -155,25 +167,27 @@ await vfs.writeFile('/src/admin.ts', code, {
|
|||
|
||||
**How it works:** Stores tags in metadata. Indexed for fast queries.
|
||||
|
||||
**Status:** ✅ Fully implemented and tested at 10K file scale
|
||||
|
||||
---
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
### Scalability to Millions of Files
|
||||
### Tested Performance at Scale
|
||||
|
||||
All semantic paths use **indexed data structures** for optimal performance:
|
||||
|
||||
| Dimension | Data Structure | Time Complexity | Million-Scale Ready |
|
||||
|-----------|---------------|-----------------|---------------------|
|
||||
| Traditional | PathCache + Graph | O(path depth) | ✅ Yes |
|
||||
| Concept | MetadataIndex (B-tree) | O(log n) | ✅ Yes* |
|
||||
| Author | MetadataIndex (B-tree) | O(log n) | ✅ Yes |
|
||||
| Time | MetadataIndex (B-tree) | O(log n) | ✅ Yes |
|
||||
| Relationship | GraphAdjacency | O(depth) | ✅ Yes |
|
||||
| Similarity | HNSW Index | O(log n) | ✅ Yes |
|
||||
| Tag | MetadataIndex (B-tree) | O(log n) | ✅ Yes* |
|
||||
| Dimension | Data Structure | Time Complexity | Tested Scale | Production Ready |
|
||||
|-----------|---------------|-----------------|--------------|------------------|
|
||||
| Traditional | PathCache + Graph | O(path depth) | Up to 10K files | ✅ Yes |
|
||||
| Concept | MetadataIndex (B-tree) | O(log n) | Up to 1K files | ⚠️ Beta |
|
||||
| Author | MetadataIndex (B-tree) | O(log n) | Up to 10K files | ✅ Yes |
|
||||
| Time | MetadataIndex (B-tree) | O(log n) | Up to 10K files | ✅ Yes |
|
||||
| Relationship | GraphAdjacency | O(depth) | Up to 10K nodes | ✅ Yes |
|
||||
| Similarity | HNSW Index | O(log n) | Up to 100K vectors | ✅ Yes |
|
||||
| Tag | MetadataIndex (B-tree) | O(log n) | Up to 10K files | ✅ Yes |
|
||||
|
||||
\* *Requires concept/tag flattening (automatic in future versions)*
|
||||
**Note:** Million-scale performance is PROJECTED based on underlying index complexity. VFS-specific testing conducted at 1K-100K scale. See `tests/vfs/` for measured performance.
|
||||
|
||||
### Cache Strategy
|
||||
|
||||
|
|
@ -301,7 +315,11 @@ console.log(id1 === id2 && id2 === id3) // true
|
|||
|
||||
---
|
||||
|
||||
## Extending with Custom Projections
|
||||
## Extending with Custom Projections 🧪 **Experimental**
|
||||
|
||||
**Status:** 🧪 Experimental - API subject to change
|
||||
|
||||
**Warning:** This API uses internal VFS interfaces that are not yet officially exposed. The registration mechanism will change in a future release to provide a stable public API.
|
||||
|
||||
Create your own semantic dimensions:
|
||||
|
||||
|
|
@ -334,12 +352,12 @@ class PriorityProjection extends BaseProjectionStrategy {
|
|||
}
|
||||
}
|
||||
|
||||
// Register custom projection
|
||||
// Register custom projection (experimental - uses internal API)
|
||||
const brain = new Brainy()
|
||||
await brain.init()
|
||||
const vfs = brain.vfs()
|
||||
|
||||
// Access via VFS internals (will be exposed in future API)
|
||||
// ⚠️ Internal API - will be replaced with public registration method
|
||||
vfs.projectionRegistry.register(new PriorityProjection())
|
||||
|
||||
// Now use it!
|
||||
|
|
@ -348,6 +366,8 @@ const highPriority = await vfs.readdir('/by-priority/high')
|
|||
|
||||
See [PROJECTION_STRATEGY_API.md](./PROJECTION_STRATEGY_API.md) for full guide.
|
||||
|
||||
**Roadmap:** Public projection registration API coming in v1.2 (see [VFS ROADMAP](./ROADMAP.md))
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
|
|
|||
|
|
@ -626,40 +626,68 @@ try {
|
|||
|
||||
### Storage Compatibility
|
||||
|
||||
VFS works with **all** Brainy storage adapters:
|
||||
VFS works with all built-in Brainy storage adapters:
|
||||
|
||||
```typescript
|
||||
// Memory (testing)
|
||||
// Memory (testing/development)
|
||||
const brain = new Brainy({ storage: { type: 'memory' } })
|
||||
|
||||
// Redis (development)
|
||||
// Filesystem (local development)
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'redis',
|
||||
url: 'redis://localhost:6379'
|
||||
type: 'filesystem',
|
||||
path: './brainy-data'
|
||||
}
|
||||
})
|
||||
|
||||
// PostgreSQL (production)
|
||||
// S3-compatible storage (production)
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'postgresql',
|
||||
connectionString: 'postgresql://user:pass@localhost/db'
|
||||
type: 's3',
|
||||
bucket: 'my-bucket',
|
||||
region: 'us-east-1'
|
||||
}
|
||||
})
|
||||
|
||||
// ChromaDB (vector-optimized)
|
||||
// Cloudflare R2 (production)
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'chroma',
|
||||
url: 'http://localhost:8000'
|
||||
type: 'r2',
|
||||
accountId: 'your-account-id',
|
||||
bucket: 'my-bucket'
|
||||
}
|
||||
})
|
||||
|
||||
// Google Cloud Storage (production)
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'gcs',
|
||||
bucket: 'my-bucket',
|
||||
projectId: 'my-project'
|
||||
}
|
||||
})
|
||||
|
||||
// Azure Blob Storage (production)
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'azure',
|
||||
accountName: 'myaccount',
|
||||
containerName: 'my-container'
|
||||
}
|
||||
})
|
||||
|
||||
// OPFS (browser-native persistence)
|
||||
const brain = new Brainy({ storage: { type: 'opfs' } })
|
||||
|
||||
// TypeAware storage (optimized for entity types)
|
||||
const brain = new Brainy({ storage: { type: 'typeaware' } })
|
||||
|
||||
// All work identically with VFS
|
||||
const vfs = new VirtualFileSystem(brain)
|
||||
```
|
||||
|
||||
**Custom Storage Adapters:** Redis, PostgreSQL, and other databases can be added via the [extension system](../api/EXTENSIBILITY.md). See `src/config/extensibleConfig.ts` for examples.
|
||||
|
||||
### Best Practices
|
||||
|
||||
#### File Organization
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue