docs: comprehensive API documentation and examples overhaul
- Add comprehensive JSDoc @example tags to all core methods (add, get, relate, find, similar, embed) - Add @deprecated warnings with migration paths for all v2.x APIs - Create VFS Quick Start Guide addressing brain-cloud integration issues - Create VFS Common Patterns guide preventing infinite recursion mistakes - Create Core API Patterns guide with modern v3.x usage examples - Create Neural API Patterns guide for AI-powered features - Create comprehensive API Decision Tree for choosing right methods - Update README with prominent VFS examples and file explorer patterns - Follow 2025 npm package documentation standards throughout 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
f80ed5e8aa
commit
3c72afd41a
8 changed files with 3530 additions and 34 deletions
71
README.md
71
README.md
|
|
@ -209,54 +209,65 @@ await brain.find("Popular JavaScript libraries similar to Vue")
|
||||||
await brain.find("Documentation about authentication from last month")
|
await brain.find("Documentation about authentication from last month")
|
||||||
```
|
```
|
||||||
|
|
||||||
### 🧠🌐 **Knowledge Graph + Virtual Filesystem - Where Ideas Come Alive**
|
### 🧠🌐 **Virtual Filesystem - Intelligent File Management**
|
||||||
|
|
||||||
**Store ANY knowledge. Connect EVERYTHING. Files are optional.**
|
**Build file explorers, IDEs, and knowledge systems that never crash from infinite recursion.**
|
||||||
|
|
||||||
- **Living Knowledge**: Characters, concepts, and systems that exist independently of files
|
- **Tree-Aware Operations**: Safe directory listing prevents recursive loops
|
||||||
- **Universal Connections**: Link files to entities, entities to concepts, anything to anything
|
- **Semantic Search**: Find files by content, not just filename
|
||||||
- **Semantic Intelligence**: Find knowledge by meaning, not by where it's stored
|
- **Production Storage**: Filesystem and cloud storage for real applications
|
||||||
- **Optional Files**: Use filesystem operations when helpful, pure knowledge when not
|
- **Zero-Config**: Works out of the box with intelligent defaults
|
||||||
- **Perfect Memory**: Every piece of knowledge remembers its entire history
|
|
||||||
- **Knowledge Evolution**: Watch ideas grow and connect across time and projects
|
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import { Brainy } from '@soulcraft/brainy'
|
import { Brainy } from '@soulcraft/brainy'
|
||||||
|
|
||||||
const brain = new Brainy({ storage: { type: 'memory' } })
|
// ✅ CORRECT: Use persistent storage for file systems
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: {
|
||||||
|
type: 'filesystem', // Persisted to disk
|
||||||
|
path: './brainy-data' // Your file storage
|
||||||
|
}
|
||||||
|
})
|
||||||
await brain.init()
|
await brain.init()
|
||||||
|
|
||||||
const vfs = brain.vfs()
|
const vfs = brain.vfs()
|
||||||
await vfs.init()
|
await vfs.init()
|
||||||
|
|
||||||
// Start with familiar files if you want
|
// ✅ Safe file operations
|
||||||
await vfs.writeFile('/story.txt', 'A tale of adventure...')
|
await vfs.writeFile('/projects/app/index.js', 'console.log("Hello")')
|
||||||
|
await vfs.mkdir('/docs')
|
||||||
|
await vfs.writeFile('/docs/README.md', '# My Project')
|
||||||
|
|
||||||
// But knowledge doesn't need files!
|
// ✅ NEVER crashes: Tree-aware directory listing
|
||||||
const alice = await vfs.createEntity({
|
const children = await vfs.getDirectChildren('/projects')
|
||||||
name: 'Alice',
|
// Returns only direct children, never the directory itself
|
||||||
type: 'character',
|
|
||||||
description: 'Brave explorer discovering quantum worlds'
|
// ✅ Build file explorers safely
|
||||||
|
const tree = await vfs.getTreeStructure('/projects', {
|
||||||
|
maxDepth: 3, // Prevent deep recursion
|
||||||
|
sort: 'name' // Organized results
|
||||||
})
|
})
|
||||||
|
|
||||||
const quantumPhysics = await vfs.createConcept({
|
// ✅ Semantic file search
|
||||||
name: 'Quantum Entanglement',
|
const reactFiles = await vfs.search('React components with hooks')
|
||||||
domain: 'physics',
|
const docs = await vfs.search('API documentation', {
|
||||||
keywords: ['superposition', 'measurement', 'correlation']
|
path: '/docs' // Search within specific directory
|
||||||
})
|
})
|
||||||
|
|
||||||
// Connect EVERYTHING - files, entities, concepts
|
// ✅ Connect related files
|
||||||
await vfs.linkEntities(alice, quantumPhysics, 'studies')
|
await vfs.addRelationship('/src/auth.js', '/tests/auth.test.js', 'tested-by')
|
||||||
await vfs.addRelationship('/story.txt', alice.id, 'features')
|
|
||||||
|
|
||||||
// Find knowledge by meaning, not location
|
// Perfect for: File explorers, IDEs, documentation systems, code analysis
|
||||||
const physics = await vfs.search('quantum mechanics')
|
|
||||||
// Returns: files, entities, concepts, relationships - ALL knowledge
|
|
||||||
|
|
||||||
// Knowledge transcends boundaries
|
|
||||||
const aliceKnowledge = await vfs.getEntityGraph(alice)
|
|
||||||
// Her relationships, appearances, evolution - her entire existence
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**🚨 Prevents Common Mistakes:**
|
||||||
|
- ❌ No infinite recursion in file trees (like brain-cloud team experienced)
|
||||||
|
- ❌ No data loss from memory storage
|
||||||
|
- ❌ No performance issues with large directories
|
||||||
|
- ❌ No need for complex fallback patterns
|
||||||
|
|
||||||
|
**[📖 VFS Quick Start →](docs/vfs/QUICK_START.md)** | **[🎯 Common Patterns →](docs/vfs/COMMON_PATTERNS.md)**
|
||||||
|
|
||||||
**Your knowledge isn't trapped anymore.** Characters live beyond stories. APIs exist beyond code files. Concepts connect across domains. This is knowledge that happens to support files, not a filesystem that happens to store knowledge.
|
**Your knowledge isn't trapped anymore.** Characters live beyond stories. APIs exist beyond code files. Concepts connect across domains. This is knowledge that happens to support files, not a filesystem that happens to store knowledge.
|
||||||
|
|
||||||
### 🎯 Zero Configuration Philosophy
|
### 🎯 Zero Configuration Philosophy
|
||||||
|
|
|
||||||
480
docs/API_DECISION_TREE.md
Normal file
480
docs/API_DECISION_TREE.md
Normal file
|
|
@ -0,0 +1,480 @@
|
||||||
|
# 🧠 Brainy API Decision Tree
|
||||||
|
|
||||||
|
*Choose the right API for your use case with confidence*
|
||||||
|
|
||||||
|
This guide helps you navigate Brainy's comprehensive API surface by asking the right questions to find the perfect method for your specific needs.
|
||||||
|
|
||||||
|
## 🎯 Quick Start: What do you want to do?
|
||||||
|
|
||||||
|
### 📝 **Adding Data**
|
||||||
|
- **Single entity** → [`brainy.add()`](#adding-single-entities)
|
||||||
|
- **Multiple entities** → [`brainy.addMany()`](#adding-multiple-entities)
|
||||||
|
- **Streaming/real-time data** → [Streaming Pipeline](#streaming-data)
|
||||||
|
|
||||||
|
### 🔍 **Finding Data**
|
||||||
|
- **Natural language search** → [`brainy.find("search query")`](#natural-language-search)
|
||||||
|
- **Structured/filtered search** → [`brainy.find({ query, where, type })`](#structured-search)
|
||||||
|
- **Similar entities** → [`brainy.similar()`](#similarity-search)
|
||||||
|
- **Get by ID** → [`brainy.get()`](#retrieval-by-id)
|
||||||
|
|
||||||
|
### 🔗 **Relationships**
|
||||||
|
- **Create relationships** → [`brainy.relate()`](#creating-relationships)
|
||||||
|
- **Query relationships** → [`brainy.getRelations()`](#querying-relationships)
|
||||||
|
- **Graph traversal** → [Graph Navigation](#graph-operations)
|
||||||
|
|
||||||
|
### 📊 **Advanced Features**
|
||||||
|
- **File management** → [VFS (Virtual File System)](#file-operations)
|
||||||
|
- **AI-powered analysis** → [Neural API](#neural-analysis)
|
||||||
|
- **Clustering/insights** → [Intelligence Systems](#intelligence-systems)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔀 Decision Tree Flow
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
A[What are you trying to do?] --> B[Store Data]
|
||||||
|
A --> C[Find Data]
|
||||||
|
A --> D[Manage Relationships]
|
||||||
|
A --> E[Work with Files]
|
||||||
|
A --> F[AI Analysis]
|
||||||
|
|
||||||
|
B --> B1[Single Item]
|
||||||
|
B --> B2[Multiple Items]
|
||||||
|
B --> B3[Real-time Stream]
|
||||||
|
|
||||||
|
C --> C1[I know the ID]
|
||||||
|
C --> C2[Natural language query]
|
||||||
|
C --> C3[Complex filters]
|
||||||
|
C --> C4[Find similar items]
|
||||||
|
|
||||||
|
D --> D1[Create relationship]
|
||||||
|
D --> D2[Query relationships]
|
||||||
|
D --> D3[Graph traversal]
|
||||||
|
|
||||||
|
E --> E1[File operations]
|
||||||
|
E --> E2[Knowledge-enhanced files]
|
||||||
|
|
||||||
|
F --> F1[Clustering]
|
||||||
|
F --> F2[Similarity analysis]
|
||||||
|
F --> F3[Insights generation]
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 Adding Data
|
||||||
|
|
||||||
|
### Adding Single Entities
|
||||||
|
|
||||||
|
**Use `brainy.add()` when:**
|
||||||
|
- Adding one entity at a time
|
||||||
|
- You need the ID immediately for further operations
|
||||||
|
- Working with user input or real-time data
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for single entities
|
||||||
|
const id = await brainy.add({
|
||||||
|
data: "New research paper on quantum computing",
|
||||||
|
type: NounType.Document,
|
||||||
|
metadata: { category: "research", priority: "high" }
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**Decision factors:**
|
||||||
|
- **Single item?** → `add()`
|
||||||
|
- **Need immediate ID?** → `add()`
|
||||||
|
- **Interactive application?** → `add()`
|
||||||
|
|
||||||
|
### Adding Multiple Entities
|
||||||
|
|
||||||
|
**Use `brainy.addMany()` when:**
|
||||||
|
- Bulk importing data
|
||||||
|
- Processing batches (>10 items)
|
||||||
|
- Performance is critical
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for bulk operations
|
||||||
|
const result = await brainy.addMany({
|
||||||
|
items: documents.map(doc => ({
|
||||||
|
data: doc.content,
|
||||||
|
type: NounType.Document,
|
||||||
|
metadata: doc.metadata
|
||||||
|
})),
|
||||||
|
chunkSize: 100,
|
||||||
|
parallel: true
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**Decision factors:**
|
||||||
|
- **Multiple items (>10)?** → `addMany()`
|
||||||
|
- **Batch processing?** → `addMany()`
|
||||||
|
- **Can tolerate some failures?** → `addMany()` with `continueOnError: true`
|
||||||
|
|
||||||
|
### Streaming Data
|
||||||
|
|
||||||
|
**Use Streaming Pipeline when:**
|
||||||
|
- Real-time data ingestion
|
||||||
|
- Processing large datasets that don't fit in memory
|
||||||
|
- Need transformation during ingestion
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for streaming
|
||||||
|
const pipeline = brainy.streaming.pipeline()
|
||||||
|
.transform(data => ({ ...data, processed: true }))
|
||||||
|
.batch(50)
|
||||||
|
.into(brainy)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 Finding Data
|
||||||
|
|
||||||
|
### Natural Language Search
|
||||||
|
|
||||||
|
**Use `brainy.find("query string")` when:**
|
||||||
|
- User is typing search queries
|
||||||
|
- You want semantic understanding
|
||||||
|
- Building search interfaces
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for user searches
|
||||||
|
const results = await brainy.find("documents about machine learning")
|
||||||
|
```
|
||||||
|
|
||||||
|
**Decision factors:**
|
||||||
|
- **User-generated query?** → Natural language `find()`
|
||||||
|
- **Semantic understanding needed?** → Natural language `find()`
|
||||||
|
- **Search interface?** → Natural language `find()`
|
||||||
|
|
||||||
|
### Structured Search
|
||||||
|
|
||||||
|
**Use `brainy.find({ query, where, type })` when:**
|
||||||
|
- Complex filtering requirements
|
||||||
|
- Combining text search with metadata filters
|
||||||
|
- Performance-critical searches
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for complex queries
|
||||||
|
const results = await brainy.find({
|
||||||
|
query: "neural networks",
|
||||||
|
type: NounType.Document,
|
||||||
|
where: {
|
||||||
|
status: "published",
|
||||||
|
year: { $gte: 2020 }
|
||||||
|
},
|
||||||
|
limit: 20
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**Decision factors:**
|
||||||
|
- **Need metadata filtering?** → Structured `find()`
|
||||||
|
- **Performance critical?** → Structured `find()`
|
||||||
|
- **Complex criteria?** → Structured `find()`
|
||||||
|
|
||||||
|
### Similarity Search
|
||||||
|
|
||||||
|
**Use `brainy.similar()` when:**
|
||||||
|
- Finding "more like this" content
|
||||||
|
- Recommendation systems
|
||||||
|
- Duplicate detection
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for recommendations
|
||||||
|
const similar = await brainy.similar({
|
||||||
|
to: "document-id-123",
|
||||||
|
limit: 10,
|
||||||
|
type: NounType.Document
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**Decision factors:**
|
||||||
|
- **"More like this" feature?** → `similar()`
|
||||||
|
- **Recommendations?** → `similar()`
|
||||||
|
- **Duplicate detection?** → `similar()`
|
||||||
|
|
||||||
|
### Retrieval by ID
|
||||||
|
|
||||||
|
**Use `brainy.get()` when:**
|
||||||
|
- You know the exact ID
|
||||||
|
- Loading specific entities
|
||||||
|
- Following relationships
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for direct access
|
||||||
|
const entity = await brainy.get("known-id-123")
|
||||||
|
```
|
||||||
|
|
||||||
|
**Decision factors:**
|
||||||
|
- **Known ID?** → `get()`
|
||||||
|
- **Direct access needed?** → `get()`
|
||||||
|
- **Following relationships?** → `get()`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔗 Relationships
|
||||||
|
|
||||||
|
### Creating Relationships
|
||||||
|
|
||||||
|
**Use `brainy.relate()` when:**
|
||||||
|
- Connecting two entities
|
||||||
|
- Building knowledge graphs
|
||||||
|
- Modeling real-world relationships
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for connections
|
||||||
|
await brainy.relate({
|
||||||
|
from: "user-123",
|
||||||
|
to: "project-456",
|
||||||
|
type: VerbType.WorksOn,
|
||||||
|
metadata: { role: "lead", since: "2024-01-01" }
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**Decision factors:**
|
||||||
|
- **Connecting entities?** → `relate()`
|
||||||
|
- **Need relationship metadata?** → `relate()`
|
||||||
|
- **Building graphs?** → `relate()`
|
||||||
|
|
||||||
|
### Querying Relationships
|
||||||
|
|
||||||
|
**Use `brainy.getRelations()` when:**
|
||||||
|
- Finding all connections for an entity
|
||||||
|
- Exploring relationship patterns
|
||||||
|
- Building relationship views
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for relationship queries
|
||||||
|
const relations = await brainy.getRelations({
|
||||||
|
from: "user-123",
|
||||||
|
type: VerbType.WorksOn
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📁 File Operations
|
||||||
|
|
||||||
|
### Basic File Operations
|
||||||
|
|
||||||
|
**Use VFS when:**
|
||||||
|
- Managing files and directories
|
||||||
|
- Need hierarchical structure
|
||||||
|
- Building file explorers
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for file management
|
||||||
|
const vfs = brainy.vfs({ storage: 'filesystem' })
|
||||||
|
await vfs.writeFile('/docs/readme.md', 'content')
|
||||||
|
const files = await vfs.getDirectChildren('/docs')
|
||||||
|
```
|
||||||
|
|
||||||
|
**Decision factors:**
|
||||||
|
- **File management?** → VFS
|
||||||
|
- **Directory structure?** → VFS
|
||||||
|
- **File explorer interface?** → VFS
|
||||||
|
|
||||||
|
### Knowledge-Enhanced Files
|
||||||
|
|
||||||
|
**Use VFS with Knowledge Layer when:**
|
||||||
|
- Need semantic file search
|
||||||
|
- Want AI-powered file insights
|
||||||
|
- Building smart file systems
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for intelligent file systems
|
||||||
|
const knowledgeVFS = await vfs.withKnowledge(brainy)
|
||||||
|
const insights = await knowledgeVFS.getFileInsights('/project')
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧠 AI Analysis
|
||||||
|
|
||||||
|
### Clustering
|
||||||
|
|
||||||
|
**Use Neural API clustering when:**
|
||||||
|
- Discovering data patterns
|
||||||
|
- Organizing large datasets
|
||||||
|
- Creating automatic categories
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for pattern discovery
|
||||||
|
const neural = brainy.neural()
|
||||||
|
const clusters = await neural.cluster({
|
||||||
|
entities: entityIds,
|
||||||
|
k: 5,
|
||||||
|
method: 'hierarchical'
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
### Intelligence Systems
|
||||||
|
|
||||||
|
**Use Triple Intelligence when:**
|
||||||
|
- Complex multi-criteria searches
|
||||||
|
- Advanced relationship queries
|
||||||
|
- Performance-critical operations
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Perfect for complex queries
|
||||||
|
const intelligence = brainy.getTripleIntelligence()
|
||||||
|
const results = await intelligence.query({
|
||||||
|
vector: queryVector,
|
||||||
|
metadata: { category: 'research' },
|
||||||
|
graph: { connected: 'user-123' }
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Performance Optimization Guide
|
||||||
|
|
||||||
|
### When Performance Matters
|
||||||
|
|
||||||
|
| Scenario | Best Choice | Why |
|
||||||
|
|----------|-------------|-----|
|
||||||
|
| **Bulk Import** | `addMany()` | Batched operations, parallel processing |
|
||||||
|
| **Metadata-only Search** | `find({ where: {...} })` | Skips vector computation |
|
||||||
|
| **Known ID Access** | `get()` | Direct index lookup |
|
||||||
|
| **Large Result Sets** | Pagination with `offset`/`limit` | Memory efficient |
|
||||||
|
| **Real-time Streams** | Streaming Pipeline | Memory efficient, scalable |
|
||||||
|
|
||||||
|
### Memory Usage Optimization
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ❌ Memory intensive
|
||||||
|
const allResults = await brainy.find({ limit: 10000 })
|
||||||
|
|
||||||
|
// ✅ Memory efficient
|
||||||
|
for (let offset = 0; offset < total; offset += 100) {
|
||||||
|
const batch = await brainy.find({
|
||||||
|
query: "...",
|
||||||
|
limit: 100,
|
||||||
|
offset
|
||||||
|
})
|
||||||
|
await processBatch(batch)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Common Use Case Patterns
|
||||||
|
|
||||||
|
### Building a Search Interface
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// User types query → Natural language search
|
||||||
|
const searchResults = await brainy.find(userQuery)
|
||||||
|
|
||||||
|
// User applies filters → Structured search
|
||||||
|
const filteredResults = await brainy.find({
|
||||||
|
query: userQuery,
|
||||||
|
where: selectedFilters,
|
||||||
|
type: selectedTypes
|
||||||
|
})
|
||||||
|
|
||||||
|
// User clicks "more like this" → Similarity search
|
||||||
|
const similar = await brainy.similar({ to: selectedId })
|
||||||
|
```
|
||||||
|
|
||||||
|
### Building a Recommendation System
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// 1. Get user's interaction history
|
||||||
|
const user = await brainy.get(userId)
|
||||||
|
|
||||||
|
// 2. Find similar users
|
||||||
|
const similarUsers = await brainy.similar({ to: userId, type: NounType.Person })
|
||||||
|
|
||||||
|
// 3. Get their liked content
|
||||||
|
const recommendations = []
|
||||||
|
for (const similarUser of similarUsers) {
|
||||||
|
const relations = await brainy.getRelations({
|
||||||
|
from: similarUser.id,
|
||||||
|
type: VerbType.Likes
|
||||||
|
})
|
||||||
|
recommendations.push(...relations)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Building a Knowledge Graph
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// 1. Add entities
|
||||||
|
const entities = await Promise.all([
|
||||||
|
brainy.add({ data: "Person: Alice", type: NounType.Person }),
|
||||||
|
brainy.add({ data: "Company: TechCorp", type: NounType.Organization }),
|
||||||
|
brainy.add({ data: "Project: AI Assistant", type: NounType.Thing })
|
||||||
|
])
|
||||||
|
|
||||||
|
// 2. Create relationships
|
||||||
|
await brainy.relate({
|
||||||
|
from: entities[0], // Alice
|
||||||
|
to: entities[1], // TechCorp
|
||||||
|
type: VerbType.WorksFor
|
||||||
|
})
|
||||||
|
|
||||||
|
await brainy.relate({
|
||||||
|
from: entities[0], // Alice
|
||||||
|
to: entities[2], // AI Assistant
|
||||||
|
type: VerbType.WorksOn
|
||||||
|
})
|
||||||
|
|
||||||
|
// 3. Query the graph
|
||||||
|
const aliceConnections = await brainy.getRelations({ from: entities[0] })
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 Migration Guide
|
||||||
|
|
||||||
|
### From v2.x to v3.x APIs
|
||||||
|
|
||||||
|
| v2.x (Deprecated) | v3.x (Current) | When to Use |
|
||||||
|
|-------------------|----------------|-------------|
|
||||||
|
| `brain.store()` | `brainy.add()` | Adding entities |
|
||||||
|
| `brain.search()` | `brainy.find()` | Searching content |
|
||||||
|
| `brain.query()` | `brainy.find({ ... })` | Complex queries |
|
||||||
|
| `brain.similar()` | `brainy.similar()` | ✅ Same API |
|
||||||
|
| `brain.connect()` | `brainy.relate()` | Creating relationships |
|
||||||
|
|
||||||
|
### Legacy Type Migration
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ❌ v2.x way
|
||||||
|
import { ISenseAugmentation } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
|
||||||
|
// ✅ v3.x way
|
||||||
|
import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎪 Decision Quick Reference
|
||||||
|
|
||||||
|
**Need to add data?**
|
||||||
|
- 1 item → `add()`
|
||||||
|
- Many items → `addMany()`
|
||||||
|
- Streaming → Pipeline
|
||||||
|
|
||||||
|
**Need to find data?**
|
||||||
|
- Know ID → `get()`
|
||||||
|
- Natural search → `find("query")`
|
||||||
|
- Complex filters → `find({ query, where })`
|
||||||
|
- Similar items → `similar()`
|
||||||
|
|
||||||
|
**Need relationships?**
|
||||||
|
- Create → `relate()`
|
||||||
|
- Query → `getRelations()`
|
||||||
|
- Complex graph → Triple Intelligence
|
||||||
|
|
||||||
|
**Need files?**
|
||||||
|
- Basic → VFS
|
||||||
|
- Smart → VFS + Knowledge Layer
|
||||||
|
|
||||||
|
**Need AI analysis?**
|
||||||
|
- Patterns → Neural clustering
|
||||||
|
- Complex queries → Triple Intelligence
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*This guide covers 95% of use cases. For edge cases or custom requirements, check the [Core API Patterns](./CORE_API_PATTERNS.md) and [Neural API Patterns](./NEURAL_API_PATTERNS.md) guides.*
|
||||||
643
docs/CORE_API_PATTERNS.md
Normal file
643
docs/CORE_API_PATTERNS.md
Normal file
|
|
@ -0,0 +1,643 @@
|
||||||
|
# 🧠 Core API Patterns: Modern Brainy v3.x
|
||||||
|
|
||||||
|
> Learn the correct patterns for Brainy's core operations. Avoid v2.x confusion and use modern, efficient APIs.
|
||||||
|
|
||||||
|
## 🚨 Critical: Use v3.x APIs Only
|
||||||
|
|
||||||
|
### ❌ **WRONG - Deprecated v2.x APIs**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - These methods don't exist in v3.x!
|
||||||
|
await brain.addNoun(text, type, metadata) // ❌ Removed
|
||||||
|
await brain.getNouns({ pagination }) // ❌ Removed
|
||||||
|
await brain.addVerb(source, target, type) // ❌ Removed
|
||||||
|
await brain.getVerbs() // ❌ Removed
|
||||||
|
await brain.deleteNoun(id) // ❌ Removed
|
||||||
|
await brain.deleteVerb(id) // ❌ Removed
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Modern v3.x APIs**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Use these modern methods instead
|
||||||
|
await brain.add({ data, type, metadata }) // Modern unified add
|
||||||
|
await brain.find({ limit: 100 }) // Natural language search
|
||||||
|
await brain.relate({ from, to, type }) // Clean relationship creation
|
||||||
|
await brain.getRelations() // Modern relationship queries
|
||||||
|
await brain.delete(id) // Unified deletion
|
||||||
|
// Relationships auto-cascade when entities are deleted
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📋 Entity Management Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - v2.x Style**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Old API patterns
|
||||||
|
import { BrainyData } from 'old-brainy' // ❌ Wrong import
|
||||||
|
|
||||||
|
const brain = new BrainyData({ // ❌ Old class name
|
||||||
|
complexConfig: true
|
||||||
|
})
|
||||||
|
|
||||||
|
const id = await brain.addNoun( // ❌ Deprecated method
|
||||||
|
"John Smith is a developer",
|
||||||
|
"Person",
|
||||||
|
{ role: "engineer" }
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Modern Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Basic entity creation
|
||||||
|
import { Brainy, NounType } from '@soulcraft/brainy'
|
||||||
|
|
||||||
|
const brain = new Brainy() // ✅ Zero config
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
const id = await brain.add({
|
||||||
|
data: "John Smith is a developer",
|
||||||
|
type: NounType.Person,
|
||||||
|
metadata: { role: "engineer", team: "backend" }
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Bulk entity creation
|
||||||
|
const entities = [
|
||||||
|
{ data: "React framework", type: NounType.Technology },
|
||||||
|
{ data: "Vue.js framework", type: NounType.Technology },
|
||||||
|
{ data: "Angular framework", type: NounType.Technology }
|
||||||
|
]
|
||||||
|
|
||||||
|
const ids = await Promise.all(
|
||||||
|
entities.map(entity => brain.add(entity))
|
||||||
|
)
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Entity with pre-computed vector
|
||||||
|
const customVector = await brain.embed("Custom text")
|
||||||
|
const vectorId = await brain.add({
|
||||||
|
data: "Optimized content",
|
||||||
|
type: NounType.Document,
|
||||||
|
vector: customVector, // Skip re-embedding
|
||||||
|
metadata: { source: "api", optimized: true }
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔍 Search & Discovery Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Confusing Old Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Mixing old and new APIs
|
||||||
|
const results1 = await brain.searchText("query") // ❌ Old method
|
||||||
|
const results2 = await brain.getNouns({ filter }) // ❌ Doesn't exist
|
||||||
|
const results3 = await brain.findSimilar(text) // ❌ Unclear naming
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Clean Search Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Natural language search
|
||||||
|
const results = await brain.find("React developers working on authentication")
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Structured search with filters
|
||||||
|
const filteredResults = await brain.find({
|
||||||
|
like: "machine learning", // Vector similarity
|
||||||
|
where: { // Metadata filtering
|
||||||
|
type: NounType.Document,
|
||||||
|
year: { $gte: 2020 },
|
||||||
|
status: "published"
|
||||||
|
},
|
||||||
|
limit: 50,
|
||||||
|
orderBy: 'relevance'
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Similarity search
|
||||||
|
const similarItems = await brain.similar({
|
||||||
|
to: existingEntityId, // Find items similar to this
|
||||||
|
threshold: 0.8, // Minimum similarity
|
||||||
|
limit: 10,
|
||||||
|
exclude: [existingEntityId] // Don't include the source
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Advanced search with relationships
|
||||||
|
const connectedResults = await brain.find({
|
||||||
|
like: "frontend frameworks",
|
||||||
|
connected: {
|
||||||
|
to: reactId, // Connected to React
|
||||||
|
via: "related-to", // Through this relationship
|
||||||
|
depth: 2 // Up to 2 hops away
|
||||||
|
}
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔗 Relationship Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Old Relationship APIs**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Old relationship patterns
|
||||||
|
await brain.addVerb(sourceId, targetId, "uses", { strength: 0.9 }) // ❌ Old API
|
||||||
|
const verbs = await brain.getVerbsBySource(sourceId) // ❌ Removed
|
||||||
|
await brain.deleteVerb(verbId) // ❌ Old pattern
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Modern Relationship Management**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Create relationships
|
||||||
|
const relationId = await brain.relate({
|
||||||
|
from: developerId,
|
||||||
|
to: frameworkId,
|
||||||
|
type: VerbType.Uses,
|
||||||
|
metadata: {
|
||||||
|
since: "2023-01-01",
|
||||||
|
proficiency: "expert",
|
||||||
|
hours_per_week: 40
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Query relationships
|
||||||
|
const relationships = await brain.getRelations({
|
||||||
|
from: developerId, // Relationships from this entity
|
||||||
|
type: VerbType.Uses, // Of this type
|
||||||
|
limit: 100
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Bidirectional relationships
|
||||||
|
await brain.relate({
|
||||||
|
from: projectId,
|
||||||
|
to: developerId,
|
||||||
|
type: VerbType.AssignedTo,
|
||||||
|
bidirectional: true, // Creates reverse relationship
|
||||||
|
metadata: { role: "lead", start_date: "2024-01-01" }
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Relationship-based discovery
|
||||||
|
const collaborators = await brain.find({
|
||||||
|
connected: {
|
||||||
|
to: currentProjectId,
|
||||||
|
via: VerbType.WorksOn,
|
||||||
|
direction: "incoming" // Who works on this project
|
||||||
|
}
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🗃️ Data Retrieval Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Inefficient Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Loading everything
|
||||||
|
const everything = await brain.getNouns({ limit: 1000000 }) // ❌ Crashes
|
||||||
|
const allData = await brain.exportAll() // ❌ Memory explosion
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Efficient Data Access**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Paginated retrieval
|
||||||
|
async function getAllEntitiesPaginated() {
|
||||||
|
const pageSize = 100
|
||||||
|
let offset = 0
|
||||||
|
let allEntities = []
|
||||||
|
|
||||||
|
while (true) {
|
||||||
|
const page = await brain.find({
|
||||||
|
limit: pageSize,
|
||||||
|
offset: offset
|
||||||
|
})
|
||||||
|
|
||||||
|
if (page.length === 0) break
|
||||||
|
|
||||||
|
allEntities.push(...page)
|
||||||
|
offset += pageSize
|
||||||
|
|
||||||
|
// Optional: Progress reporting
|
||||||
|
console.log(`Loaded ${allEntities.length} entities...`)
|
||||||
|
}
|
||||||
|
|
||||||
|
return allEntities
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Streaming large datasets
|
||||||
|
async function* streamEntities() {
|
||||||
|
const pageSize = 50
|
||||||
|
let offset = 0
|
||||||
|
|
||||||
|
while (true) {
|
||||||
|
const page = await brain.find({
|
||||||
|
limit: pageSize,
|
||||||
|
offset: offset
|
||||||
|
})
|
||||||
|
|
||||||
|
if (page.length === 0) break
|
||||||
|
|
||||||
|
for (const entity of page) {
|
||||||
|
yield entity
|
||||||
|
}
|
||||||
|
|
||||||
|
offset += pageSize
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Usage
|
||||||
|
for await (const entity of streamEntities()) {
|
||||||
|
await processEntity(entity)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Specific entity retrieval
|
||||||
|
const entity = await brain.get(entityId)
|
||||||
|
if (entity) {
|
||||||
|
console.log('Entity data:', entity.data)
|
||||||
|
console.log('Metadata:', entity.metadata)
|
||||||
|
} else {
|
||||||
|
console.log('Entity not found')
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔄 Update & Delete Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Manual Update Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Recreating entities
|
||||||
|
await brain.delete(oldId)
|
||||||
|
const newId = await brain.add(updatedData) // ❌ Loses relationships
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Update Operations**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Update entity data
|
||||||
|
await brain.update(entityId, {
|
||||||
|
data: "Updated content here",
|
||||||
|
metadata: {
|
||||||
|
lastModified: Date.now(),
|
||||||
|
version: "2.0"
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Partial metadata updates
|
||||||
|
await brain.updateMetadata(entityId, {
|
||||||
|
status: "published",
|
||||||
|
tags: ["important", "featured"]
|
||||||
|
// Merges with existing metadata
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Safe deletion with cascade options
|
||||||
|
await brain.delete(entityId, {
|
||||||
|
cascade: true, // Delete related relationships
|
||||||
|
backup: true // Create backup before deletion
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Bulk operations
|
||||||
|
const updateOperations = entities.map(entity => ({
|
||||||
|
id: entity.id,
|
||||||
|
changes: { status: "processed" }
|
||||||
|
}))
|
||||||
|
|
||||||
|
await brain.updateMany(updateOperations)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🧮 Vector & Embedding Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Manual Vector Handling**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Manual embedding without understanding
|
||||||
|
const vector = await brain.embed(text)
|
||||||
|
// Store vector somewhere manually // ❌ Missing integration
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Smart Vector Operations**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Automatic embedding (recommended)
|
||||||
|
const id = await brain.add({
|
||||||
|
data: "Content to be embedded",
|
||||||
|
type: NounType.Document
|
||||||
|
// Vector computed automatically
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Pre-computed vectors for optimization
|
||||||
|
const texts = ["Text 1", "Text 2", "Text 3"]
|
||||||
|
const vectors = await Promise.all(
|
||||||
|
texts.map(text => brain.embed(text))
|
||||||
|
)
|
||||||
|
|
||||||
|
const entities = await Promise.all(
|
||||||
|
texts.map((text, i) => brain.add({
|
||||||
|
data: text,
|
||||||
|
type: NounType.Document,
|
||||||
|
vector: vectors[i] // Skip re-embedding
|
||||||
|
}))
|
||||||
|
)
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Vector similarity search
|
||||||
|
const queryVector = await brain.embed("search query")
|
||||||
|
const similar = await brain.similar({
|
||||||
|
vector: queryVector, // Use vector directly
|
||||||
|
threshold: 0.75,
|
||||||
|
limit: 20
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Compare vectors directly
|
||||||
|
const vector1 = await brain.embed("First text")
|
||||||
|
const vector2 = await brain.embed("Second text")
|
||||||
|
const similarity = brain.computeSimilarity(vector1, vector2)
|
||||||
|
console.log(`Similarity: ${similarity}`)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🏗️ Configuration Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Over-Configuration**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Complex configurations that break
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: {
|
||||||
|
type: 'complex',
|
||||||
|
options: {
|
||||||
|
nested: {
|
||||||
|
configuration: true,
|
||||||
|
that: "breaks"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
embedding: {
|
||||||
|
customModel: "broken-model",
|
||||||
|
dimensions: 999999
|
||||||
|
}
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Smart Configuration**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Zero configuration (recommended)
|
||||||
|
const brain = new Brainy() // Auto-detects everything
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Simple storage selection
|
||||||
|
const fsBrain = new Brainy({
|
||||||
|
storage: { type: 'filesystem', path: './data' }
|
||||||
|
})
|
||||||
|
|
||||||
|
const cloudBrain = new Brainy({
|
||||||
|
storage: { type: 's3', bucket: 'my-data' }
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Production configuration
|
||||||
|
const prodBrain = new Brainy({
|
||||||
|
storage: {
|
||||||
|
type: 's3',
|
||||||
|
bucket: process.env.BRAINY_BUCKET,
|
||||||
|
region: process.env.AWS_REGION
|
||||||
|
},
|
||||||
|
silent: true, // No console output
|
||||||
|
distributed: true, // Enable clustering
|
||||||
|
cache: { maxSize: 10000 } // Larger cache
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Development vs production
|
||||||
|
const isDev = process.env.NODE_ENV === 'development'
|
||||||
|
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: isDev
|
||||||
|
? { type: 'memory' } // Fast for dev
|
||||||
|
: { type: 'filesystem', path: './brainy-data' }, // Persistent for prod
|
||||||
|
silent: !isDev, // Verbose in dev, quiet in prod
|
||||||
|
cache: { maxSize: isDev ? 100 : 5000 }
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔄 Migration from v2.x
|
||||||
|
|
||||||
|
### ✅ **Migration Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// If you have old v2.x code, here's how to migrate:
|
||||||
|
|
||||||
|
// OLD v2.x:
|
||||||
|
// await brain.addNoun(text, type, metadata)
|
||||||
|
// NEW v3.x:
|
||||||
|
await brain.add({ data: text, type, metadata })
|
||||||
|
|
||||||
|
// OLD v2.x:
|
||||||
|
// await brain.getNouns({ pagination: { limit: 100 } })
|
||||||
|
// NEW v3.x:
|
||||||
|
await brain.find({ limit: 100 })
|
||||||
|
|
||||||
|
// OLD v2.x:
|
||||||
|
// await brain.addVerb(sourceId, targetId, verbType, metadata)
|
||||||
|
// NEW v3.x:
|
||||||
|
await brain.relate({ from: sourceId, to: targetId, type: verbType, metadata })
|
||||||
|
|
||||||
|
// OLD v2.x:
|
||||||
|
// await brain.searchText(query)
|
||||||
|
// NEW v3.x:
|
||||||
|
await brain.find(query) // More powerful natural language search
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🚀 Performance Patterns
|
||||||
|
|
||||||
|
### ✅ **High-Performance Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Batch operations
|
||||||
|
const entities = [/* large array */]
|
||||||
|
const batchSize = 100
|
||||||
|
|
||||||
|
for (let i = 0; i < entities.length; i += batchSize) {
|
||||||
|
const batch = entities.slice(i, i + batchSize)
|
||||||
|
await Promise.all(
|
||||||
|
batch.map(entity => brain.add(entity))
|
||||||
|
)
|
||||||
|
|
||||||
|
// Optional: Rate limiting
|
||||||
|
await new Promise(resolve => setTimeout(resolve, 100))
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Connection pooling for distributed
|
||||||
|
const brain = new Brainy({
|
||||||
|
distributed: true,
|
||||||
|
connectionPool: {
|
||||||
|
min: 5,
|
||||||
|
max: 50,
|
||||||
|
acquireTimeoutMillis: 30000
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Efficient caching
|
||||||
|
const brain = new Brainy({
|
||||||
|
cache: {
|
||||||
|
maxSize: 10000, // Number of items
|
||||||
|
ttl: 300000, // 5 minutes
|
||||||
|
updateAgeOnGet: true // LRU behavior
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Memory-conscious operations
|
||||||
|
const results = await brain.find({
|
||||||
|
query: "large dataset query",
|
||||||
|
limit: 1000, // Reasonable limit
|
||||||
|
includeVectors: false // Exclude vectors if not needed
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🛡️ Error Handling Patterns
|
||||||
|
|
||||||
|
### ✅ **Robust Error Handling**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Specific error handling
|
||||||
|
try {
|
||||||
|
const result = await brain.add({ data, type, metadata })
|
||||||
|
return result
|
||||||
|
} catch (error) {
|
||||||
|
if (error.code === 'DUPLICATE_ENTITY') {
|
||||||
|
console.log('Entity already exists, updating instead...')
|
||||||
|
return await brain.update(error.existingId, { data, metadata })
|
||||||
|
} else if (error.code === 'STORAGE_FULL') {
|
||||||
|
throw new Error('Storage capacity exceeded')
|
||||||
|
} else if (error.code === 'EMBEDDING_FAILED') {
|
||||||
|
console.warn('Embedding failed, retrying with simpler text...')
|
||||||
|
return await brain.add({
|
||||||
|
data: data.substring(0, 1000), // Truncate
|
||||||
|
type,
|
||||||
|
metadata
|
||||||
|
})
|
||||||
|
}
|
||||||
|
throw error
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Retry with exponential backoff
|
||||||
|
async function resilientAdd(data: any, maxRetries = 3) {
|
||||||
|
for (let attempt = 1; attempt <= maxRetries; attempt++) {
|
||||||
|
try {
|
||||||
|
return await brain.add(data)
|
||||||
|
} catch (error) {
|
||||||
|
if (attempt === maxRetries) throw error
|
||||||
|
|
||||||
|
const delay = Math.pow(2, attempt) * 1000
|
||||||
|
console.warn(`Attempt ${attempt} failed, retrying in ${delay}ms...`)
|
||||||
|
await new Promise(resolve => setTimeout(resolve, delay))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Graceful degradation
|
||||||
|
async function robustSearch(query: string) {
|
||||||
|
try {
|
||||||
|
// Try advanced semantic search first
|
||||||
|
return await brain.find({
|
||||||
|
like: query,
|
||||||
|
threshold: 0.8,
|
||||||
|
limit: 50
|
||||||
|
})
|
||||||
|
} catch (error) {
|
||||||
|
console.warn('Semantic search failed, falling back to basic search:', error.message)
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Fallback to simple text search
|
||||||
|
return await brain.find(query)
|
||||||
|
} catch (fallbackError) {
|
||||||
|
console.error('All search methods failed:', fallbackError.message)
|
||||||
|
return [] // Return empty results rather than crash
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📊 Monitoring Patterns
|
||||||
|
|
||||||
|
### ✅ **Production Monitoring**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Performance monitoring
|
||||||
|
const startTime = Date.now()
|
||||||
|
const result = await brain.add(data)
|
||||||
|
const duration = Date.now() - startTime
|
||||||
|
|
||||||
|
if (duration > 1000) {
|
||||||
|
console.warn(`Slow add operation: ${duration}ms`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Health checks
|
||||||
|
async function healthCheck() {
|
||||||
|
try {
|
||||||
|
// Test basic operations
|
||||||
|
const testId = await brain.add({
|
||||||
|
data: "health check",
|
||||||
|
type: NounType.System,
|
||||||
|
metadata: { test: true }
|
||||||
|
})
|
||||||
|
|
||||||
|
await brain.get(testId)
|
||||||
|
await brain.delete(testId)
|
||||||
|
|
||||||
|
return { status: 'healthy', timestamp: Date.now() }
|
||||||
|
} catch (error) {
|
||||||
|
return {
|
||||||
|
status: 'unhealthy',
|
||||||
|
error: error.message,
|
||||||
|
timestamp: Date.now()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Metrics collection
|
||||||
|
class BrainyMetrics {
|
||||||
|
private metrics = {
|
||||||
|
operations: 0,
|
||||||
|
errors: 0,
|
||||||
|
totalTime: 0
|
||||||
|
}
|
||||||
|
|
||||||
|
async timedOperation<T>(operation: () => Promise<T>): Promise<T> {
|
||||||
|
const start = Date.now()
|
||||||
|
try {
|
||||||
|
const result = await operation()
|
||||||
|
this.metrics.operations++
|
||||||
|
this.metrics.totalTime += Date.now() - start
|
||||||
|
return result
|
||||||
|
} catch (error) {
|
||||||
|
this.metrics.errors++
|
||||||
|
throw error
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
getStats() {
|
||||||
|
return {
|
||||||
|
...this.metrics,
|
||||||
|
avgTime: this.metrics.totalTime / this.metrics.operations || 0,
|
||||||
|
errorRate: this.metrics.errors / this.metrics.operations || 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎯 Summary: Modern Brainy v3.x Best Practices
|
||||||
|
|
||||||
|
| ❌ **Avoid v2.x** | ✅ **Use v3.x** |
|
||||||
|
|------------------|----------------|
|
||||||
|
| `addNoun()` | `add()` |
|
||||||
|
| `getNouns()` | `find()` |
|
||||||
|
| `addVerb()` | `relate()` |
|
||||||
|
| `getVerbs()` | `getRelations()` |
|
||||||
|
| `deleteNoun()` | `delete()` |
|
||||||
|
| Complex configs | Zero-config with `new Brainy()` |
|
||||||
|
| Manual pagination | Built-in smart pagination |
|
||||||
|
| String-based search | Natural language queries |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**🎉 Following these patterns gives you:**
|
||||||
|
- 🚀 **Modern APIs** that are actively maintained
|
||||||
|
- ⚡ **Better performance** with intelligent defaults
|
||||||
|
- 🛡️ **Robust error handling** with specific error types
|
||||||
|
- 📈 **Scalable patterns** for production applications
|
||||||
|
- 🧠 **Natural language** search capabilities
|
||||||
|
|
||||||
|
**Next:** [Neural API Patterns →](./NEURAL_API_PATTERNS.md) | [VFS Patterns →](./vfs/COMMON_PATTERNS.md)
|
||||||
736
docs/NEURAL_API_PATTERNS.md
Normal file
736
docs/NEURAL_API_PATTERNS.md
Normal file
|
|
@ -0,0 +1,736 @@
|
||||||
|
# 🧠 Neural API Patterns: AI-Powered Intelligence
|
||||||
|
|
||||||
|
> Learn the correct patterns for Brainy's Neural API. Avoid performance pitfalls and use AI features effectively.
|
||||||
|
|
||||||
|
## 🚨 Critical: Access Neural APIs Correctly
|
||||||
|
|
||||||
|
### ❌ **WRONG - Outdated Access Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Outdated documentation patterns
|
||||||
|
import { BrainyData } from '@soulcraft/brainy' // ❌ Wrong import
|
||||||
|
const brain = new BrainyData() // ❌ Old class name
|
||||||
|
|
||||||
|
// These may not work as expected:
|
||||||
|
const neural = brain.neural // ❌ May be undefined
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Modern Neural Access**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Use modern Brainy class
|
||||||
|
import { Brainy } from '@soulcraft/brainy'
|
||||||
|
|
||||||
|
const brain = new Brainy()
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
// ✅ Neural API is available after initialization
|
||||||
|
const clusters = await brain.neural.clusters()
|
||||||
|
const similarity = await brain.neural.similar('item1', 'item2')
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔍 Similarity Analysis Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Inefficient Similarity Checks**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - N² comparisons
|
||||||
|
const items = await brain.find({ limit: 1000 })
|
||||||
|
const similarities = []
|
||||||
|
|
||||||
|
for (const item1 of items) {
|
||||||
|
for (const item2 of items) {
|
||||||
|
if (item1.id !== item2.id) {
|
||||||
|
const sim = await brain.neural.similar(item1.id, item2.id) // ❌ Millions of calls
|
||||||
|
similarities.push({ from: item1.id, to: item2.id, score: sim })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Efficient Similarity Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Find neighbors (much more efficient)
|
||||||
|
const item = await brain.get('target-item-id')
|
||||||
|
const neighbors = await brain.neural.neighbors(item.id, {
|
||||||
|
limit: 10, // Top 10 most similar
|
||||||
|
threshold: 0.7, // Minimum similarity
|
||||||
|
includeScores: true // Include similarity scores
|
||||||
|
})
|
||||||
|
|
||||||
|
console.log(`Found ${neighbors.length} similar items`)
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Batch similarity for specific pairs
|
||||||
|
const itemPairs = [
|
||||||
|
['item1', 'item2'],
|
||||||
|
['item1', 'item3'],
|
||||||
|
['item2', 'item3']
|
||||||
|
]
|
||||||
|
|
||||||
|
const similarities = await Promise.all(
|
||||||
|
itemPairs.map(async ([a, b]) => ({
|
||||||
|
from: a,
|
||||||
|
to: b,
|
||||||
|
score: await brain.neural.similar(a, b)
|
||||||
|
}))
|
||||||
|
)
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Text-to-text similarity (no need for IDs)
|
||||||
|
const textSimilarity = await brain.neural.similar(
|
||||||
|
"Machine learning is fascinating",
|
||||||
|
"AI and deep learning are interesting",
|
||||||
|
{ detailed: true } // Get explanation of similarity
|
||||||
|
)
|
||||||
|
|
||||||
|
console.log(`Similarity: ${textSimilarity.score}`)
|
||||||
|
console.log(`Explanation: ${textSimilarity.explanation}`)
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Vector-level similarity for optimization
|
||||||
|
const vector1 = await brain.embed("First concept")
|
||||||
|
const vector2 = await brain.embed("Second concept")
|
||||||
|
const vectorSimilarity = await brain.neural.similar(vector1, vector2)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎯 Clustering Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Uncontrolled Clustering**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Clustering everything without limits
|
||||||
|
const everything = await brain.find({ limit: 100000 }) // ❌ Too much data
|
||||||
|
const clusters = await brain.neural.clusters() // ❌ May crash or timeout
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Smart Clustering Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Controlled clustering with limits
|
||||||
|
const recentItems = await brain.find({
|
||||||
|
where: {
|
||||||
|
createdAt: { $gte: Date.now() - 30 * 24 * 60 * 60 * 1000 } // Last 30 days
|
||||||
|
},
|
||||||
|
limit: 1000 // Reasonable limit
|
||||||
|
})
|
||||||
|
|
||||||
|
const clusters = await brain.neural.clusters(
|
||||||
|
recentItems.map(item => item.id),
|
||||||
|
{
|
||||||
|
algorithm: 'kmeans', // Reliable algorithm
|
||||||
|
maxClusters: 10, // Reasonable number
|
||||||
|
threshold: 0.75, // High similarity required
|
||||||
|
iterations: 50 // Convergence limit
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Domain-specific clustering
|
||||||
|
const techDocs = await brain.find({
|
||||||
|
where: { category: 'technology', type: 'document' },
|
||||||
|
limit: 500
|
||||||
|
})
|
||||||
|
|
||||||
|
const techClusters = await brain.neural.clusterByDomain(
|
||||||
|
'category', // Group by this field
|
||||||
|
{
|
||||||
|
items: techDocs.map(doc => doc.id),
|
||||||
|
minClusterSize: 3, // Minimum items per cluster
|
||||||
|
maxClusters: 8
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Temporal clustering for time-series data
|
||||||
|
const timebasedClusters = await brain.neural.clusterByTime(
|
||||||
|
'createdAt', // Time field
|
||||||
|
'week', // Time window (hour, day, week, month)
|
||||||
|
{
|
||||||
|
items: recentItems.map(item => item.id),
|
||||||
|
overlap: 0.2, // 20% overlap between windows
|
||||||
|
minPerWindow: 5 // Minimum items per time window
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Streaming clustering for large datasets
|
||||||
|
async function clusterLargeDataset() {
|
||||||
|
const clusterStream = brain.neural.clusterStream({
|
||||||
|
batchSize: 100, // Process 100 items at a time
|
||||||
|
updateInterval: 1000, // Update clusters every 1000 items
|
||||||
|
maxMemory: 512 * 1024 * 1024 // 512MB memory limit
|
||||||
|
})
|
||||||
|
|
||||||
|
const allClusters = []
|
||||||
|
for await (const batch of clusterStream) {
|
||||||
|
console.log(`Processed ${batch.processed} items, found ${batch.clusters.length} clusters`)
|
||||||
|
allClusters.push(...batch.clusters)
|
||||||
|
}
|
||||||
|
|
||||||
|
return allClusters
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔍 Neighbor Discovery Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Manual Similarity Searches**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Reinventing neighbor search
|
||||||
|
async function findSimilarManually(targetId: string) {
|
||||||
|
const allItems = await brain.find({ limit: 10000 }) // ❌ Load everything
|
||||||
|
const similarities = []
|
||||||
|
|
||||||
|
for (const item of allItems) {
|
||||||
|
if (item.id !== targetId) {
|
||||||
|
const score = await brain.neural.similar(targetId, item.id) // ❌ Slow
|
||||||
|
if (score > 0.7) {
|
||||||
|
similarities.push({ id: item.id, score })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return similarities.sort((a, b) => b.score - a.score).slice(0, 10) // ❌ Inefficient
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Optimized Neighbor Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Basic neighbor search
|
||||||
|
const neighbors = await brain.neural.neighbors('target-item-id', {
|
||||||
|
limit: 20, // Top 20 neighbors
|
||||||
|
threshold: 0.6, // Minimum similarity
|
||||||
|
includeMetadata: true, // Include item metadata
|
||||||
|
includeDistances: true // Include exact similarity scores
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Filtered neighbor search
|
||||||
|
const filteredNeighbors = await brain.neural.neighbors('article-id', {
|
||||||
|
limit: 10,
|
||||||
|
filter: {
|
||||||
|
type: 'document', // Only find similar documents
|
||||||
|
status: 'published', // Only published content
|
||||||
|
language: 'en' // Only English content
|
||||||
|
},
|
||||||
|
excludeIds: ['self-id', 'duplicate-id'] // Exclude specific items
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Multi-level neighbor discovery
|
||||||
|
async function discoverNeighborNetwork(rootId: string, maxDepth = 2) {
|
||||||
|
const network = new Map()
|
||||||
|
const visited = new Set()
|
||||||
|
const queue = [{ id: rootId, depth: 0 }]
|
||||||
|
|
||||||
|
while (queue.length > 0) {
|
||||||
|
const { id, depth } = queue.shift()!
|
||||||
|
|
||||||
|
if (visited.has(id) || depth >= maxDepth) continue
|
||||||
|
visited.add(id)
|
||||||
|
|
||||||
|
const neighbors = await brain.neural.neighbors(id, {
|
||||||
|
limit: 5,
|
||||||
|
threshold: 0.8
|
||||||
|
})
|
||||||
|
|
||||||
|
network.set(id, neighbors)
|
||||||
|
|
||||||
|
// Add neighbors to queue for next depth level
|
||||||
|
if (depth < maxDepth - 1) {
|
||||||
|
neighbors.forEach(neighbor => {
|
||||||
|
queue.push({ id: neighbor.id, depth: depth + 1 })
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return network
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Recommendation engine
|
||||||
|
async function getRecommendations(userId: string) {
|
||||||
|
// Get user's liked items
|
||||||
|
const userItems = await brain.find({
|
||||||
|
connected: { to: userId, via: 'liked-by' }
|
||||||
|
})
|
||||||
|
|
||||||
|
// Find neighbors for each liked item
|
||||||
|
const allNeighbors = await Promise.all(
|
||||||
|
userItems.map(item =>
|
||||||
|
brain.neural.neighbors(item.id, {
|
||||||
|
limit: 10,
|
||||||
|
threshold: 0.7,
|
||||||
|
excludeConnected: { to: userId, via: 'liked-by' } // Exclude already liked
|
||||||
|
})
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
// Aggregate and rank recommendations
|
||||||
|
const recommendations = new Map()
|
||||||
|
allNeighbors.flat().forEach(neighbor => {
|
||||||
|
const current = recommendations.get(neighbor.id) || { score: 0, count: 0 }
|
||||||
|
current.score += neighbor.score
|
||||||
|
current.count += 1
|
||||||
|
recommendations.set(neighbor.id, current)
|
||||||
|
})
|
||||||
|
|
||||||
|
// Return top recommendations by average score
|
||||||
|
return Array.from(recommendations.entries())
|
||||||
|
.map(([id, stats]) => ({
|
||||||
|
id,
|
||||||
|
avgScore: stats.score / stats.count,
|
||||||
|
mentions: stats.count
|
||||||
|
}))
|
||||||
|
.sort((a, b) => b.avgScore - a.avgScore)
|
||||||
|
.slice(0, 10)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🏗️ Hierarchy & Structure Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Manual Hierarchy Building**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Building hierarchies manually
|
||||||
|
async function buildHierarchyManually(rootId: string) {
|
||||||
|
const root = await brain.get(rootId)
|
||||||
|
const allItems = await brain.find({ limit: 1000 }) // ❌ Load everything
|
||||||
|
|
||||||
|
// Manual tree building with nested loops
|
||||||
|
const hierarchy = { root, children: [] }
|
||||||
|
// ... complex manual logic
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Semantic Hierarchy Patterns**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Automatic semantic hierarchy
|
||||||
|
const hierarchy = await brain.neural.hierarchy('root-concept-id', {
|
||||||
|
maxDepth: 4, // Maximum tree depth
|
||||||
|
minSimilarity: 0.6, // Minimum similarity for inclusion
|
||||||
|
branchingFactor: 5, // Maximum children per node
|
||||||
|
algorithm: 'semantic' // Use semantic clustering
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Domain-specific hierarchy
|
||||||
|
const techHierarchy = await brain.neural.hierarchy('technology-id', {
|
||||||
|
filter: { category: 'technology' },
|
||||||
|
weights: {
|
||||||
|
semantic: 0.7, // 70% based on content similarity
|
||||||
|
metadata: 0.3 // 30% based on metadata similarity
|
||||||
|
},
|
||||||
|
includeMetrics: true // Include hierarchy quality metrics
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Multi-root hierarchy for complex domains
|
||||||
|
async function buildMultiRootHierarchy(rootIds: string[]) {
|
||||||
|
const hierarchies = await Promise.all(
|
||||||
|
rootIds.map(rootId =>
|
||||||
|
brain.neural.hierarchy(rootId, {
|
||||||
|
maxDepth: 3,
|
||||||
|
crossReference: true // Allow cross-hierarchy connections
|
||||||
|
})
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
// Merge hierarchies and find connections
|
||||||
|
const merged = {
|
||||||
|
roots: hierarchies,
|
||||||
|
connections: await findCrossHierarchyConnections(hierarchies)
|
||||||
|
}
|
||||||
|
|
||||||
|
return merged
|
||||||
|
}
|
||||||
|
|
||||||
|
async function findCrossHierarchyConnections(hierarchies: any[]) {
|
||||||
|
const connections = []
|
||||||
|
|
||||||
|
for (let i = 0; i < hierarchies.length; i++) {
|
||||||
|
for (let j = i + 1; j < hierarchies.length; j++) {
|
||||||
|
const leafNodes1 = extractLeafNodes(hierarchies[i])
|
||||||
|
const leafNodes2 = extractLeafNodes(hierarchies[j])
|
||||||
|
|
||||||
|
// Find connections between leaf nodes of different hierarchies
|
||||||
|
for (const leaf1 of leafNodes1) {
|
||||||
|
const neighbors = await brain.neural.neighbors(leaf1.id, {
|
||||||
|
limit: 5,
|
||||||
|
threshold: 0.8,
|
||||||
|
filter: { id: { $in: leafNodes2.map(l => l.id) } }
|
||||||
|
})
|
||||||
|
|
||||||
|
connections.push(...neighbors.map(n => ({
|
||||||
|
from: leaf1.id,
|
||||||
|
to: n.id,
|
||||||
|
hierarchyPair: [i, j],
|
||||||
|
similarity: n.score
|
||||||
|
})))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return connections
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🚨 Outlier Detection Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Manual Outlier Detection**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Manual statistical outlier detection
|
||||||
|
async function findOutliersManually() {
|
||||||
|
const items = await brain.find({ limit: 1000 })
|
||||||
|
const similarities = []
|
||||||
|
|
||||||
|
// Calculate average similarity for each item (expensive)
|
||||||
|
for (const item of items) {
|
||||||
|
let totalSim = 0
|
||||||
|
let count = 0
|
||||||
|
for (const other of items) {
|
||||||
|
if (item.id !== other.id) {
|
||||||
|
totalSim += await brain.neural.similar(item.id, other.id) // ❌ N² operations
|
||||||
|
count++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
similarities.push({ id: item.id, avgSimilarity: totalSim / count })
|
||||||
|
}
|
||||||
|
|
||||||
|
// Manual outlier calculation
|
||||||
|
const threshold = calculateManualThreshold(similarities) // ❌ Complex statistics
|
||||||
|
return similarities.filter(s => s.avgSimilarity < threshold)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - AI-Powered Outlier Detection**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Automatic outlier detection
|
||||||
|
const outliers = await brain.neural.outliers({
|
||||||
|
threshold: 0.3, // Items with < 30% avg similarity to others
|
||||||
|
method: 'isolation-forest', // AI-based outlier detection
|
||||||
|
contamination: 0.1, // Expect ~10% outliers
|
||||||
|
includeReasons: true // Explain why each item is an outlier
|
||||||
|
})
|
||||||
|
|
||||||
|
console.log(`Found ${outliers.length} outliers`)
|
||||||
|
outliers.forEach(outlier => {
|
||||||
|
console.log(`Outlier: ${outlier.id}, Score: ${outlier.score}`)
|
||||||
|
console.log(`Reason: ${outlier.reason}`)
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Domain-specific outlier detection
|
||||||
|
const techOutliers = await brain.neural.outliers({
|
||||||
|
filter: { category: 'technology' },
|
||||||
|
features: ['content', 'metadata.tags', 'metadata.complexity'],
|
||||||
|
method: 'local-outlier-factor',
|
||||||
|
neighbors: 20 // Consider 20 nearest neighbors
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Temporal outlier detection
|
||||||
|
const recentOutliers = await brain.neural.outliers({
|
||||||
|
timeWindow: '7days', // Look at last 7 days
|
||||||
|
baseline: '30days', // Compare to 30-day baseline
|
||||||
|
method: 'statistical', // Use statistical methods
|
||||||
|
autoThreshold: true // Automatically determine threshold
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Streaming outlier detection
|
||||||
|
async function detectOutliersInStream() {
|
||||||
|
const outlierStream = brain.neural.outlierStream({
|
||||||
|
batchSize: 50,
|
||||||
|
updateInterval: 100, // Check every 100 new items
|
||||||
|
adaptiveThreshold: true // Threshold adapts as data changes
|
||||||
|
})
|
||||||
|
|
||||||
|
for await (const batch of outlierStream) {
|
||||||
|
console.log(`Batch ${batch.batchNumber}: ${batch.outliers.length} outliers detected`)
|
||||||
|
|
||||||
|
// Process outliers immediately
|
||||||
|
for (const outlier of batch.outliers) {
|
||||||
|
await handleOutlier(outlier)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleOutlier(outlier: any) {
|
||||||
|
// Flag for manual review
|
||||||
|
await brain.update(outlier.id, {
|
||||||
|
metadata: {
|
||||||
|
flagged: true,
|
||||||
|
outlierScore: outlier.score,
|
||||||
|
outlierReason: outlier.reason,
|
||||||
|
flaggedAt: Date.now()
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📊 Visualization Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Manual Visualization Data Preparation**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Manual coordinate calculation
|
||||||
|
async function prepareVisualizationManually() {
|
||||||
|
const items = await brain.find({ limit: 500 })
|
||||||
|
const coordinates = []
|
||||||
|
|
||||||
|
// Manual dimensionality reduction (complex math)
|
||||||
|
for (const item of items) {
|
||||||
|
const vector = await brain.embed(item.data)
|
||||||
|
// Complex PCA/t-SNE calculations manually
|
||||||
|
const x = complexMathFunction(vector) // ❌ Error-prone
|
||||||
|
const y = anotherComplexFunction(vector)
|
||||||
|
coordinates.push({ id: item.id, x, y })
|
||||||
|
}
|
||||||
|
|
||||||
|
return coordinates
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - AI-Powered Visualization**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Automatic 2D visualization
|
||||||
|
const visualization = await brain.neural.visualize({
|
||||||
|
dimensions: 2, // 2D plot
|
||||||
|
algorithm: 'umap', // UMAP for better clustering preservation
|
||||||
|
maxItems: 1000, // Performance limit
|
||||||
|
includeMetadata: true, // Include item metadata in output
|
||||||
|
colorBy: 'cluster' // Color points by cluster membership
|
||||||
|
})
|
||||||
|
|
||||||
|
// Result format:
|
||||||
|
// {
|
||||||
|
// points: [{ id, x, y, cluster, metadata }, ...],
|
||||||
|
// clusters: [{ id, centroid: [x, y], members: [...] }, ...],
|
||||||
|
// stats: { stress, kruskalStress, trustworthiness }
|
||||||
|
// }
|
||||||
|
|
||||||
|
// ✅ Pattern 2: 3D visualization for complex data
|
||||||
|
const viz3D = await brain.neural.visualize({
|
||||||
|
dimensions: 3,
|
||||||
|
algorithm: 'tsne',
|
||||||
|
perplexity: 30, // t-SNE parameter
|
||||||
|
learningRate: 200, // t-SNE learning rate
|
||||||
|
iterations: 1000 // Number of optimization steps
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Interactive visualization with filtering
|
||||||
|
const interactiveViz = await brain.neural.visualize({
|
||||||
|
filter: {
|
||||||
|
type: 'document',
|
||||||
|
createdAt: { $gte: Date.now() - 7 * 24 * 60 * 60 * 1000 }
|
||||||
|
},
|
||||||
|
groupBy: 'category', // Group points by metadata field
|
||||||
|
showLabels: true, // Include text labels
|
||||||
|
labelField: 'title', // Field to use for labels
|
||||||
|
includeEdges: true, // Show connections between similar items
|
||||||
|
edgeThreshold: 0.8 // Only show high-similarity connections
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Real-time visualization updates
|
||||||
|
class LiveVisualization {
|
||||||
|
private visualization: any = null
|
||||||
|
private updateInterval: NodeJS.Timeout | null = null
|
||||||
|
|
||||||
|
async start() {
|
||||||
|
// Initial visualization
|
||||||
|
this.visualization = await brain.neural.visualize({
|
||||||
|
dimensions: 2,
|
||||||
|
algorithm: 'umap',
|
||||||
|
maxItems: 500,
|
||||||
|
includeMetadata: true
|
||||||
|
})
|
||||||
|
|
||||||
|
// Update every 30 seconds
|
||||||
|
this.updateInterval = setInterval(async () => {
|
||||||
|
await this.update()
|
||||||
|
}, 30000)
|
||||||
|
}
|
||||||
|
|
||||||
|
async update() {
|
||||||
|
// Get recent items
|
||||||
|
const recentItems = await brain.find({
|
||||||
|
where: {
|
||||||
|
createdAt: { $gte: Date.now() - 30000 } // Last 30 seconds
|
||||||
|
},
|
||||||
|
limit: 50
|
||||||
|
})
|
||||||
|
|
||||||
|
if (recentItems.length > 0) {
|
||||||
|
// Incrementally update visualization
|
||||||
|
const updates = await brain.neural.updateVisualization(
|
||||||
|
this.visualization.id,
|
||||||
|
{
|
||||||
|
newItems: recentItems.map(item => item.id),
|
||||||
|
algorithm: 'incremental' // Faster incremental updates
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
this.visualization = { ...this.visualization, ...updates }
|
||||||
|
this.onUpdate(updates)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
onUpdate(updates: any) {
|
||||||
|
// Emit updates to frontend
|
||||||
|
console.log(`Visualization updated: ${updates.newPoints.length} new points`)
|
||||||
|
}
|
||||||
|
|
||||||
|
stop() {
|
||||||
|
if (this.updateInterval) {
|
||||||
|
clearInterval(this.updateInterval)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🚀 Performance Optimization Patterns
|
||||||
|
|
||||||
|
### ✅ **High-Performance Neural Operations**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Batch processing for similarity
|
||||||
|
async function batchSimilarityCalculation(itemPairs: Array<[string, string]>) {
|
||||||
|
const batchSize = 100
|
||||||
|
const results = []
|
||||||
|
|
||||||
|
for (let i = 0; i < itemPairs.length; i += batchSize) {
|
||||||
|
const batch = itemPairs.slice(i, i + batchSize)
|
||||||
|
|
||||||
|
const batchResults = await Promise.all(
|
||||||
|
batch.map(async ([a, b]) => ({
|
||||||
|
from: a,
|
||||||
|
to: b,
|
||||||
|
similarity: await brain.neural.similar(a, b)
|
||||||
|
}))
|
||||||
|
)
|
||||||
|
|
||||||
|
results.push(...batchResults)
|
||||||
|
|
||||||
|
// Progress reporting
|
||||||
|
console.log(`Processed ${Math.min(i + batchSize, itemPairs.length)}/${itemPairs.length} pairs`)
|
||||||
|
}
|
||||||
|
|
||||||
|
return results
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Caching expensive operations
|
||||||
|
class NeuralCache {
|
||||||
|
private clusterCache = new Map()
|
||||||
|
private similarityCache = new Map()
|
||||||
|
private readonly TTL = 5 * 60 * 1000 // 5 minutes
|
||||||
|
|
||||||
|
async getClusters(options: any) {
|
||||||
|
const key = JSON.stringify(options)
|
||||||
|
const cached = this.clusterCache.get(key)
|
||||||
|
|
||||||
|
if (cached && Date.now() - cached.timestamp < this.TTL) {
|
||||||
|
return cached.data
|
||||||
|
}
|
||||||
|
|
||||||
|
const clusters = await brain.neural.clusters(undefined, options)
|
||||||
|
this.clusterCache.set(key, {
|
||||||
|
data: clusters,
|
||||||
|
timestamp: Date.now()
|
||||||
|
})
|
||||||
|
|
||||||
|
return clusters
|
||||||
|
}
|
||||||
|
|
||||||
|
async getSimilarity(id1: string, id2: string) {
|
||||||
|
// Create consistent cache key regardless of order
|
||||||
|
const key = [id1, id2].sort().join('-')
|
||||||
|
const cached = this.similarityCache.get(key)
|
||||||
|
|
||||||
|
if (cached && Date.now() - cached.timestamp < this.TTL) {
|
||||||
|
return cached.data
|
||||||
|
}
|
||||||
|
|
||||||
|
const similarity = await brain.neural.similar(id1, id2)
|
||||||
|
this.similarityCache.set(key, {
|
||||||
|
data: similarity,
|
||||||
|
timestamp: Date.now()
|
||||||
|
})
|
||||||
|
|
||||||
|
return similarity
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Memory-efficient streaming
|
||||||
|
async function processLargeDatasetEfficiently() {
|
||||||
|
const stream = brain.neural.clusterStream({
|
||||||
|
batchSize: 50, // Small batches for memory efficiency
|
||||||
|
maxMemoryMB: 256, // Memory limit
|
||||||
|
diskCache: true, // Use disk for temporary storage
|
||||||
|
compression: true // Compress cached data
|
||||||
|
})
|
||||||
|
|
||||||
|
const results = []
|
||||||
|
let totalProcessed = 0
|
||||||
|
|
||||||
|
for await (const batch of stream) {
|
||||||
|
// Process batch immediately, don't accumulate in memory
|
||||||
|
const processedBatch = await processBatch(batch)
|
||||||
|
|
||||||
|
// Save to disk or send to another service
|
||||||
|
await saveBatchToDisk(processedBatch)
|
||||||
|
|
||||||
|
totalProcessed += batch.items.length
|
||||||
|
console.log(`Processed ${totalProcessed} items`)
|
||||||
|
|
||||||
|
// Clear memory
|
||||||
|
batch.items = null
|
||||||
|
}
|
||||||
|
|
||||||
|
return { totalProcessed }
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 4: Parallel processing with worker threads
|
||||||
|
async function parallelNeuralProcessing(items: string[]) {
|
||||||
|
const numWorkers = require('os').cpus().length
|
||||||
|
const batchSize = Math.ceil(items.length / numWorkers)
|
||||||
|
|
||||||
|
const workers = []
|
||||||
|
for (let i = 0; i < numWorkers; i++) {
|
||||||
|
const batch = items.slice(i * batchSize, (i + 1) * batchSize)
|
||||||
|
if (batch.length > 0) {
|
||||||
|
workers.push(processWorkerBatch(batch))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const results = await Promise.all(workers)
|
||||||
|
return results.flat()
|
||||||
|
}
|
||||||
|
|
||||||
|
async function processWorkerBatch(batch: string[]) {
|
||||||
|
// This would run in a worker thread in real implementation
|
||||||
|
return Promise.all(
|
||||||
|
batch.map(async itemId => {
|
||||||
|
const neighbors = await brain.neural.neighbors(itemId, { limit: 5 })
|
||||||
|
return { itemId, neighbors }
|
||||||
|
})
|
||||||
|
)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎯 Summary: Neural API Best Practices
|
||||||
|
|
||||||
|
| ❌ **Avoid These Patterns** | ✅ **Use These Instead** |
|
||||||
|
|---------------------------|------------------------|
|
||||||
|
| Manual similarity loops | `brain.neural.neighbors()` |
|
||||||
|
| Uncontrolled clustering | Limit items and set maxClusters |
|
||||||
|
| Manual outlier detection | `brain.neural.outliers()` |
|
||||||
|
| Manual visualization prep | `brain.neural.visualize()` |
|
||||||
|
| Loading entire datasets | Streaming and batch processing |
|
||||||
|
| No caching | Cache expensive operations |
|
||||||
|
| Blocking operations | Parallel and async patterns |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**🎉 Following these patterns gives you:**
|
||||||
|
- 🚀 **Optimized performance** with intelligent algorithms
|
||||||
|
- 🧠 **AI-powered insights** instead of manual statistics
|
||||||
|
- 📊 **Rich visualizations** for data exploration
|
||||||
|
- 🎯 **Accurate clustering** with semantic understanding
|
||||||
|
- 🚨 **Smart outlier detection** for quality control
|
||||||
|
- ⚡ **Scalable processing** for large datasets
|
||||||
|
|
||||||
|
**Next:** [Augmentation Patterns →](./AUGMENTATION_PATTERNS.md) | [Core API Patterns →](./CORE_API_PATTERNS.md)
|
||||||
582
docs/vfs/COMMON_PATTERNS.md
Normal file
582
docs/vfs/COMMON_PATTERNS.md
Normal file
|
|
@ -0,0 +1,582 @@
|
||||||
|
# 🎯 VFS Common Patterns: Do This, Not That
|
||||||
|
|
||||||
|
> Learn the correct patterns for using Brainy VFS. Avoid the mistakes that cause crashes, poor performance, and API confusion.
|
||||||
|
|
||||||
|
## 🚨 Critical Pattern: Safe Tree Operations
|
||||||
|
|
||||||
|
### ❌ **WRONG - Causes Infinite Recursion**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Directory appears as its own child!
|
||||||
|
function buildFileTree(allItems, parentPath) {
|
||||||
|
return allItems.filter(item => {
|
||||||
|
// This includes the parent directory itself!
|
||||||
|
return item.path.startsWith(parentPath)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// Result: /dir -> /dir -> /dir -> ∞ (crashes browser/server)
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Tree-Aware Methods**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Direct children for UI trees
|
||||||
|
async function loadDirectoryUI(path: string) {
|
||||||
|
const children = await vfs.getDirectChildren(path)
|
||||||
|
|
||||||
|
// Guaranteed: No self-inclusion, no recursion
|
||||||
|
return children.map(child => ({
|
||||||
|
name: child.metadata.name,
|
||||||
|
path: child.metadata.path,
|
||||||
|
type: child.metadata.vfsType,
|
||||||
|
hasChildren: child.metadata.vfsType === 'directory'
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Complete tree structure
|
||||||
|
async function buildCompleteTree(path: string) {
|
||||||
|
return await vfs.getTreeStructure(path, {
|
||||||
|
maxDepth: 5, // Prevent deep recursion
|
||||||
|
includeHidden: false, // Skip .hidden files
|
||||||
|
sort: 'name' // Organized output
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Detailed inspection
|
||||||
|
async function inspectPath(path: string) {
|
||||||
|
const info = await vfs.inspect(path)
|
||||||
|
return {
|
||||||
|
current: info.node,
|
||||||
|
children: info.children, // Direct children only
|
||||||
|
parent: info.parent, // Parent directory
|
||||||
|
stats: info.stats // Size, permissions, etc.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🗃️ Storage Configuration Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Memory Storage for Files**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Data disappears when process exits!
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: { type: 'memory' } // ❌ Temporary only
|
||||||
|
})
|
||||||
|
|
||||||
|
// Files written here are lost forever on restart
|
||||||
|
await vfs.writeFile('/important.doc', content)
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Persistent Storage**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Filesystem storage (development)
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: {
|
||||||
|
type: 'filesystem',
|
||||||
|
path: './brainy-data' // Persisted to disk
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Cloud storage (production)
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: {
|
||||||
|
type: 's3',
|
||||||
|
bucket: 'my-vfs-data',
|
||||||
|
region: 'us-west-2'
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Auto-detection (recommended)
|
||||||
|
const brain = new Brainy() // Automatically chooses best storage
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔍 Search Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Manual String Matching**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Missing semantic understanding
|
||||||
|
async function findFilesOldWay(query: string) {
|
||||||
|
const allFiles = await vfs.readdir('/', { recursive: true })
|
||||||
|
return allFiles.filter(file =>
|
||||||
|
file.includes(query.toLowerCase()) // ❌ Basic string match
|
||||||
|
)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Semantic Search**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Content-aware search
|
||||||
|
async function findFilesByContent(query: string) {
|
||||||
|
return await vfs.search(query, {
|
||||||
|
type: 'file', // Only search files
|
||||||
|
limit: 50, // Reasonable limit
|
||||||
|
threshold: 0.7 // Minimum relevance
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Filtered search
|
||||||
|
async function findInDirectory(query: string, basePath: string) {
|
||||||
|
return await vfs.search(query, {
|
||||||
|
path: basePath, // Limit to specific directory
|
||||||
|
includeContent: true, // Include file content in results
|
||||||
|
sort: 'relevance' // Best matches first
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Metadata-based search
|
||||||
|
async function findByAttributes(criteria: any) {
|
||||||
|
return await vfs.find({
|
||||||
|
where: criteria, // MongoDB-style queries
|
||||||
|
orderBy: 'modified', // Sort by last modified
|
||||||
|
limit: 100
|
||||||
|
})
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🏗️ Initialization Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Race Conditions**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Not waiting for initialization
|
||||||
|
const brain = new Brainy()
|
||||||
|
const vfs = brain.vfs()
|
||||||
|
|
||||||
|
// ❌ Using VFS before it's ready
|
||||||
|
await vfs.writeFile('/file.txt', 'content') // May fail!
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Proper Initialization**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Sequential initialization
|
||||||
|
async function initializeVFS() {
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: { type: 'filesystem', path: './data' }
|
||||||
|
})
|
||||||
|
|
||||||
|
// Wait for brain to be ready
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
// Then initialize VFS
|
||||||
|
const vfs = brain.vfs()
|
||||||
|
await vfs.init()
|
||||||
|
|
||||||
|
// Now safe to use
|
||||||
|
return vfs
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: With error handling
|
||||||
|
async function robustVFSInit() {
|
||||||
|
try {
|
||||||
|
const brain = new Brainy()
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
const vfs = brain.vfs()
|
||||||
|
await vfs.init()
|
||||||
|
|
||||||
|
// Verify it's working
|
||||||
|
await vfs.stat('/') // Should not throw
|
||||||
|
|
||||||
|
return vfs
|
||||||
|
} catch (error) {
|
||||||
|
console.error('VFS initialization failed:', error)
|
||||||
|
throw new Error(`Cannot initialize VFS: ${error.message}`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Singleton pattern for apps
|
||||||
|
class VFSManager {
|
||||||
|
private static instance: any = null
|
||||||
|
|
||||||
|
static async getInstance() {
|
||||||
|
if (!this.instance) {
|
||||||
|
const brain = new Brainy()
|
||||||
|
await brain.init()
|
||||||
|
this.instance = brain.vfs()
|
||||||
|
await this.instance.init()
|
||||||
|
}
|
||||||
|
return this.instance
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📝 File Operation Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Blocking Operations**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Blocking the main thread
|
||||||
|
async function badFileProcessing(files: string[]) {
|
||||||
|
for (const file of files) {
|
||||||
|
const content = await vfs.readFile(file) // ❌ Sequential
|
||||||
|
await processContent(content) // ❌ Blocking
|
||||||
|
await vfs.writeFile(file + '.processed', result)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Efficient Operations**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Parallel processing
|
||||||
|
async function efficientProcessing(files: string[]) {
|
||||||
|
const operations = files.map(async (file) => {
|
||||||
|
try {
|
||||||
|
const content = await vfs.readFile(file)
|
||||||
|
const result = await processContent(content)
|
||||||
|
await vfs.writeFile(file + '.processed', result)
|
||||||
|
return { file, success: true }
|
||||||
|
} catch (error) {
|
||||||
|
return { file, success: false, error: error.message }
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
return await Promise.allSettled(operations)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Batch operations with limits
|
||||||
|
async function batchProcessFiles(files: string[], batchSize = 10) {
|
||||||
|
const results = []
|
||||||
|
|
||||||
|
for (let i = 0; i < files.length; i += batchSize) {
|
||||||
|
const batch = files.slice(i, i + batchSize)
|
||||||
|
const batchResults = await Promise.all(
|
||||||
|
batch.map(file => processFile(file))
|
||||||
|
)
|
||||||
|
results.push(...batchResults)
|
||||||
|
|
||||||
|
// Optional: Add delay between batches
|
||||||
|
if (i + batchSize < files.length) {
|
||||||
|
await new Promise(resolve => setTimeout(resolve, 100))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return results
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Streaming for large files
|
||||||
|
async function streamLargeFile(filePath: string) {
|
||||||
|
const stream = await vfs.createReadStream(filePath, {
|
||||||
|
highWaterMark: 64 * 1024 // 64KB chunks
|
||||||
|
})
|
||||||
|
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
let content = ''
|
||||||
|
|
||||||
|
stream.on('data', (chunk) => {
|
||||||
|
content += chunk.toString()
|
||||||
|
})
|
||||||
|
|
||||||
|
stream.on('end', () => resolve(content))
|
||||||
|
stream.on('error', reject)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔗 Relationship Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Manual Relationship Tracking**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Reinventing the graph
|
||||||
|
const fileRelationships = new Map() // ❌ Manual tracking
|
||||||
|
|
||||||
|
function linkFiles(sourceFile: string, targetFile: string, relationship: string) {
|
||||||
|
if (!fileRelationships.has(sourceFile)) {
|
||||||
|
fileRelationships.set(sourceFile, [])
|
||||||
|
}
|
||||||
|
fileRelationships.get(sourceFile).push({ target: targetFile, type: relationship })
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Use Built-in Relationships**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Semantic relationships
|
||||||
|
async function createFileRelationships() {
|
||||||
|
// Link test to source file
|
||||||
|
await vfs.addRelationship(
|
||||||
|
'/src/auth.ts',
|
||||||
|
'/tests/auth.test.ts',
|
||||||
|
'tested-by'
|
||||||
|
)
|
||||||
|
|
||||||
|
// Link documentation to implementation
|
||||||
|
await vfs.addRelationship(
|
||||||
|
'/docs/api.md',
|
||||||
|
'/src/api.ts',
|
||||||
|
'documents'
|
||||||
|
)
|
||||||
|
|
||||||
|
// Link dependency relationships
|
||||||
|
await vfs.addRelationship(
|
||||||
|
'/src/index.ts',
|
||||||
|
'/src/utils.ts',
|
||||||
|
'imports'
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Query relationships
|
||||||
|
async function findRelatedFiles(filePath: string) {
|
||||||
|
// Find all files related to this one
|
||||||
|
const related = await vfs.getRelated(filePath, {
|
||||||
|
depth: 2, // Include relationships of relationships
|
||||||
|
types: ['tests', 'documents', 'imports'], // Filter relationship types
|
||||||
|
direction: 'both' // Both incoming and outgoing
|
||||||
|
})
|
||||||
|
|
||||||
|
return related
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Relationship-based search
|
||||||
|
async function findTestFiles(sourceFile: string) {
|
||||||
|
return await vfs.search('', {
|
||||||
|
connected: {
|
||||||
|
to: sourceFile,
|
||||||
|
via: 'tested-by',
|
||||||
|
direction: 'incoming'
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🚀 Performance Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Loading Everything**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Loading massive directories
|
||||||
|
async function loadEntireProject() {
|
||||||
|
const allFiles = await vfs.getTreeStructure('/', {
|
||||||
|
// ❌ No limits, could be millions of files
|
||||||
|
})
|
||||||
|
|
||||||
|
return allFiles // ❌ Crashes on large projects
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Smart Loading**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Paginated loading
|
||||||
|
async function loadDirectoryPage(path: string, page = 0, size = 50) {
|
||||||
|
const children = await vfs.getDirectChildren(path, {
|
||||||
|
limit: size,
|
||||||
|
offset: page * size,
|
||||||
|
sort: 'name'
|
||||||
|
})
|
||||||
|
|
||||||
|
const total = await vfs.getChildrenCount(path)
|
||||||
|
|
||||||
|
return {
|
||||||
|
items: children,
|
||||||
|
page,
|
||||||
|
size,
|
||||||
|
total,
|
||||||
|
hasMore: (page + 1) * size < total
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Lazy loading with caching
|
||||||
|
class FileTreeCache {
|
||||||
|
private cache = new Map()
|
||||||
|
|
||||||
|
async getDirectory(path: string) {
|
||||||
|
if (this.cache.has(path)) {
|
||||||
|
return this.cache.get(path)
|
||||||
|
}
|
||||||
|
|
||||||
|
const children = await vfs.getDirectChildren(path)
|
||||||
|
this.cache.set(path, children)
|
||||||
|
|
||||||
|
// Auto-expire cache after 5 minutes
|
||||||
|
setTimeout(() => this.cache.delete(path), 5 * 60 * 1000)
|
||||||
|
|
||||||
|
return children
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Progressive disclosure
|
||||||
|
async function buildLazyTree(rootPath: string) {
|
||||||
|
const tree = await vfs.getTreeStructure(rootPath, {
|
||||||
|
maxDepth: 1, // Only immediate children
|
||||||
|
lazy: true // Enable lazy loading for subdirectories
|
||||||
|
})
|
||||||
|
|
||||||
|
// Expand directories on demand
|
||||||
|
tree.expandDirectory = async (path: string) => {
|
||||||
|
const subtree = await vfs.getTreeStructure(path, {
|
||||||
|
maxDepth: 1
|
||||||
|
})
|
||||||
|
return subtree.children
|
||||||
|
}
|
||||||
|
|
||||||
|
return tree
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔧 Error Handling Patterns
|
||||||
|
|
||||||
|
### ❌ **WRONG - Silent Failures**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Ignoring errors
|
||||||
|
async function badErrorHandling(path: string) {
|
||||||
|
try {
|
||||||
|
return await vfs.readFile(path)
|
||||||
|
} catch (error) {
|
||||||
|
return null // ❌ Silent failure
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **CORRECT - Robust Error Handling**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Pattern 1: Specific error handling
|
||||||
|
async function robustFileRead(path: string) {
|
||||||
|
try {
|
||||||
|
return await vfs.readFile(path)
|
||||||
|
} catch (error) {
|
||||||
|
if (error.code === 'ENOENT') {
|
||||||
|
throw new Error(`File not found: ${path}`)
|
||||||
|
} else if (error.code === 'EACCES') {
|
||||||
|
throw new Error(`Permission denied: ${path}`)
|
||||||
|
} else if (error.code === 'EISDIR') {
|
||||||
|
throw new Error(`Path is a directory, not a file: ${path}`)
|
||||||
|
} else {
|
||||||
|
throw new Error(`Failed to read ${path}: ${error.message}`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 2: Retry with backoff
|
||||||
|
async function resilientOperation(operation: () => Promise<any>, maxRetries = 3) {
|
||||||
|
for (let attempt = 1; attempt <= maxRetries; attempt++) {
|
||||||
|
try {
|
||||||
|
return await operation()
|
||||||
|
} catch (error) {
|
||||||
|
if (attempt === maxRetries) {
|
||||||
|
throw error
|
||||||
|
}
|
||||||
|
|
||||||
|
// Exponential backoff
|
||||||
|
const delay = Math.pow(2, attempt) * 1000
|
||||||
|
await new Promise(resolve => setTimeout(resolve, delay))
|
||||||
|
|
||||||
|
console.warn(`Attempt ${attempt} failed, retrying in ${delay}ms...`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Pattern 3: Graceful degradation
|
||||||
|
async function gracefulFileExplorer(path: string) {
|
||||||
|
try {
|
||||||
|
// Try the optimal method first
|
||||||
|
return await vfs.getDirectChildren(path)
|
||||||
|
} catch (error) {
|
||||||
|
console.warn('Direct children failed, trying basic readdir:', error.message)
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Fallback to basic directory listing
|
||||||
|
const entries = await vfs.readdir(path)
|
||||||
|
return entries.map(name => ({
|
||||||
|
metadata: { name, path: `${path}/${name}` },
|
||||||
|
// Missing detailed metadata, but functional
|
||||||
|
}))
|
||||||
|
} catch (fallbackError) {
|
||||||
|
console.error('All methods failed:', fallbackError.message)
|
||||||
|
return [] // Empty result rather than crash
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📚 Integration Patterns
|
||||||
|
|
||||||
|
### ✅ **React Hook Pattern**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
function useVFS() {
|
||||||
|
const [vfs, setVFS] = useState<any>(null)
|
||||||
|
const [loading, setLoading] = useState(true)
|
||||||
|
const [error, setError] = useState<string | null>(null)
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
async function initVFS() {
|
||||||
|
try {
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: { type: 'filesystem', path: './brainy-data' }
|
||||||
|
})
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
const vfsInstance = brain.vfs()
|
||||||
|
await vfsInstance.init()
|
||||||
|
|
||||||
|
setVFS(vfsInstance)
|
||||||
|
} catch (err) {
|
||||||
|
setError(err.message)
|
||||||
|
} finally {
|
||||||
|
setLoading(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
initVFS()
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
return { vfs, loading, error }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ **Express.js Middleware Pattern**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
function vfsMiddleware() {
|
||||||
|
let vfsInstance: any = null
|
||||||
|
|
||||||
|
return async (req: any, res: any, next: any) => {
|
||||||
|
if (!vfsInstance) {
|
||||||
|
try {
|
||||||
|
const brain = new Brainy()
|
||||||
|
await brain.init()
|
||||||
|
vfsInstance = brain.vfs()
|
||||||
|
await vfsInstance.init()
|
||||||
|
} catch (error) {
|
||||||
|
return res.status(500).json({ error: 'VFS initialization failed' })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
req.vfs = vfsInstance
|
||||||
|
next()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎯 Summary: Do This, Not That
|
||||||
|
|
||||||
|
| ❌ **Avoid These Patterns** | ✅ **Use These Instead** |
|
||||||
|
|---------------------------|------------------------|
|
||||||
|
| Manual tree filtering | `vfs.getDirectChildren()` |
|
||||||
|
| Memory storage for files | Filesystem or cloud storage |
|
||||||
|
| Sequential file operations | Parallel processing with limits |
|
||||||
|
| Manual relationship tracking | Built-in `vfs.addRelationship()` |
|
||||||
|
| Loading entire directories | Paginated/lazy loading |
|
||||||
|
| Silent error handling | Specific error types and fallbacks |
|
||||||
|
| Blocking synchronous calls | Async/await with proper error handling |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**🎉 Following these patterns will give you:**
|
||||||
|
- 🚫 **Zero infinite recursion** in file explorers
|
||||||
|
- ⚡ **Fast performance** even with large directories
|
||||||
|
- 🔄 **Reliable error recovery** and graceful degradation
|
||||||
|
- 🧠 **Semantic intelligence** for powerful file search
|
||||||
|
- 📈 **Scalable architecture** that grows with your needs
|
||||||
|
|
||||||
|
**Next Steps:** Check out the [VFS API Guide](./VFS_API_GUIDE.md) for complete method documentation.
|
||||||
332
docs/vfs/QUICK_START.md
Normal file
332
docs/vfs/QUICK_START.md
Normal file
|
|
@ -0,0 +1,332 @@
|
||||||
|
# 🚀 VFS Quick Start: 5-Minute File Explorer Setup
|
||||||
|
|
||||||
|
> Get a working, production-ready file explorer with Brainy VFS in 5 minutes. Avoid common pitfalls and use the correct APIs.
|
||||||
|
|
||||||
|
## 📋 What You'll Build
|
||||||
|
|
||||||
|
A file explorer that:
|
||||||
|
- ✅ **Never crashes** from infinite recursion
|
||||||
|
- ✅ **Uses filesystem storage** correctly
|
||||||
|
- ✅ **Leverages semantic search** to find files by content
|
||||||
|
- ✅ **Handles large directories** efficiently
|
||||||
|
- ✅ **Follows modern Brainy v3.x APIs**
|
||||||
|
|
||||||
|
## ⚡ Step 1: Basic Setup (1 minute)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm install @soulcraft/brainy
|
||||||
|
```
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { Brainy } from '@soulcraft/brainy'
|
||||||
|
|
||||||
|
// ✅ CORRECT: Use filesystem storage for production
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: {
|
||||||
|
type: 'filesystem',
|
||||||
|
path: './brainy-data' // Your data directory
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
// ✅ CORRECT: Initialize VFS
|
||||||
|
const vfs = brain.vfs()
|
||||||
|
await vfs.init()
|
||||||
|
|
||||||
|
console.log('🎉 VFS ready!')
|
||||||
|
```
|
||||||
|
|
||||||
|
> **🚨 Common Mistake**: Don't use `storage: { type: 'memory' }` for file explorers - your data will disappear when the process exits!
|
||||||
|
|
||||||
|
## 📁 Step 2: Safe Directory Listing (2 minutes)
|
||||||
|
|
||||||
|
**❌ WRONG - This causes infinite recursion:**
|
||||||
|
```typescript
|
||||||
|
// DON'T DO THIS - Causes directory to appear as its own child!
|
||||||
|
const badItems = allNodes.filter(node => node.path.startsWith(dirPath))
|
||||||
|
```
|
||||||
|
|
||||||
|
**✅ CORRECT - Use tree-aware methods:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Method 1: Get direct children (recommended for UI)
|
||||||
|
async function loadDirectoryContents(path: string) {
|
||||||
|
try {
|
||||||
|
const children = await vfs.getDirectChildren(path)
|
||||||
|
|
||||||
|
// Sort directories first, then files
|
||||||
|
return children.sort((a, b) => {
|
||||||
|
if (a.metadata.vfsType === 'directory' && b.metadata.vfsType === 'file') return -1
|
||||||
|
if (a.metadata.vfsType === 'file' && b.metadata.vfsType === 'directory') return 1
|
||||||
|
return a.metadata.name.localeCompare(b.metadata.name)
|
||||||
|
})
|
||||||
|
} catch (error) {
|
||||||
|
console.error(`Failed to load ${path}:`, error.message)
|
||||||
|
return []
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Method 2: Get complete tree structure (for full trees)
|
||||||
|
async function loadFullTree(path: string) {
|
||||||
|
const tree = await vfs.getTreeStructure(path, {
|
||||||
|
maxDepth: 3, // Prevent deep recursion
|
||||||
|
includeHidden: false, // Skip hidden files
|
||||||
|
sort: 'name'
|
||||||
|
})
|
||||||
|
return tree
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Method 3: Get detailed path info
|
||||||
|
async function inspectPath(path: string) {
|
||||||
|
const info = await vfs.inspect(path)
|
||||||
|
return {
|
||||||
|
isDirectory: info.node.metadata.vfsType === 'directory',
|
||||||
|
children: info.children,
|
||||||
|
parent: info.parent,
|
||||||
|
stats: info.stats
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔍 Step 3: Add Semantic Search (1 minute)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Find files by content, not just filename
|
||||||
|
async function searchFiles(query: string, basePath: string = '/') {
|
||||||
|
const results = await vfs.search(query, {
|
||||||
|
path: basePath, // Limit search to specific directory
|
||||||
|
limit: 50, // Reasonable limit
|
||||||
|
type: 'file' // Only search files, not directories
|
||||||
|
})
|
||||||
|
|
||||||
|
return results.map(result => ({
|
||||||
|
path: result.path,
|
||||||
|
score: result.score,
|
||||||
|
type: result.type,
|
||||||
|
size: result.size,
|
||||||
|
modified: result.modified
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Example usage
|
||||||
|
const reactFiles = await searchFiles('React components with hooks', '/src')
|
||||||
|
const docs = await searchFiles('API documentation', '/docs')
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🖥️ Step 4: Complete File Explorer Component (1 minute)
|
||||||
|
|
||||||
|
Here's a complete React component using the correct patterns:
|
||||||
|
|
||||||
|
```tsx
|
||||||
|
import React, { useState, useEffect } from 'react'
|
||||||
|
import { Brainy } from '@soulcraft/brainy'
|
||||||
|
|
||||||
|
export function FileExplorer() {
|
||||||
|
const [vfs, setVfs] = useState(null)
|
||||||
|
const [currentPath, setCurrentPath] = useState('/')
|
||||||
|
const [items, setItems] = useState([])
|
||||||
|
const [loading, setLoading] = useState(true)
|
||||||
|
const [searchQuery, setSearchQuery] = useState('')
|
||||||
|
|
||||||
|
// Initialize VFS
|
||||||
|
useEffect(() => {
|
||||||
|
async function initVFS() {
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: { type: 'filesystem', path: './brainy-data' }
|
||||||
|
})
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
const vfsInstance = brain.vfs()
|
||||||
|
await vfsInstance.init()
|
||||||
|
|
||||||
|
setVfs(vfsInstance)
|
||||||
|
setLoading(false)
|
||||||
|
}
|
||||||
|
initVFS()
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
// Load directory contents
|
||||||
|
const loadDirectory = async (path: string) => {
|
||||||
|
if (!vfs) return
|
||||||
|
|
||||||
|
setLoading(true)
|
||||||
|
try {
|
||||||
|
// ✅ CORRECT: Use getDirectChildren to prevent recursion
|
||||||
|
const children = await vfs.getDirectChildren(path)
|
||||||
|
|
||||||
|
// Sort directories first
|
||||||
|
const sorted = children.sort((a, b) => {
|
||||||
|
if (a.metadata.vfsType === 'directory' && b.metadata.vfsType === 'file') return -1
|
||||||
|
if (a.metadata.vfsType === 'file' && b.metadata.vfsType === 'directory') return 1
|
||||||
|
return a.metadata.name.localeCompare(b.metadata.name)
|
||||||
|
})
|
||||||
|
|
||||||
|
setItems(sorted)
|
||||||
|
setCurrentPath(path)
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Failed to load directory:', error)
|
||||||
|
setItems([])
|
||||||
|
} finally {
|
||||||
|
setLoading(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Search files
|
||||||
|
const handleSearch = async () => {
|
||||||
|
if (!vfs || !searchQuery.trim()) {
|
||||||
|
loadDirectory(currentPath)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
setLoading(true)
|
||||||
|
try {
|
||||||
|
const results = await vfs.search(searchQuery, {
|
||||||
|
path: currentPath,
|
||||||
|
limit: 100
|
||||||
|
})
|
||||||
|
setItems(results)
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Search failed:', error)
|
||||||
|
} finally {
|
||||||
|
setLoading(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Initial load
|
||||||
|
useEffect(() => {
|
||||||
|
if (vfs) {
|
||||||
|
loadDirectory('/')
|
||||||
|
}
|
||||||
|
}, [vfs])
|
||||||
|
|
||||||
|
if (loading && !vfs) {
|
||||||
|
return <div>Initializing VFS...</div>
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="file-explorer">
|
||||||
|
{/* Search bar */}
|
||||||
|
<div className="search-bar">
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
value={searchQuery}
|
||||||
|
onChange={(e) => setSearchQuery(e.target.value)}
|
||||||
|
placeholder="Search files by content..."
|
||||||
|
onKeyPress={(e) => e.key === 'Enter' && handleSearch()}
|
||||||
|
/>
|
||||||
|
<button onClick={handleSearch}>Search</button>
|
||||||
|
{searchQuery && (
|
||||||
|
<button onClick={() => { setSearchQuery(''); loadDirectory(currentPath) }}>
|
||||||
|
Clear
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{/* Current path */}
|
||||||
|
<div className="current-path">
|
||||||
|
📁 {currentPath}
|
||||||
|
{currentPath !== '/' && (
|
||||||
|
<button onClick={() => loadDirectory(currentPath.split('/').slice(0, -1).join('/') || '/')}>
|
||||||
|
⬆️ Up
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{/* File list */}
|
||||||
|
{loading ? (
|
||||||
|
<div>Loading...</div>
|
||||||
|
) : (
|
||||||
|
<div className="file-list">
|
||||||
|
{items.map((item) => (
|
||||||
|
<div
|
||||||
|
key={item.id}
|
||||||
|
className={`file-item ${item.metadata.vfsType}`}
|
||||||
|
onClick={() => {
|
||||||
|
if (item.metadata.vfsType === 'directory') {
|
||||||
|
loadDirectory(item.metadata.path)
|
||||||
|
} else {
|
||||||
|
console.log('Open file:', item.metadata.path)
|
||||||
|
// Add your file opening logic here
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<span className="icon">
|
||||||
|
{item.metadata.vfsType === 'directory' ? '📁' : '📄'}
|
||||||
|
</span>
|
||||||
|
<span className="name">{item.metadata.name}</span>
|
||||||
|
<span className="size">
|
||||||
|
{item.metadata.size ? `${Math.round(item.metadata.size / 1024)}KB` : ''}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
{items.length === 0 && (
|
||||||
|
<div className="empty">
|
||||||
|
{searchQuery ? 'No results found' : 'Empty directory'}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎯 What We Just Avoided
|
||||||
|
|
||||||
|
By using this quick start, you avoided these common mistakes:
|
||||||
|
|
||||||
|
❌ **Infinite Recursion**: Using naive filtering that includes directories as their own children
|
||||||
|
❌ **Memory Storage**: Losing data when process restarts
|
||||||
|
❌ **Old APIs**: Using deprecated `addNoun`, `getNouns`, `addVerb` methods
|
||||||
|
❌ **Complex Fallbacks**: Implementing unnecessary fallback patterns when proper methods exist
|
||||||
|
❌ **Poor Performance**: Not using tree-aware methods designed for file explorers
|
||||||
|
|
||||||
|
## 🚀 Next Steps
|
||||||
|
|
||||||
|
Your file explorer is now working! Here's what to explore next:
|
||||||
|
|
||||||
|
1. **[File Operations](./VFS_API_GUIDE.md#file-operations)** - Read, write, and manipulate files
|
||||||
|
2. **[Semantic Features](./VFS_KNOWLEDGE_LAYER.md)** - Connect files to concepts and entities
|
||||||
|
3. **[Performance Optimization](./building-file-explorers.md#performance)** - Handle large directories efficiently
|
||||||
|
4. **[Advanced Search](./VFS_API_GUIDE.md#search-operations)** - Complex queries and filters
|
||||||
|
|
||||||
|
## 🆘 Common Issues & Solutions
|
||||||
|
|
||||||
|
### "Module not found" errors
|
||||||
|
```bash
|
||||||
|
# Make sure you're using the right import
|
||||||
|
npm ls @soulcraft/brainy # Check version
|
||||||
|
npm install @soulcraft/brainy@latest # Update if needed
|
||||||
|
```
|
||||||
|
|
||||||
|
### "VFS not initialized" errors
|
||||||
|
```typescript
|
||||||
|
// Always await both init() calls
|
||||||
|
await brain.init()
|
||||||
|
const vfs = brain.vfs()
|
||||||
|
await vfs.init() // Don't forget this!
|
||||||
|
```
|
||||||
|
|
||||||
|
### Slow directory loading
|
||||||
|
```typescript
|
||||||
|
// Add pagination for large directories
|
||||||
|
const children = await vfs.getDirectChildren(path, {
|
||||||
|
limit: 100, // Load only first 100 items
|
||||||
|
offset: 0 // Start from beginning
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
### Search not finding files
|
||||||
|
```typescript
|
||||||
|
// Make sure files are imported into VFS first
|
||||||
|
await vfs.importDirectory('./my-files', {
|
||||||
|
recursive: true,
|
||||||
|
extractMetadata: true // Enable content understanding
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**🎉 Congratulations!** You now have a working file explorer that uses modern Brainy APIs correctly. No more infinite recursion, no more deprecated methods, no more confusion.
|
||||||
|
|
||||||
|
**Need help?** Check out our [Complete VFS Guide](./VFS_API_GUIDE.md) or [Common Patterns](./COMMON_PATTERNS.md).
|
||||||
570
src/brainy.ts
570
src/brainy.ts
|
|
@ -227,6 +227,50 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Add an entity to the database
|
* Add an entity to the database
|
||||||
|
*
|
||||||
|
* @param params - Parameters for adding the entity
|
||||||
|
* @returns Promise that resolves to the entity ID
|
||||||
|
*
|
||||||
|
* @example Basic entity creation
|
||||||
|
* ```typescript
|
||||||
|
* const id = await brain.add({
|
||||||
|
* data: "John Smith is a software engineer",
|
||||||
|
* type: NounType.Person,
|
||||||
|
* metadata: { role: "engineer", team: "backend" }
|
||||||
|
* })
|
||||||
|
* console.log(`Created entity: ${id}`)
|
||||||
|
* ```
|
||||||
|
*
|
||||||
|
* @example Adding with custom ID
|
||||||
|
* ```typescript
|
||||||
|
* const customId = await brain.add({
|
||||||
|
* id: "user-12345",
|
||||||
|
* data: "Important document content",
|
||||||
|
* type: NounType.Document,
|
||||||
|
* metadata: { priority: "high", department: "legal" }
|
||||||
|
* })
|
||||||
|
* ```
|
||||||
|
*
|
||||||
|
* @example Using pre-computed vector (optimization)
|
||||||
|
* ```typescript
|
||||||
|
* const vector = await brain.embed("Optimized content")
|
||||||
|
* const id = await brain.add({
|
||||||
|
* data: "Optimized content",
|
||||||
|
* type: NounType.Document,
|
||||||
|
* vector: vector, // Skip re-embedding
|
||||||
|
* metadata: { optimized: true }
|
||||||
|
* })
|
||||||
|
* ```
|
||||||
|
*
|
||||||
|
* @example Multi-tenant usage
|
||||||
|
* ```typescript
|
||||||
|
* const id = await brain.add({
|
||||||
|
* data: "Customer feedback",
|
||||||
|
* type: NounType.Message,
|
||||||
|
* service: "customer-portal", // Multi-tenancy
|
||||||
|
* metadata: { rating: 5, verified: true }
|
||||||
|
* })
|
||||||
|
* ```
|
||||||
*/
|
*/
|
||||||
async add(params: AddParams<T>): Promise<string> {
|
async add(params: AddParams<T>): Promise<string> {
|
||||||
await this.ensureInitialized()
|
await this.ensureInitialized()
|
||||||
|
|
@ -283,6 +327,74 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Get an entity by ID
|
* Get an entity by ID
|
||||||
|
*
|
||||||
|
* @param id - The unique identifier of the entity to retrieve
|
||||||
|
* @returns Promise that resolves to the entity if found, null if not found
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Basic entity retrieval
|
||||||
|
* const entity = await brainy.get('user-123')
|
||||||
|
* if (entity) {
|
||||||
|
* console.log('Found entity:', entity.data)
|
||||||
|
* console.log('Created at:', new Date(entity.createdAt))
|
||||||
|
* } else {
|
||||||
|
* console.log('Entity not found')
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Working with typed entities
|
||||||
|
* interface User {
|
||||||
|
* name: string
|
||||||
|
* email: string
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* const brainy = new Brainy<User>({ storage: 'filesystem' })
|
||||||
|
* const user = await brainy.get('user-456')
|
||||||
|
* if (user) {
|
||||||
|
* // TypeScript knows user.metadata is of type User
|
||||||
|
* console.log(`Hello ${user.metadata.name}`)
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Safe retrieval with error handling
|
||||||
|
* try {
|
||||||
|
* const entity = await brainy.get('document-789')
|
||||||
|
* if (!entity) {
|
||||||
|
* throw new Error('Document not found')
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* // Process the entity
|
||||||
|
* return {
|
||||||
|
* id: entity.id,
|
||||||
|
* content: entity.data,
|
||||||
|
* type: entity.type,
|
||||||
|
* metadata: entity.metadata
|
||||||
|
* }
|
||||||
|
* } catch (error) {
|
||||||
|
* console.error('Failed to retrieve entity:', error)
|
||||||
|
* return null
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Batch retrieval pattern
|
||||||
|
* const ids = ['doc-1', 'doc-2', 'doc-3']
|
||||||
|
* const entities = await Promise.all(
|
||||||
|
* ids.map(id => brainy.get(id))
|
||||||
|
* )
|
||||||
|
* const foundEntities = entities.filter(entity => entity !== null)
|
||||||
|
* console.log(`Found ${foundEntities.length} out of ${ids.length} entities`)
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Using with async iteration
|
||||||
|
* const entityIds = ['user-1', 'user-2', 'user-3']
|
||||||
|
*
|
||||||
|
* for (const id of entityIds) {
|
||||||
|
* const entity = await brainy.get(id)
|
||||||
|
* if (entity) {
|
||||||
|
* console.log(`Processing ${entity.type}: ${id}`)
|
||||||
|
* // Process entity...
|
||||||
|
* }
|
||||||
|
* }
|
||||||
*/
|
*/
|
||||||
async get(id: string): Promise<Entity<T> | null> {
|
async get(id: string): Promise<Entity<T> | null> {
|
||||||
await this.ensureInitialized()
|
await this.ensureInitialized()
|
||||||
|
|
@ -431,6 +543,107 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Create a relationship between entities
|
* Create a relationship between entities
|
||||||
|
*
|
||||||
|
* @param params - Parameters for creating the relationship
|
||||||
|
* @returns Promise that resolves to the relationship ID
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Basic relationship creation
|
||||||
|
* const userId = await brainy.add({
|
||||||
|
* data: { name: 'John', role: 'developer' },
|
||||||
|
* type: NounType.Person
|
||||||
|
* })
|
||||||
|
* const projectId = await brainy.add({
|
||||||
|
* data: { name: 'AI Assistant', status: 'active' },
|
||||||
|
* type: NounType.Thing
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* const relationId = await brainy.relate({
|
||||||
|
* from: userId,
|
||||||
|
* to: projectId,
|
||||||
|
* type: VerbType.WorksOn
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Bidirectional relationships
|
||||||
|
* const friendshipId = await brainy.relate({
|
||||||
|
* from: 'user-1',
|
||||||
|
* to: 'user-2',
|
||||||
|
* type: VerbType.Knows,
|
||||||
|
* bidirectional: true // Creates both directions automatically
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Weighted relationships for importance/strength
|
||||||
|
* const collaborationId = await brainy.relate({
|
||||||
|
* from: 'team-lead',
|
||||||
|
* to: 'project-alpha',
|
||||||
|
* type: VerbType.LeadsOn,
|
||||||
|
* weight: 0.9, // High importance/strength
|
||||||
|
* metadata: {
|
||||||
|
* startDate: '2024-01-15',
|
||||||
|
* responsibility: 'technical leadership',
|
||||||
|
* hoursPerWeek: 40
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Typed relationships with custom metadata
|
||||||
|
* interface CollaborationMeta {
|
||||||
|
* role: string
|
||||||
|
* startDate: string
|
||||||
|
* skillLevel: number
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* const brainy = new Brainy<CollaborationMeta>({ storage: 'filesystem' })
|
||||||
|
* const relationId = await brainy.relate({
|
||||||
|
* from: 'developer-123',
|
||||||
|
* to: 'project-456',
|
||||||
|
* type: VerbType.WorksOn,
|
||||||
|
* weight: 0.85,
|
||||||
|
* metadata: {
|
||||||
|
* role: 'frontend developer',
|
||||||
|
* startDate: '2024-03-01',
|
||||||
|
* skillLevel: 8
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Creating complex relationship networks
|
||||||
|
* const entities = []
|
||||||
|
* // Create entities
|
||||||
|
* for (let i = 0; i < 5; i++) {
|
||||||
|
* const id = await brainy.add({
|
||||||
|
* data: { name: `Entity ${i}`, value: i * 10 },
|
||||||
|
* type: NounType.Thing
|
||||||
|
* })
|
||||||
|
* entities.push(id)
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* // Create hierarchical relationships
|
||||||
|
* for (let i = 0; i < entities.length - 1; i++) {
|
||||||
|
* await brainy.relate({
|
||||||
|
* from: entities[i],
|
||||||
|
* to: entities[i + 1],
|
||||||
|
* type: VerbType.DependsOn,
|
||||||
|
* weight: (i + 1) / entities.length
|
||||||
|
* })
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Error handling for invalid relationships
|
||||||
|
* try {
|
||||||
|
* await brainy.relate({
|
||||||
|
* from: 'nonexistent-entity',
|
||||||
|
* to: 'another-entity',
|
||||||
|
* type: VerbType.RelatedTo
|
||||||
|
* })
|
||||||
|
* } catch (error) {
|
||||||
|
* if (error.message.includes('not found')) {
|
||||||
|
* console.log('One or both entities do not exist')
|
||||||
|
* // Handle missing entities...
|
||||||
|
* }
|
||||||
|
* }
|
||||||
*/
|
*/
|
||||||
async relate(params: RelateParams<T>): Promise<string> {
|
async relate(params: RelateParams<T>): Promise<string> {
|
||||||
await this.ensureInitialized()
|
await this.ensureInitialized()
|
||||||
|
|
@ -557,6 +770,142 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
/**
|
/**
|
||||||
* Unified find method - supports natural language and structured queries
|
* Unified find method - supports natural language and structured queries
|
||||||
* Implements Triple Intelligence with parallel search optimization
|
* Implements Triple Intelligence with parallel search optimization
|
||||||
|
*
|
||||||
|
* @param query - Natural language string or structured FindParams object
|
||||||
|
* @returns Promise that resolves to array of search results with scores
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Natural language queries (most common)
|
||||||
|
* const results = await brainy.find('users who work on AI projects')
|
||||||
|
* const docs = await brainy.find('documents about machine learning')
|
||||||
|
* const code = await brainy.find('JavaScript functions for data processing')
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Structured queries with filtering
|
||||||
|
* const results = await brainy.find({
|
||||||
|
* query: 'artificial intelligence',
|
||||||
|
* type: NounType.Document,
|
||||||
|
* limit: 5,
|
||||||
|
* where: {
|
||||||
|
* status: 'published',
|
||||||
|
* author: 'expert'
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* // Process results
|
||||||
|
* for (const result of results) {
|
||||||
|
* console.log(`Found: ${result.entity.data} (score: ${result.score})`)
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Metadata-only filtering (no vector search)
|
||||||
|
* const activeUsers = await brainy.find({
|
||||||
|
* type: NounType.Person,
|
||||||
|
* where: {
|
||||||
|
* status: 'active',
|
||||||
|
* department: 'engineering'
|
||||||
|
* },
|
||||||
|
* service: 'user-management'
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Vector similarity search with custom vectors
|
||||||
|
* const queryVector = await brainy.embed('machine learning algorithms')
|
||||||
|
* const similar = await brainy.find({
|
||||||
|
* vector: queryVector,
|
||||||
|
* limit: 10,
|
||||||
|
* type: [NounType.Document, NounType.Thing]
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Proximity search (find entities similar to existing ones)
|
||||||
|
* const relatedContent = await brainy.find({
|
||||||
|
* near: 'document-123', // Find entities similar to this one
|
||||||
|
* limit: 8,
|
||||||
|
* where: {
|
||||||
|
* published: true
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Pagination for large result sets
|
||||||
|
* const firstPage = await brainy.find({
|
||||||
|
* query: 'research papers',
|
||||||
|
* limit: 20,
|
||||||
|
* offset: 0
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* const secondPage = await brainy.find({
|
||||||
|
* query: 'research papers',
|
||||||
|
* limit: 20,
|
||||||
|
* offset: 20
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Complex search with multiple criteria
|
||||||
|
* const results = await brainy.find({
|
||||||
|
* query: 'machine learning models',
|
||||||
|
* type: [NounType.Thing, NounType.Document],
|
||||||
|
* where: {
|
||||||
|
* accuracy: { $gte: 0.9 }, // Metadata filtering
|
||||||
|
* framework: { $in: ['tensorflow', 'pytorch'] }
|
||||||
|
* },
|
||||||
|
* service: 'ml-pipeline',
|
||||||
|
* limit: 15
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Empty query returns all entities (paginated)
|
||||||
|
* const allEntities = await brainy.find({
|
||||||
|
* limit: 50,
|
||||||
|
* offset: 0
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Performance-optimized search patterns
|
||||||
|
* // Fast metadata-only search (no vector computation)
|
||||||
|
* const fastResults = await brainy.find({
|
||||||
|
* type: NounType.Person,
|
||||||
|
* where: { active: true },
|
||||||
|
* limit: 100
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* // Combined vector + metadata for precision
|
||||||
|
* const preciseResults = await brainy.find({
|
||||||
|
* query: 'senior developers',
|
||||||
|
* where: {
|
||||||
|
* experience: { $gte: 5 },
|
||||||
|
* skills: { $includes: 'javascript' }
|
||||||
|
* },
|
||||||
|
* limit: 10
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Error handling and result processing
|
||||||
|
* try {
|
||||||
|
* const results = await brainy.find('complex query here')
|
||||||
|
*
|
||||||
|
* if (results.length === 0) {
|
||||||
|
* console.log('No results found')
|
||||||
|
* return
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* // Filter by confidence threshold
|
||||||
|
* const highConfidence = results.filter(r => r.score > 0.7)
|
||||||
|
*
|
||||||
|
* // Sort by score (already sorted by default)
|
||||||
|
* const topResults = results.slice(0, 5)
|
||||||
|
*
|
||||||
|
* return topResults.map(r => ({
|
||||||
|
* id: r.id,
|
||||||
|
* content: r.entity.data,
|
||||||
|
* confidence: r.score,
|
||||||
|
* metadata: r.entity.metadata
|
||||||
|
* }))
|
||||||
|
* } catch (error) {
|
||||||
|
* console.error('Search failed:', error)
|
||||||
|
* return []
|
||||||
|
* }
|
||||||
*/
|
*/
|
||||||
async find(query: string | FindParams<T>): Promise<Result<T>[]> {
|
async find(query: string | FindParams<T>): Promise<Result<T>[]> {
|
||||||
await this.ensureInitialized()
|
await this.ensureInitialized()
|
||||||
|
|
@ -786,7 +1135,113 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Find similar entities
|
* Find similar entities using vector similarity
|
||||||
|
*
|
||||||
|
* @param params - Parameters specifying the target for similarity search
|
||||||
|
* @returns Promise that resolves to array of similar entities with similarity scores
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Find entities similar to a specific entity by ID
|
||||||
|
* const similarDocs = await brainy.similar({
|
||||||
|
* to: 'document-123',
|
||||||
|
* limit: 10
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* // Process similarity results
|
||||||
|
* for (const result of similarDocs) {
|
||||||
|
* console.log(`Similar entity: ${result.entity.data} (similarity: ${result.score})`)
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Find similar entities with type filtering
|
||||||
|
* const similarUsers = await brainy.similar({
|
||||||
|
* to: 'user-456',
|
||||||
|
* type: NounType.Person,
|
||||||
|
* limit: 5,
|
||||||
|
* where: {
|
||||||
|
* active: true,
|
||||||
|
* department: 'engineering'
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Find similar using a custom vector
|
||||||
|
* const customVector = await brainy.embed('artificial intelligence research')
|
||||||
|
* const similar = await brainy.similar({
|
||||||
|
* to: customVector,
|
||||||
|
* limit: 8,
|
||||||
|
* type: [NounType.Document, NounType.Thing]
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Find similar using an entity object
|
||||||
|
* const sourceEntity = await brainy.get('research-paper-789')
|
||||||
|
* if (sourceEntity) {
|
||||||
|
* const relatedPapers = await brainy.similar({
|
||||||
|
* to: sourceEntity,
|
||||||
|
* limit: 12,
|
||||||
|
* where: {
|
||||||
|
* published: true,
|
||||||
|
* category: 'machine-learning'
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Content recommendation system
|
||||||
|
* async function getRecommendations(userId: string) {
|
||||||
|
* // Get user's recent interactions
|
||||||
|
* const user = await brainy.get(userId)
|
||||||
|
* if (!user) return []
|
||||||
|
*
|
||||||
|
* // Find similar content
|
||||||
|
* const recommendations = await brainy.similar({
|
||||||
|
* to: userId,
|
||||||
|
* type: NounType.Document,
|
||||||
|
* limit: 20,
|
||||||
|
* where: {
|
||||||
|
* published: true,
|
||||||
|
* language: 'en'
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* // Filter out already seen content
|
||||||
|
* return recommendations.filter(rec =>
|
||||||
|
* !user.metadata.viewedItems?.includes(rec.id)
|
||||||
|
* )
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Duplicate detection system
|
||||||
|
* async function findPotentialDuplicates(entityId: string) {
|
||||||
|
* const duplicates = await brainy.similar({
|
||||||
|
* to: entityId,
|
||||||
|
* limit: 10
|
||||||
|
* })
|
||||||
|
*
|
||||||
|
* // High similarity might indicate duplicates
|
||||||
|
* const highSimilarity = duplicates.filter(d => d.score > 0.95)
|
||||||
|
*
|
||||||
|
* if (highSimilarity.length > 0) {
|
||||||
|
* console.log('Potential duplicates found:', highSimilarity.map(d => d.id))
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* return highSimilarity
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Error handling for missing entities
|
||||||
|
* try {
|
||||||
|
* const similar = await brainy.similar({
|
||||||
|
* to: 'nonexistent-entity',
|
||||||
|
* limit: 5
|
||||||
|
* })
|
||||||
|
* } catch (error) {
|
||||||
|
* if (error.message.includes('not found')) {
|
||||||
|
* console.log('Source entity does not exist')
|
||||||
|
* // Handle missing source entity
|
||||||
|
* }
|
||||||
|
* }
|
||||||
*/
|
*/
|
||||||
async similar(params: SimilarParams<T>): Promise<Result<T>[]> {
|
async similar(params: SimilarParams<T>): Promise<Result<T>[]> {
|
||||||
await this.ensureInitialized()
|
await this.ensureInitialized()
|
||||||
|
|
@ -1818,8 +2273,117 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Embed data into vector
|
* Embed data into vector representation
|
||||||
* Handles any data type by converting to string representation
|
* Handles any data type by intelligently converting to string representation
|
||||||
|
*
|
||||||
|
* @param data - Any data to convert to vector (string, object, array, etc.)
|
||||||
|
* @returns Promise that resolves to a numerical vector representation
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Basic string embedding
|
||||||
|
* const vector = await brainy.embed('machine learning algorithms')
|
||||||
|
* console.log('Vector dimensions:', vector.length)
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Object embedding with intelligent field extraction
|
||||||
|
* const documentVector = await brainy.embed({
|
||||||
|
* title: 'AI Research Paper',
|
||||||
|
* content: 'This paper discusses neural networks...',
|
||||||
|
* author: 'Dr. Smith',
|
||||||
|
* category: 'machine-learning'
|
||||||
|
* })
|
||||||
|
* // Uses 'content' field for embedding by default
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Different object field priorities
|
||||||
|
* // Priority: data > content > text > name > title > description
|
||||||
|
* const vectors = await Promise.all([
|
||||||
|
* brainy.embed({ data: 'primary content' }), // Uses 'data'
|
||||||
|
* brainy.embed({ content: 'main content' }), // Uses 'content'
|
||||||
|
* brainy.embed({ text: 'text content' }), // Uses 'text'
|
||||||
|
* brainy.embed({ name: 'entity name' }), // Uses 'name'
|
||||||
|
* brainy.embed({ title: 'document title' }), // Uses 'title'
|
||||||
|
* brainy.embed({ description: 'description text' }) // Uses 'description'
|
||||||
|
* ])
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Array embedding for batch processing
|
||||||
|
* const batchVectors = await brainy.embed([
|
||||||
|
* 'first document',
|
||||||
|
* 'second document',
|
||||||
|
* { content: 'third document as object' },
|
||||||
|
* { title: 'fourth document' }
|
||||||
|
* ])
|
||||||
|
* // Returns vector representing all items combined
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Complex object handling
|
||||||
|
* const complexData = {
|
||||||
|
* user: { name: 'John', role: 'developer' },
|
||||||
|
* project: { name: 'AI Assistant', status: 'active' },
|
||||||
|
* metrics: { score: 0.95, performance: 'excellent' }
|
||||||
|
* }
|
||||||
|
* const vector = await brainy.embed(complexData)
|
||||||
|
* // Converts entire object to JSON for embedding
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Pre-computing vectors for performance optimization
|
||||||
|
* const documents = [
|
||||||
|
* { id: 'doc1', content: 'Document 1 content...' },
|
||||||
|
* { id: 'doc2', content: 'Document 2 content...' },
|
||||||
|
* { id: 'doc3', content: 'Document 3 content...' }
|
||||||
|
* ]
|
||||||
|
*
|
||||||
|
* // Pre-compute all vectors
|
||||||
|
* const vectors = await Promise.all(
|
||||||
|
* documents.map(doc => brainy.embed(doc.content))
|
||||||
|
* )
|
||||||
|
*
|
||||||
|
* // Add entities with pre-computed vectors (faster)
|
||||||
|
* for (let i = 0; i < documents.length; i++) {
|
||||||
|
* await brainy.add({
|
||||||
|
* data: documents[i],
|
||||||
|
* type: NounType.Document,
|
||||||
|
* vector: vectors[i] // Skip embedding computation
|
||||||
|
* })
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Custom embedding for search queries
|
||||||
|
* async function searchWithCustomEmbedding(query: string) {
|
||||||
|
* // Enhance query for better matching
|
||||||
|
* const enhancedQuery = `search: ${query} relevant information`
|
||||||
|
* const queryVector = await brainy.embed(enhancedQuery)
|
||||||
|
*
|
||||||
|
* // Use pre-computed vector for search
|
||||||
|
* return brainy.find({
|
||||||
|
* vector: queryVector,
|
||||||
|
* limit: 10
|
||||||
|
* })
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Handling edge cases gracefully
|
||||||
|
* const edgeCases = await Promise.all([
|
||||||
|
* brainy.embed(null), // Returns vector for empty string
|
||||||
|
* brainy.embed(undefined), // Returns vector for empty string
|
||||||
|
* brainy.embed(''), // Returns vector for empty string
|
||||||
|
* brainy.embed(42), // Converts number to string
|
||||||
|
* brainy.embed(true), // Converts boolean to string
|
||||||
|
* brainy.embed([]), // Empty array handling
|
||||||
|
* brainy.embed({}) // Empty object handling
|
||||||
|
* ])
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // Using with similarity comparisons
|
||||||
|
* const doc1Vector = await brainy.embed('artificial intelligence research')
|
||||||
|
* const doc2Vector = await brainy.embed('machine learning algorithms')
|
||||||
|
*
|
||||||
|
* // Find entities similar to doc1Vector
|
||||||
|
* const similar = await brainy.find({
|
||||||
|
* vector: doc1Vector,
|
||||||
|
* limit: 5
|
||||||
|
* })
|
||||||
*/
|
*/
|
||||||
async embed(data: any): Promise<Vector> {
|
async embed(data: any): Promise<Vector> {
|
||||||
// Handle different data types intelligently
|
// Handle different data types intelligently
|
||||||
|
|
|
||||||
|
|
@ -48,25 +48,173 @@ export type AugmentationContext = AC
|
||||||
// REMOVED: Old augmentation type system for 2.0 clean architecture
|
// REMOVED: Old augmentation type system for 2.0 clean architecture
|
||||||
// All augmentations now use the unified BrainyAugmentation interface from brainyAugmentation.ts
|
// All augmentations now use the unified BrainyAugmentation interface from brainyAugmentation.ts
|
||||||
|
|
||||||
// Temporary exports for compilation - TO BE REMOVED
|
/**
|
||||||
|
* @deprecated Use BrainyAugmentation from '../augmentations/brainyAugmentation.js' instead
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { IAugmentation } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
*/
|
||||||
export type IAugmentation = BrainyAugmentation
|
export type IAugmentation = BrainyAugmentation
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated Augmentation types are now unified under BrainyAugmentation interface
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { AugmentationType } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
* if (augmentation.type === AugmentationType.SENSE) { ... }
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
* // Use the unified BrainyAugmentation interface directly
|
||||||
|
*/
|
||||||
export enum AugmentationType { SENSE = 'sense', CONDUIT = 'conduit', COGNITION = 'cognition', MEMORY = 'memory', PERCEPTION = 'perception', DIALOG = 'dialog', ACTIVATION = 'activation', WEBSOCKET = 'webSocket', SYNAPSE = 'synapse' }
|
export enum AugmentationType { SENSE = 'sense', CONDUIT = 'conduit', COGNITION = 'cognition', MEMORY = 'memory', PERCEPTION = 'perception', DIALOG = 'dialog', ACTIVATION = 'activation', WEBSOCKET = 'webSocket', SYNAPSE = 'synapse' }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated Use BrainyAugmentation interface directly instead of namespace types
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { BrainyAugmentations } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
* class MyAugmentation implements BrainyAugmentations.ISenseAugmentation { ... }
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
* class MyAugmentation implements BrainyAugmentation { ... }
|
||||||
|
*/
|
||||||
export namespace BrainyAugmentations {
|
export namespace BrainyAugmentations {
|
||||||
|
/** @deprecated Use BrainyAugmentation instead */
|
||||||
export type ISenseAugmentation = BrainyAugmentation
|
export type ISenseAugmentation = BrainyAugmentation
|
||||||
|
/** @deprecated Use BrainyAugmentation instead */
|
||||||
export type IConduitAugmentation = BrainyAugmentation
|
export type IConduitAugmentation = BrainyAugmentation
|
||||||
|
/** @deprecated Use BrainyAugmentation instead */
|
||||||
export type ICognitionAugmentation = BrainyAugmentation
|
export type ICognitionAugmentation = BrainyAugmentation
|
||||||
|
/** @deprecated Use BrainyAugmentation instead */
|
||||||
export type IMemoryAugmentation = BrainyAugmentation
|
export type IMemoryAugmentation = BrainyAugmentation
|
||||||
|
/** @deprecated Use BrainyAugmentation instead */
|
||||||
export type IPerceptionAugmentation = BrainyAugmentation
|
export type IPerceptionAugmentation = BrainyAugmentation
|
||||||
|
/** @deprecated Use BrainyAugmentation instead */
|
||||||
export type IDialogAugmentation = BrainyAugmentation
|
export type IDialogAugmentation = BrainyAugmentation
|
||||||
|
/** @deprecated Use BrainyAugmentation instead */
|
||||||
export type IActivationAugmentation = BrainyAugmentation
|
export type IActivationAugmentation = BrainyAugmentation
|
||||||
|
/** @deprecated Use BrainyAugmentation instead */
|
||||||
export type ISynapseAugmentation = BrainyAugmentation
|
export type ISynapseAugmentation = BrainyAugmentation
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated Use BrainyAugmentation from '../augmentations/brainyAugmentation.js' instead
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { ISenseAugmentation } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
* class MySense implements ISenseAugmentation { ... }
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
* class MySense implements BrainyAugmentation { ... }
|
||||||
|
*/
|
||||||
export type ISenseAugmentation = BrainyAugmentation
|
export type ISenseAugmentation = BrainyAugmentation
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated Use BrainyAugmentation from '../augmentations/brainyAugmentation.js' instead
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { IConduitAugmentation } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
*/
|
||||||
export type IConduitAugmentation = BrainyAugmentation
|
export type IConduitAugmentation = BrainyAugmentation
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated Use BrainyAugmentation from '../augmentations/brainyAugmentation.js' instead
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { ICognitionAugmentation } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
*/
|
||||||
export type ICognitionAugmentation = BrainyAugmentation
|
export type ICognitionAugmentation = BrainyAugmentation
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated Use BrainyAugmentation from '../augmentations/brainyAugmentation.js' instead
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { IMemoryAugmentation } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
*/
|
||||||
export type IMemoryAugmentation = BrainyAugmentation
|
export type IMemoryAugmentation = BrainyAugmentation
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated Use BrainyAugmentation from '../augmentations/brainyAugmentation.js' instead
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { IPerceptionAugmentation } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
*/
|
||||||
export type IPerceptionAugmentation = BrainyAugmentation
|
export type IPerceptionAugmentation = BrainyAugmentation
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated Use BrainyAugmentation from '../augmentations/brainyAugmentation.js' instead
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { IDialogAugmentation } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
*/
|
||||||
export type IDialogAugmentation = BrainyAugmentation
|
export type IDialogAugmentation = BrainyAugmentation
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated Use BrainyAugmentation from '../augmentations/brainyAugmentation.js' instead
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { IActivationAugmentation } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
*/
|
||||||
export type IActivationAugmentation = BrainyAugmentation
|
export type IActivationAugmentation = BrainyAugmentation
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated Use BrainyAugmentation from '../augmentations/brainyAugmentation.js' instead
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { ISynapseAugmentation } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
*/
|
||||||
export type ISynapseAugmentation = BrainyAugmentation
|
export type ISynapseAugmentation = BrainyAugmentation
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @deprecated WebSocket support is now built into BrainyAugmentation interface
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* // ❌ Old way (v2.x):
|
||||||
|
* import { IWebSocketSupport } from '@soulcraft/brainy/types/augmentations'
|
||||||
|
* class MyAugmentation implements IWebSocketSupport { ... }
|
||||||
|
*
|
||||||
|
* // ✅ New way (v3.x):
|
||||||
|
* import { BrainyAugmentation } from '@soulcraft/brainy'
|
||||||
|
* class MyAugmentation implements BrainyAugmentation {
|
||||||
|
* // WebSocket functionality is now part of the unified interface
|
||||||
|
* }
|
||||||
|
*/
|
||||||
export interface IWebSocketSupport {}
|
export interface IWebSocketSupport {}
|
||||||
Loading…
Add table
Add a link
Reference in a new issue