brainy/docs/augmentations/DEVELOPER-GUIDE.md
David Snelling 9c87982a7d 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™
MAJOR RELEASE: Complete evolution of Brainy with groundbreaking features and performance.

🎯 KEY FEATURES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Triple Intelligence™ Engine
  - Unified Vector + Metadata + Graph search
  - O(log n) performance on all operations
  - 3ms average search latency at any scale

 API Consolidation
  - 15+ search methods → 2 clean APIs
  - search() for vector similarity
  - find() for natural language queries

 Natural Language Processing
  - 220+ pre-computed NLP patterns
  - Instant context understanding
  - "Show me recent React components with tests"

 Zero Configuration
  - Works instantly, no setup required
  - Built-in embedding models (no API keys)
  - Smart defaults for everything
  - Automatic optimization

 Enterprise Features (Free for Everyone)
  - Scales to 10M+ items
  - Write-Ahead Logging (WAL) for durability
  - Distributed architecture with sharding
  - Read/write separation
  - Connection pooling & request deduplication
  - Built-in monitoring & health checks

 Universal Compatibility
  - Node.js, Browser, Edge Workers
  - 4 Storage Adapters (Memory, FileSystem, OPFS, S3)
  - TypeScript with full type safety
  - Worker-based embeddings

📦 WHAT'S INCLUDED:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Core AI Database with HNSW indexing
• 19 Production-ready augmentations
• Universal Memory Manager
• Complete CLI with all commands
• Brain Cloud integration (soulcraft.com)
• Comprehensive documentation
• 52 test files with 400+ tests
• Migration guide from 1.x

📊 PERFORMANCE:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Initialize: 450ms (24MB memory)
• Search: 3ms average (up to 10M items)
• Metadata Filter: 0.8ms (O(log n))
• Bulk Import: 2.3s per 1000 items
• Production Scale: 5.8ms at 10M items

🔧 TECHNICAL IMPROVEMENTS:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• TypeScript compilation: 153 errors → 0
• Memory usage: 200MB → 24MB baseline
• Circular dependencies resolved
• Worker thread communication fixed
• Storage adapter consistency
• Request coalescing for 3x performance

🛠️ CLI FEATURES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• brainy add - Smart data ingestion
• brainy find - Natural language search
• brainy search - Vector similarity
• brainy chat - AI conversation mode
• brainy cloud - Brain Cloud integration
• brainy augment - Manage extensions
• 100% API compatibility

📚 DOCUMENTATION:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Professional README with examples
• Quick Start guide (5 minutes)
• Enterprise Features guide
• Migration guide from 1.x
• API reference
• Architecture documentation

🌟 USE CASES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• AI memory layer for chatbots
• Semantic document search
• Code intelligence platforms
• Knowledge management systems
• Real-time recommendation engines
• Customer support automation

MIT License - Enterprise features included free for everyone.
No premium tiers, no paywalls, no limits.

Built with ❤️ by the Brainy community.
Visit https://soulcraft.com for Brain Cloud integration.
2025-08-26 12:32:21 -07:00

12 KiB

🛠️ Brainy Augmentation Developer Guide

How to create, test, and use augmentations in Brainy 2.0

Quick Start: Your First Augmentation

import { BaseAugmentation, BrainyAugmentation, AugmentationContext } from '@soulcraft/brainy'

export class MyFirstAugmentation extends BaseAugmentation {
  readonly name = 'my-first-augmentation'
  readonly timing = 'after' as const       // When to run: before | after | both
  readonly operations = ['addNoun'] as const  // Which operations to hook
  readonly priority = 10                    // Execution order (lower = first)
  
  protected async onInit(): Promise<void> {
    // Initialize your augmentation
    console.log('MyFirstAugmentation initialized!')
  }
  
  async execute<T = any>(
    operation: string,
    params: any,
    context?: AugmentationContext
  ): Promise<T | void> {
    // Your augmentation logic
    if (operation === 'addNoun') {
      console.log('Noun added:', params.noun)
      // You can access the brain instance
      const stats = await context?.brain.getStatistics()
      console.log('Total nouns:', stats.totalNouns)
    }
  }
  
  protected async onShutdown(): Promise<void> {
    // Cleanup
    console.log('MyFirstAugmentation shutting down')
  }
}

Using Your Augmentation

import { BrainyData } from '@soulcraft/brainy'
import { MyFirstAugmentation } from './my-first-augmentation'

const brain = new BrainyData()

// Register before init()
brain.augmentations.register(new MyFirstAugmentation())

await brain.init()

// Now your augmentation runs automatically!
await brain.addNoun('Hello World')
// Console: "Noun added: { id: '...', vector: [...], metadata: {} }"

Augmentation Lifecycle

1. Registration Phase

const aug = new MyAugmentation()
brain.augmentations.register(aug)  // Before brain.init()!

2. Initialization Phase

await brain.init()  // Calls aug.initialize() internally
// Your onInit() method runs here

3. Execution Phase

await brain.addNoun('data')  // Your execute() method runs

4. Shutdown Phase

await brain.shutdown()  // Your onShutdown() method runs

Timing Options

before - Modify Input

class ValidationAugmentation extends BaseAugmentation {
  readonly timing = 'before' as const
  
  async execute<T>(operation: string, params: any): Promise<any> {
    if (operation === 'addNoun') {
      // Validate and/or modify params
      if (!params.content) {
        throw new Error('Content required')
      }
      // Return modified params
      return { ...params, validated: true }
    }
  }
}

after - React to Results

class LoggingAugmentation extends BaseAugmentation {
  readonly timing = 'after' as const
  
  async execute<T>(operation: string, params: any): Promise<void> {
    if (operation === 'search') {
      console.log(`Search for "${params.query}" returned ${params.result.length} results`)
    }
    // Don't return anything - just observe
  }
}

both - Before AND After

class TimingAugmentation extends BaseAugmentation {
  readonly timing = 'both' as const
  private startTime?: number
  
  async execute<T>(operation: string, params: any, context?: AugmentationContext): Promise<void> {
    if (!this.startTime) {
      // Before execution
      this.startTime = Date.now()
    } else {
      // After execution
      const duration = Date.now() - this.startTime
      console.log(`${operation} took ${duration}ms`)
      this.startTime = undefined
    }
  }
}

Operation Hooks

Core Operations You Can Hook

readonly operations = [
  'addNoun',        // Adding data
  'updateNoun',     // Updating data
  'deleteNoun',     // Deleting data
  'getNoun',        // Retrieving data
  'search',         // Searching
  'find',           // Triple Intelligence queries
  'addVerb',        // Adding relationships
  'deleteVerb',     // Removing relationships
  'clear',          // Clearing data
  'all'            // Hook ALL operations
] as const

Example: Multi-Operation Hook

class AuditAugmentation extends BaseAugmentation {
  readonly operations = ['addNoun', 'updateNoun', 'deleteNoun'] as const
  
  async execute<T>(operation: string, params: any): Promise<void> {
    // Log all data modifications
    await this.logToAuditTrail(operation, params)
  }
}

Accessing Brain Context

class ContextAwareAugmentation extends BaseAugmentation {
  async execute<T>(
    operation: string,
    params: any,
    context?: AugmentationContext
  ): Promise<void> {
    // Access the brain instance
    const brain = context?.brain
    if (!brain) return
    
    // Use any brain method
    const stats = await brain.getStatistics()
    const size = await brain.size()
    const results = await brain.search('query')
    
    // Access other augmentations
    const cache = brain.augmentations.get('cache')
    if (cache) {
      await cache.clear()
    }
  }
}

Real-World Examples

1. Backup Augmentation

class BackupAugmentation extends BaseAugmentation {
  readonly name = 'backup'
  readonly timing = 'after' as const
  readonly operations = ['addNoun', 'updateNoun', 'deleteNoun'] as const
  readonly priority = 5
  
  private changes = 0
  private readonly backupThreshold = 100
  
  async execute<T>(operation: string, params: any, context?: AugmentationContext): Promise<void> {
    this.changes++
    
    if (this.changes >= this.backupThreshold) {
      await this.performBackup(context?.brain)
      this.changes = 0
    }
  }
  
  private async performBackup(brain?: any): Promise<void> {
    if (!brain) return
    const backup = await brain.backup()
    await this.saveToCloud(backup)
    console.log('Automatic backup completed')
  }
}

2. Rate Limiting Augmentation

class RateLimitAugmentation extends BaseAugmentation {
  readonly name = 'rate-limit'
  readonly timing = 'before' as const
  readonly operations = ['search', 'find'] as const
  readonly priority = 100  // High priority - run first
  
  private requests = new Map<string, number[]>()
  private readonly limit = 100  // 100 requests
  private readonly window = 60000  // per minute
  
  async execute<T>(operation: string, params: any): Promise<void> {
    const now = Date.now()
    const key = params.userId || 'anonymous'
    
    // Get request timestamps
    const timestamps = this.requests.get(key) || []
    
    // Remove old timestamps
    const recent = timestamps.filter(t => now - t < this.window)
    
    // Check limit
    if (recent.length >= this.limit) {
      throw new Error('Rate limit exceeded')
    }
    
    // Add current request
    recent.push(now)
    this.requests.set(key, recent)
  }
}

3. Encryption Augmentation

class EncryptionAugmentation extends BaseAugmentation {
  readonly name = 'encryption'
  readonly timing = 'both' as const
  readonly operations = ['addNoun', 'getNoun'] as const
  readonly priority = 90  // Run early
  
  async execute<T>(operation: string, params: any): Promise<any> {
    if (operation === 'addNoun') {
      // Encrypt before storing
      if (params.metadata?.sensitive) {
        params.content = await this.encrypt(params.content)
        params.encrypted = true
      }
      return params
    }
    
    if (operation === 'getNoun' && params.result?.encrypted) {
      // Decrypt after retrieval
      params.result.content = await this.decrypt(params.result.content)
      delete params.result.encrypted
      return params.result
    }
  }
}

Testing Your Augmentation

import { describe, it, expect } from 'vitest'
import { BrainyData } from '@soulcraft/brainy'
import { MyAugmentation } from './my-augmentation'

describe('MyAugmentation', () => {
  it('should hook into addNoun', async () => {
    const brain = new BrainyData({ storage: 'memory' })
    const aug = new MyAugmentation()
    
    // Spy on the execute method
    const executeSpy = vi.spyOn(aug, 'execute')
    
    brain.augmentations.register(aug)
    await brain.init()
    
    // Trigger the augmentation
    await brain.addNoun('test data')
    
    // Verify it was called
    expect(executeSpy).toHaveBeenCalledWith(
      'addNoun',
      expect.objectContaining({ content: 'test data' }),
      expect.any(Object)
    )
  })
})

Best Practices

1. Use Proper Timing

  • before: Validation, modification, rate limiting
  • after: Logging, metrics, side effects
  • both: Timing, tracing, wrapping

2. Set Appropriate Priority

// Priority guidelines
100: Critical (auth, rate limiting)
50:  Important (validation, transformation)
10:  Normal (logging, metrics)
1:   Optional (debugging, tracing)

3. Handle Errors Gracefully

async execute<T>(operation: string, params: any): Promise<void> {
  try {
    await this.riskyOperation()
  } catch (error) {
    // Log but don't break the main operation
    console.error(`Augmentation error in ${this.name}:`, error)
    // Optionally report to monitoring
    this.reportError(error)
  }
}

4. Be Performance Conscious

class CachedAugmentation extends BaseAugmentation {
  private cache = new Map<string, any>()
  
  async execute<T>(operation: string, params: any): Promise<any> {
    const key = this.getCacheKey(params)
    
    // Check cache first
    if (this.cache.has(key)) {
      return this.cache.get(key)
    }
    
    // Expensive operation
    const result = await this.expensiveOperation(params)
    this.cache.set(key, result)
    
    return result
  }
}

5. Clean Up Resources

protected async onShutdown(): Promise<void> {
  // Close connections
  await this.connection?.close()
  
  // Clear intervals
  clearInterval(this.interval)
  
  // Flush buffers
  await this.flush()
  
  // Clear caches
  this.cache.clear()
}

Publishing Your Augmentation (Future)

Package Structure

my-augmentation/
├── src/
│   └── index.ts          # Your augmentation
├── dist/                 # Built output
├── tests/
│   └── augmentation.test.ts
├── package.json
├── tsconfig.json
└── README.md

package.json

{
  "name": "@mycompany/brainy-custom-augmentation",
  "version": "1.0.0",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
  "keywords": ["brainy-augmentation"],
  "peerDependencies": {
    "@soulcraft/brainy": ">=2.0.0"
  },
  "brainy": {
    "type": "augmentation",
    "class": "CustomAugmentation",
    "timing": "after",
    "operations": ["addNoun"],
    "priority": 10
  }
}

Future: Brain Cloud Registry

# Coming in 2.1+
npm run build
npm test
brainy publish  # Publishes to brain-cloud registry

FAQ

Q: Can I modify the operation result?

A: Yes, if timing: 'before', return modified params. If timing: 'after', you can see but not modify results.

Q: Can augmentations communicate?

A: Yes, through the context: context.brain.augmentations.get('other-augmentation')

Q: What if my augmentation fails?

A: Handle errors internally. Don't break the main operation unless critical.

Q: Can I use async operations?

A: Yes, everything is async-friendly.

Q: How do I access storage directly?

A: Through context: context.brain.storage (but prefer using brain methods)


Get Help


Start building your augmentation today! The marketplace is coming in 2.1 🚀