brainy/docs/augmentations/COMPLETE-REFERENCE.md

🔌 Brainy v4.0.0 Augmentations Complete Reference

All augmentations that power Brainy's extensibility - with locations, usage, and examples

⚠️ v4.0.0 Update: Updated for metadata structure changes and billion-scale optimizations

Quick Start

import { Brainy } from '@soulcraft/brainy'

const brain = new Brainy({
  // Augmentations auto-configure based on environment
  storage: 'auto',     // Storage augmentation
  cache: true,         // Cache augmentation
  index: true          // Index augmentation
})

await brain.init()  // Augmentations initialize automatically

v4.0.0 Augmentation Architecture

Key Improvements for Billion-Scale Performance

1. Metadata/Vector Separation: Augmentations now work with separated metadata and vectors

   • Metadata stored separately from vector data
   • 99.2% memory reduction for type tracking
   • Two-file storage pattern for optimal I/O

2. Type System Enforcement: All metadata requires type fields

   • NounMetadata requires noun: NounType
   • VerbMetadata requires verb: VerbType
   • Type inference system available as public API

3. Storage Adapter Pattern: Internal vs public method distinction

   • _methods: Return pure structures (HNSWNoun, HNSWVerb)
   • Public methods: Return WithMetadata types
   • MetadataEnforcer Proxy ensures proper access

What This Means for Augmentation Users

✅ If you use built-in augmentations: No changes needed! They're all updated for v4.0.0.

⚠️ If you created custom storage augmentations: Update your storage adapter to:

• Wrap metadata with required noun/verb fields
• Follow the internal/public method pattern
• Use two-file storage approach

⚠️ If you access relationship data: Change verb.type to verb.verb

Core Concepts

What are Augmentations?

Augmentations are modular extensions that add functionality to Brainy without cluttering the core API. They follow a unified interface and can be:

• Auto-enabled: Based on configuration (cache, index, storage)
• Manually registered: For custom functionality
• Chained: Multiple augmentations work together seamlessly
• Billion-scale ready: Optimized for datasets with billions of nouns and verbs

Augmentation Lifecycle

1. Registration: Augmentations register before init()
2. Initialization: Two-phase init (storage first, then others)
3. Execution: Hook into operations (before/after/both)
4. Shutdown: Clean teardown on brain.shutdown()

---

Storage Augmentations (8 total)

MemoryStorageAugmentation

Location: src/augmentations/storageAugmentations.ts
Auto-enabled: When storage: 'memory' or in test environments
Purpose: In-memory storage for testing and temporary data

const brain = new Brainy({ storage: 'memory' })

FileSystemStorageAugmentation

Location: src/augmentations/storageAugmentations.ts
Auto-enabled: When storage: 'filesystem' or Node.js detected
Purpose: Persistent file-based storage for Node.js applications

const brain = new Brainy({ 
  storage: { type: 'filesystem', path: './data' }
})

OPFSStorageAugmentation

Location: src/augmentations/storageAugmentations.ts
Auto-enabled: When storage: 'opfs' or browser with OPFS support
Purpose: Browser-based persistent storage using Origin Private File System

const brain = new Brainy({ storage: 'opfs' })

S3StorageAugmentation

Location: src/augmentations/storageAugmentations.ts
Manual: Requires AWS credentials
Purpose: AWS S3-compatible cloud storage

const brain = new Brainy({ 
  storage: {
    type: 's3',
    bucket: 'my-bucket',
    region: 'us-east-1',
    credentials: { accessKeyId, secretAccessKey }
  }
})

R2StorageAugmentation

Location: src/augmentations/storageAugmentations.ts
Manual: Requires Cloudflare credentials
Purpose: Cloudflare R2 storage (S3-compatible)

const brain = new Brainy({ 
  storage: {
    type: 'r2',
    accountId: 'xxx',
    bucket: 'my-bucket',
    credentials: { accessKeyId, secretAccessKey }
  }
})

GCSStorageAugmentation

Location: src/augmentations/storageAugmentations.ts
Manual: Requires Google Cloud credentials
Purpose: Google Cloud Storage

const brain = new Brainy({ 
  storage: {
    type: 'gcs',
    bucket: 'my-bucket',
    projectId: 'my-project'
  }
})

StorageAugmentation (base)

Location: src/augmentations/storageAugmentation.ts
Purpose: Base class for custom storage implementations

DynamicStorageAugmentation

Location: src/augmentations/storageAugmentation.ts
Purpose: Runtime storage adapter switching

---

Performance Augmentations (7 total)

CacheAugmentation

Location: src/augmentations/cacheAugmentation.ts
Auto-enabled: When cache: true (default)
Purpose: LRU cache for search results and frequent queries

brain.clearCache()           // Exposed via API
brain.getCacheStats()        // Cache hit/miss statistics

IndexAugmentation

Location: src/augmentations/indexAugmentation.ts
Auto-enabled: When index: true (default)
Purpose: Metadata indexing for O(1) field lookups

brain.rebuildMetadataIndex() // Exposed via API
// Enables fast where queries:
brain.find({ where: { category: 'tech' } })

MetricsAugmentation

Location: src/augmentations/metricsAugmentation.ts
Auto-enabled: Always active
Purpose: Performance metrics and statistics collection

brain.getStats()        // Comprehensive metrics

MonitoringAugmentation

Location: src/augmentations/monitoringAugmentation.ts
Manual: Register for detailed monitoring
Purpose: Real-time performance monitoring and alerts

BatchProcessingAugmentation

Location: src/augmentations/batchProcessingAugmentation.ts
Auto-enabled: For batch operations
Purpose: Optimizes bulk add/update/delete operations

brain.addNouns([...])        // Automatically batched

RequestDeduplicatorAugmentation

Location: src/augmentations/requestDeduplicatorAugmentation.ts
Auto-enabled: Always active
Purpose: Prevents duplicate concurrent operations

ConnectionPoolAugmentation

Location: src/augmentations/connectionPoolAugmentation.ts
Auto-enabled: For network storage
Purpose: Connection pooling for cloud storage adapters

---

Data Integrity Augmentations (3 total)

Auto-enabled: When wal: true
Purpose: Write-ahead logging for crash recovery

const brain = new Brainy({ wal: true })
// Automatic recovery on restart after crash

EntityRegistryAugmentation

Location: src/augmentations/entityRegistryAugmentation.ts
Auto-enabled: For streaming operations
Purpose: High-speed deduplication for real-time data

// Prevents duplicate entities in streaming scenarios
brain.add(data) // Automatically deduplicated

AutoRegisterEntitiesAugmentation

Location: src/augmentations/entityRegistryAugmentation.ts
Manual: For automatic entity discovery
Purpose: Auto-discovers and registers entities from data

---

Intelligence Augmentations (2 total)

NeuralImportAugmentation

Location: src/augmentations/neuralImport.ts
Manual: Via brain.neuralImport()
Purpose: AI-powered smart data import

const result = await brain.neuralImport(data, {
  confidenceThreshold: 0.7,
  autoApply: true
})
// Automatically detects entities and relationships

IntelligentVerbScoringAugmentation

Location: src/augmentations/intelligentVerbScoringAugmentation.ts
Auto-enabled: When verbs are used
Purpose: ML-based relationship strength scoring

brain.verbScoring.train(feedback)
brain.verbScoring.getScore(verbId)

---

Communication Augmentations (4 total)

APIServerAugmentation

Location: src/augmentations/apiServerAugmentation.ts
Manual: For server deployments
Purpose: REST/WebSocket/MCP API server

const augmentation = new APIServerAugmentation()
await brain.registerAugmentation(augmentation)
// Exposes full Brainy API over network

WebSocketConduitAugmentation

Location: src/augmentations/conduitAugmentations.ts
Manual: For Brainy-to-Brainy sync
Purpose: Real-time sync between Brainy instances

const conduit = new WebSocketConduitAugmentation()
await conduit.establishConnection('ws://other-brain')

ServerSearchConduitAugmentation

Location: src/augmentations/serverSearchAugmentations.ts
Manual: For client-server search
Purpose: Search remote Brainy instance, cache locally

ServerSearchActivationAugmentation

Location: src/augmentations/serverSearchAugmentations.ts
Manual: Works with ServerSearchConduit
Purpose: Triggers and manages server search operations

---

External Integration (2 total)

SynapseAugmentation (base)

Location: src/augmentations/synapseAugmentation.ts
Purpose: Base class for external platform integrations

// Example: NotionSynapse, SlackSynapse, etc.
class NotionSynapse extends SynapseAugmentation {
  async fetchData() { /* Notion API calls */ }
  async pushData() { /* Sync to Notion */ }
}

ExampleFileSystemSynapse

Location: src/augmentations/synapseAugmentation.ts
Purpose: Example implementation for file system sync

---

Augmentation Configuration

Auto-Configuration

const brain = new Brainy({
  // These auto-register augmentations:
  storage: 'auto',        // Storage augmentation
  cache: true,           // Cache augmentation  
  index: true,           // Index augmentation
  metrics: true         // Metrics augmentation
})

Manual Registration

const brain = new Brainy()

// Register before init()
const customAug = new MyCustomAugmentation()
await brain.registerAugmentation(customAug)

await brain.init()

Creating Custom Augmentations

import { BaseAugmentation } from '@soulcraft/brainy'

class MyAugmentation extends BaseAugmentation {
  readonly name = 'my-augmentation'
  readonly timing = 'after'  // before | after | both
  readonly operations = ['addNoun', 'search']  // Which ops to hook
  readonly priority = 10      // Execution order (lower = earlier)
  
  protected async onInit(): Promise<void> {
    // Initialize your augmentation
  }
  
  async execute<T>(
    operation: string,
    params: any,
    context?: AugmentationContext
  ): Promise<T | void> {
    // Your augmentation logic
    if (operation === 'addNoun') {
      console.log('Noun added:', params)
    }
  }
  
  protected async onShutdown(): Promise<void> {
    // Cleanup
  }
}

---

Augmentation Timing & Priority

Timing Options

• before: Runs before the operation (can modify params)
• after: Runs after the operation (can see results)
• both: Runs before AND after

Priority (lower = earlier)

1. Storage augmentations (priority: 0)
2. Cache/Index augmentations (priority: 5-10)
3. Monitoring/Metrics (priority: 15-20)
4. Conduits/Synapses (priority: 20-30)

---

Key Integration Points

Where Augmentations Hook In

Brainy Constructor:

• Storage augmentations register based on config
• Cache/Index augmentations auto-register if enabled

brain.init():

• Two-phase initialization (storage first, then others)
• Augmentations can access brain instance via context

Operations (addNoun, search, etc.):

• Augmentations execute based on timing and operations filter
• Can modify params (before) or see results (after)

brain.shutdown():

• All augmentations cleaned up in reverse order

---

Performance Impact

Most augmentations have minimal overhead:

• Cache: ~1ms per search (saves 10-100ms on hits)
• Index: ~1ms per operation (saves 100ms+ on queries)
• Metrics: <1ms per operation
• Storage: Varies by adapter (memory: 0ms, S3: 50-200ms)

---

Best Practices

1. Let auto-configuration work: Most apps need zero manual config
2. Storage first: Always configure storage before other augmentations
3. Use built-in augmentations: They're optimized and battle-tested
4. Custom augmentations: Extend BaseAugmentation for consistency
5. Respect timing: Use 'before' to modify, 'after' to observe
6. Mind priority: Lower numbers execute first

---

Troubleshooting

Augmentation not working?

// Check if registered
brain.listAugmentations()

// Check if enabled
brain.isAugmentationEnabled('cache')

// Enable/disable at runtime
brain.enableAugmentation('cache')
brain.disableAugmentation('cache')

Performance issues?

// Check augmentation overhead
const stats = brain.getStats()
console.log(stats.augmentations)

// Disable non-critical augmentations
brain.disableAugmentation('monitoring')

---

---

Augmentations make Brainy infinitely extensible while keeping the core API clean and simple!