brainy/docs/augmentations/COMPLETE-REFERENCE.md

🔌 Brainy Augmentations Complete Reference

All augmentations that power Brainy's extensibility - with locations, usage, and examples

⚠️ Update: Updated for metadata structure changes and billion-scale optimizations

Quick Start

import { Brainy } from '@soulcraft/brainy'

const brain = new Brainy({
  // Augmentations auto-configure based on environment
  storage: { type: 'auto', rootDirectory: './brainy-data' }, // Storage augmentation
  cache: true, // Cache augmentation
  index: true // Index augmentation
})

await brain.init() // Augmentations initialize automatically

Augmentation Architecture

Key Improvements for Billion-Scale Performance

1. Metadata/Vector Separation: Augmentations now work with separated metadata and vectors

• Metadata stored separately from vector data
• 99.2% memory reduction for type tracking
• Two-file storage pattern for optimal I/O

2. Type System Enforcement: All metadata requires type fields

• NounMetadata requires noun: NounType
• VerbMetadata requires verb: VerbType
• Type inference system available as public API

3. Storage Adapter Pattern: Internal vs public method distinction

• _methods: Return pure structures (HNSWNoun, HNSWVerb)
• Public methods: Return WithMetadata types
• MetadataEnforcer Proxy ensures proper access

What This Means for Augmentation Users

✅ If you use built-in augmentations: No changes needed! They're all updated.

⚠️ If you created custom storage augmentations: Update your storage adapter to:

• Wrap metadata with required noun/verb fields
• Follow the internal/public method pattern
• Use two-file storage approach

⚠️ If you access relationship data: Change verb.type to verb.verb

Core Concepts

What are Augmentations?

Augmentations are modular extensions that add functionality to Brainy without cluttering the core API. They follow a unified interface and can be:

• Auto-enabled: Based on configuration (cache, index, storage)
• Manually registered: For custom functionality
• Chained: Multiple augmentations work together seamlessly
• Billion-scale ready: Optimized for datasets with billions of nouns and verbs

Augmentation Lifecycle

1. Registration: Augmentations register before init()
2. Initialization: Two-phase init (storage first, then others)
3. Execution: Hook into operations (before/after/both)
4. Shutdown: Clean teardown on brain.shutdown()

---

Storage Augmentations

MemoryStorageAugmentation

Location: src/augmentations/storageAugmentations.ts Auto-enabled: When storage: { type: 'memory' } or in test environments Purpose: In-memory storage for testing and temporary data

const brain = new Brainy({ storage: { type: 'memory' } })

FileSystemStorageAugmentation

Location: src/augmentations/storageAugmentations.ts Auto-enabled: When storage: { type: 'filesystem' } or Node.js detected Purpose: Persistent file-based storage for Node.js applications

const brain = new Brainy({
  storage: { type: 'filesystem', rootDirectory: './data' }
})

For off-site backup, snapshot rootDirectory from your scheduler using gsutil rsync, aws s3 sync, rclone, or tar — there are no cloud storage augmentations.

StorageAugmentation (base)

Location: src/augmentations/storageAugmentation.ts Purpose: Base class for custom storage implementations

DynamicStorageAugmentation

Location: src/augmentations/storageAugmentation.ts Purpose: Runtime storage adapter switching

---

Performance Augmentations (7 total)

CacheAugmentation

Location: src/augmentations/cacheAugmentation.ts Auto-enabled: When cache: true (default) Purpose: LRU cache for search results and frequent queries

brain.clearCache() // Exposed via API
brain.getCacheStats() // Cache hit/miss statistics

IndexAugmentation

Location: src/augmentations/indexAugmentation.ts Auto-enabled: When index: true (default) Purpose: Metadata indexing for O(1) field lookups

brain.rebuildMetadataIndex() // Exposed via API
// Enables fast where queries:
brain.find({ where: { category: 'tech' } })

MetricsAugmentation

Location: src/augmentations/metricsAugmentation.ts Auto-enabled: Always active Purpose: Performance metrics and statistics collection

brain.getStats() // Comprehensive metrics

MonitoringAugmentation

Location: src/augmentations/monitoringAugmentation.ts Manual: Register for detailed monitoring Purpose: Real-time performance monitoring and alerts

BatchProcessingAugmentation

Location: src/augmentations/batchProcessingAugmentation.ts Auto-enabled: For batch operations Purpose: Optimizes bulk add/update/delete operations

brain.addNouns([...]) // Automatically batched

RequestDeduplicatorAugmentation

Location: src/augmentations/requestDeduplicatorAugmentation.ts Auto-enabled: Always active Purpose: Prevents duplicate concurrent operations

---

Data Integrity Augmentations (3 total)

Auto-enabled: When wal: true Purpose: Write-ahead logging for crash recovery

const brain = new Brainy({ wal: true })
// Automatic recovery on restart after crash

EntityRegistryAugmentation

Location: src/augmentations/entityRegistryAugmentation.ts Auto-enabled: For streaming operations Purpose: High-speed deduplication for real-time data

// Prevents duplicate entities in streaming scenarios
brain.add(data) // Automatically deduplicated

AutoRegisterEntitiesAugmentation

Location: src/augmentations/entityRegistryAugmentation.ts Manual: For automatic entity discovery Purpose: Auto-discovers and registers entities from data

---

Intelligence Augmentations (2 total)

NeuralImportAugmentation

Location: src/augmentations/neuralImport.ts Manual: Via brain.neuralImport() Purpose: AI-powered smart data import

const result = await brain.neuralImport(data, {
 confidenceThreshold: 0.7,
 autoApply: true
})
// Automatically detects entities and relationships

IntelligentVerbScoringAugmentation

Location: src/augmentations/intelligentVerbScoringAugmentation.ts Auto-enabled: When verbs are used Purpose: ML-based relationship strength scoring

brain.verbScoring.train(feedback)
brain.verbScoring.getScore(verbId)

---

Communication Augmentations (4 total)

APIServerAugmentation

Location: src/augmentations/apiServerAugmentation.ts Manual: For server deployments Purpose: REST/WebSocket/MCP API server

const augmentation = new APIServerAugmentation()
await brain.registerAugmentation(augmentation)
// Exposes full Brainy API over network

WebSocketConduitAugmentation

Location: src/augmentations/conduitAugmentations.ts Manual: For Brainy-to-Brainy sync Purpose: Real-time sync between Brainy instances

const conduit = new WebSocketConduitAugmentation()
await conduit.establishConnection('ws://other-brain')

ServerSearchConduitAugmentation

Location: src/augmentations/serverSearchAugmentations.ts Manual: For client-server search Purpose: Search remote Brainy instance, cache locally

ServerSearchActivationAugmentation

Location: src/augmentations/serverSearchAugmentations.ts Manual: Works with ServerSearchConduit Purpose: Triggers and manages server search operations

---

External Integration (2 total)

SynapseAugmentation (base)

Location: src/augmentations/synapseAugmentation.ts Purpose: Base class for external platform integrations

// Example: NotionSynapse, SlackSynapse, etc.
class NotionSynapse extends SynapseAugmentation {
 async fetchData() { /* Notion API calls */ }
 async pushData() { /* Sync to Notion */ }
}

ExampleFileSystemSynapse

Location: src/augmentations/synapseAugmentation.ts Purpose: Example implementation for file system sync

---

Augmentation Configuration

Auto-Configuration

const brain = new Brainy({
  // These auto-register augmentations:
  storage: { type: 'auto', rootDirectory: './brainy-data' }, // Storage augmentation
  cache: true,   // Cache augmentation
  index: true,   // Index augmentation
  metrics: true  // Metrics augmentation
})

Manual Registration

const brain = new Brainy()

// Register before init()
const customAug = new MyCustomAugmentation()
await brain.registerAugmentation(customAug)

await brain.init()

Creating Custom Augmentations

import { BaseAugmentation } from '@soulcraft/brainy'

class MyAugmentation extends BaseAugmentation {
 readonly name = 'my-augmentation'
 readonly timing = 'after' // before | after | both
 readonly operations = ['addNoun', 'search'] // Which ops to hook
 readonly priority = 10 // Execution order (lower = earlier)

 protected async onInit(): Promise<void> {
 // Initialize your augmentation
 }

 async execute<T>(
 operation: string,
 params: any,
 context?: AugmentationContext
 ): Promise<T | void> {
 // Your augmentation logic
 if (operation === 'addNoun') {
 console.log('Noun added:', params)
 }
 }

 protected async onShutdown(): Promise<void> {
 // Cleanup
 }
}

---

Augmentation Timing & Priority

Timing Options

• before: Runs before the operation (can modify params)
• after: Runs after the operation (can see results)
• both: Runs before AND after

Priority (lower = earlier)

1. Storage augmentations (priority: 0)
2. Cache/Index augmentations (priority: 5-10)
3. Monitoring/Metrics (priority: 15-20)
4. Conduits/Synapses (priority: 20-30)

---

Key Integration Points

Where Augmentations Hook In

Brainy Constructor:

• Storage augmentations register based on config
• Cache/Index augmentations auto-register if enabled

brain.init():

• Two-phase initialization (storage first, then others)
• Augmentations can access brain instance via context

Operations (addNoun, search, etc.):

• Augmentations execute based on timing and operations filter
• Can modify params (before) or see results (after)

brain.shutdown():

• All augmentations cleaned up in reverse order

---

Performance Impact

Most augmentations have minimal overhead:

• Cache: ~1ms per search (saves 10-100ms on hits)
• Index: ~1ms per operation (saves 100ms+ on queries)
• Metrics: <1ms per operation
• Storage: Varies by adapter (memory: 0ms, filesystem: 1-10ms)

---

Best Practices

1. Let auto-configuration work: Most apps need zero manual config
2. Storage first: Always configure storage before other augmentations
3. Use built-in augmentations: They're optimized and battle-tested
4. Custom augmentations: Extend BaseAugmentation for consistency
5. Respect timing: Use 'before' to modify, 'after' to observe
6. Mind priority: Lower numbers execute first

---

Troubleshooting

Augmentation not working?

// Check if registered
brain.listAugmentations()

// Check if enabled
brain.isAugmentationEnabled('cache')

// Enable/disable at runtime
brain.enableAugmentation('cache')
brain.disableAugmentation('cache')

Performance issues?

// Check augmentation overhead
const stats = brain.getStats()
console.log(stats.augmentations)

// Disable non-critical augmentations
brain.disableAugmentation('monitoring')

---

---

Augmentations make Brainy infinitely extensible while keeping the core API clean and simple!