brainy/docs/CREATING-AUGMENTATIONS.md

# Creating Augmentations for Brainy

> **Updated for v4.0.0** - Includes metadata structure changes and type system improvements

## The BrainyAugmentation Interface

Every augmentation implements this simple yet powerful interface:

```typescript
interface BrainyAugmentation {
  // Identification
  name: string                    // Unique name for your augmentation

  // Execution control
  timing: 'before' | 'after' | 'around' | 'replace'  // When to execute
  operations: string[]            // Which operations to intercept
  priority: number                 // Execution order (higher = first)

  // Lifecycle methods
  initialize(context: AugmentationContext): Promise<void>
  execute<T>(operation: string, params: any, next: () => Promise<T>): Promise<T>
  shutdown?(): Promise<void>      // Optional cleanup
}
```

## v4.0.0 Breaking Changes for Augmentation Developers

### 1. Metadata Structure Separation
v4.0.0 introduces strict metadata/vector separation for billion-scale performance:

```typescript
// ✅ v4.0.0: Metadata has required type field
interface NounMetadata {
  noun: NounType               // Required! Must be a valid noun type
  [key: string]: any          // Your custom metadata
}

interface VerbMetadata {
  verb: VerbType               // Required! Must be a valid verb type
  sourceId: string
  targetId: string
  [key: string]: any
}
```

### 2. Storage Adapter Return Types
Storage adapters now return different types at different boundaries:

```typescript
// Internal methods: Pure structures (no metadata)
abstract _getNoun(id: string): Promise<HNSWNoun | null>

// Public API: WithMetadata structures
abstract getNoun(id: string): Promise<HNSWNounWithMetadata | null>
```

### 3. Verb Property Renamed
The verb relationship field changed from `type` to `verb`:

```typescript
// ❌ v3.x
verb.type === 'relatedTo'

// ✅ v4.0.0
verb.verb === 'relatedTo'
```

## Creating a Storage Augmentation

Storage augmentations are special - they provide the storage backend for Brainy.

### Important: v4.0.0 Storage Requirements

Your storage adapter MUST:
1. **Wrap metadata** with required `noun`/`verb` fields
2. **Return pure structures** from internal `_methods`
3. **Return WithMetadata types** from public methods

```typescript
import { StorageAugmentation } from 'brainy/augmentations'
import { BaseStorageAdapter, HNSWNoun, HNSWNounWithMetadata, NounMetadata } from 'brainy'

export class MyCustomStorage extends BaseStorageAdapter {
  // Internal method: Returns pure structure
  async _getNoun(id: string): Promise<HNSWNoun | null> {
    const data = await this.fetchFromDatabase(id)
    return data ? {
      id: data.id,
      vector: data.vector,
      nounType: data.type
    } : null
  }

  // Public method: Returns WithMetadata structure
  async getNoun(id: string): Promise<HNSWNounWithMetadata | null> {
    const noun = await this._getNoun(id)
    if (!noun) return null

    // Fetch metadata separately (v4.0.0 pattern)
    const metadata = await this.getNounMetadata(id)

    return {
      ...noun,
      metadata: metadata || { noun: noun.nounType || 'thing' }
    }
  }

  // CRITICAL: Always save with proper metadata structure
  async saveNoun(noun: HNSWNoun, metadata?: NounMetadata): Promise<void> {
    // Validate metadata has required 'noun' field
    if (!metadata?.noun) {
      throw new Error('v4.0.0: NounMetadata requires "noun" field')
    }

    await this.database.save({
      id: noun.id,
      vector: noun.vector,
      nounType: noun.nounType,
      metadata: metadata  // Stored separately in v4.0.0
    })
  }
}

export class MyStorageAugmentation extends StorageAugmentation {
  private config: MyStorageConfig

  constructor(config: MyStorageConfig) {
    super()
    this.name = 'my-custom-storage'
    this.config = config
  }

  // Called during storage resolution phase
  async provideStorage(): Promise<StorageAdapter> {
    const storage = new MyCustomStorage(this.config)
    this.storageAdapter = storage
    return storage
  }

  // Called during augmentation initialization
  protected async onInitialize(): Promise<void> {
    await this.storageAdapter!.init()
    this.log(`Custom storage initialized`)
  }
}
```

### Using Your Storage Augmentation

```typescript
// Register before brain.init()
const brain = new Brainy()
brain.augmentations.register(new MyStorageAugmentation({
  connectionString: 'redis://localhost:6379'
}))
await brain.init()  // Will use your storage!
```

## Creating a Feature Augmentation

Here's a complete example of a caching augmentation:

```typescript
import { BaseAugmentation, BrainyAugmentation } from 'brainy/augmentations'

export class CachingAugmentation extends BaseAugmentation {
  private cache = new Map<string, any>()
  
  constructor() {
    super()
    this.name = 'smart-cache'
    this.timing = 'around'        // Wrap operations
    this.operations = ['search']   // Only cache searches
    this.priority = 50             // Mid-priority
  }
  
  async execute<T>(operation: string, params: any, next: () => Promise<T>): Promise<T> {
    if (operation === 'search') {
      // Check cache
      const cacheKey = JSON.stringify(params)
      if (this.cache.has(cacheKey)) {
        this.log('Cache hit!')
        return this.cache.get(cacheKey)
      }
      
      // Execute and cache
      const result = await next()
      this.cache.set(cacheKey, result)
      return result
    }
    
    // Pass through other operations
    return next()
  }
  
  protected async onInitialize(): Promise<void> {
    this.log('Cache initialized')
  }
  
  async shutdown(): Promise<void> {
    this.cache.clear()
    await super.shutdown()
  }
}
```

## The Four Timing Modes

### 1. `before` - Pre-processing
```typescript
timing = 'before'
async execute(op, params, next) {
  // Validate/transform input
  const validated = await validate(params)
  return next(validated)  // Pass modified params
}
```

### 2. `after` - Post-processing  
```typescript
timing = 'after'
async execute(op, params, next) {
  const result = await next()
  // Log, analyze, or modify result
  console.log(`Operation ${op} returned:`, result)
  return result
}
```

### 3. `around` - Wrapping (middleware)
```typescript
timing = 'around'
async execute(op, params, next) {
  console.log('Starting', op)
  try {
    const result = await next()
    console.log('Success', op)
    return result
  } catch (error) {
    console.log('Failed', op, error)
    throw error
  }
}
```

### 4. `replace` - Complete replacement
```typescript
timing = 'replace'
async execute(op, params, next) {
  // Don't call next() - replace entirely!
  return myCustomImplementation(params)
}
```

## Operations You Can Intercept

Common operations in Brainy:
- `'storage'` - Storage resolution (special)
- `'add'` - Adding data
- `'search'`, `'similar'` - Searching
- `'update'`, `'delete'` - Modifications
- `'saveNoun'`, `'saveVerb'` - Storage operations
- `'all'` - Intercept everything

## Context Available to Augmentations

```typescript
interface AugmentationContext {
  brain: Brainy         // The brain instance
  storage: StorageAdapter   // Storage backend
  config: BrainyConfig  // Configuration
  log: (message: string, level?: 'info' | 'warn' | 'error') => void
}
```

## Real-World Examples

### 1. Redis Storage Augmentation
```typescript
export class RedisStorageAugmentation extends StorageAugmentation {
  async provideStorage(): Promise<StorageAdapter> {
    return new RedisAdapter({
      host: 'localhost',
      port: 6379,
      // Implement full StorageAdapter interface
    })
  }
}
```

### 2. Audit Trail Augmentation
```typescript
export class AuditAugmentation extends BaseAugmentation {
  timing = 'after'
  operations = ['add', 'update', 'delete']
  
  async execute(op, params, next) {
    const result = await next()
    
    // Log to audit trail
    await this.logAudit({
      operation: op,
      params,
      result,
      timestamp: new Date(),
      user: this.context.config.currentUser
    })
    
    return result
  }
}
```

### 3. Rate Limiting Augmentation
```typescript
export class RateLimitAugmentation extends BaseAugmentation {
  timing = 'before'
  operations = ['search']
  private limiter = new RateLimiter({ rps: 100 })
  
  async execute(op, params, next) {
    await this.limiter.acquire()  // Wait if rate limited
    return next()
  }
}
```

## Publishing to Brain Cloud Marketplace

Future capability for premium augmentations:

```typescript
// package.json
{
  "name": "@brain-cloud/redis-storage",
  "brainy": {
    "type": "augmentation",
    "category": "storage",
    "premium": true
  }
}

// Users can install via:
// brainy augment install redis-storage
```

## Best Practices

### General Practices

1. **Use BaseAugmentation** - Provides common functionality
2. **Set appropriate priority** - Storage (100), System (80-99), Features (10-50)
3. **Be selective with operations** - Don't use 'all' unless necessary
4. **Handle errors gracefully** - Don't break the chain
5. **Clean up in shutdown()** - Release resources
6. **Log appropriately** - Use context.log() for consistent output
7. **Document your augmentation** - Include examples

### v4.0.0 Specific Best Practices

8. **Always include `noun` field** when creating/modifying NounMetadata:
   ```typescript
   const metadata: NounMetadata = {
     noun: 'thing',  // REQUIRED!
     yourField: 'value'
   }
   ```

9. **Use `verb` property** not `type` when working with relationships:
   ```typescript
   // ✅ Correct
   if (verb.verb === 'relatedTo') { ... }

   // ❌ Wrong (v3.x pattern)
   if (verb.type === 'relatedTo') { ... }
   ```

10. **Access metadata correctly** from storage:
    ```typescript
    // ✅ Correct - metadata is already structured
    const nounType = noun.metadata.noun

    // ⚠️ Fallback pattern for robustness
    const nounType = noun.metadata?.noun || 'thing'
    ```

11. **Respect the two-file storage pattern** - Don't mix vector and metadata operations:
    ```typescript
    // ✅ Good - Separate concerns
    await storage.saveNoun(noun)
    await storage.saveMetadata(noun.id, metadata)

    // ❌ Bad - Mixing concerns
    await storage.saveNounWithEverything(combinedData)
    ```

## Testing Your Augmentation

```typescript
import { Brainy } from 'brainy'
import { MyAugmentation } from './my-augmentation'

describe('MyAugmentation', () => {
  let brain: Brainy
  
  beforeEach(async () => {
    brain = new Brainy()
    brain.augmentations.register(new MyAugmentation())
    await brain.init()
  })
  
  afterEach(async () => {
    await brain.destroy()
  })
  
  it('should enhance searches', async () => {
    // Test your augmentation's effect
    const results = await brain.search('test')
    expect(results).toHaveProperty('enhanced', true)
  })
})
```

## Summary

Augmentations are Brainy's extension system. They can:
- Replace storage backends
- Add caching layers
- Implement audit trails
- Add rate limiting
- Sync with external systems
- Transform data
- And much more!

The unified BrainyAugmentation interface makes it easy to create powerful extensions while maintaining consistency across the entire system.