brainy/docs/augmentations/CONFIGURATION.md
David Snelling 2a94fca875 feat: Brainy 3.0 - Production-ready Triple Intelligence database
Major improvements and simplifications:
- Simplified to Q8-only model precision (99% accuracy, 75% smaller)
- Removed WAL augmentation (not needed with modern filesystems)
- Eliminated all fake/stub code - 100% production-ready
- Added comprehensive cloud deployment support (Docker, K8s, AWS, GCP)
- Enhanced distributed system capabilities
- Improved Triple Intelligence find() implementation
- Added streaming pipeline for large-scale operations
- Comprehensive test coverage with new test suites

Breaking changes:
- Renamed BrainyData to Brainy (simpler, cleaner)
- Removed FP32 model option (Q8 provides 99% accuracy)
- Removed deprecated augmentations

Performance improvements:
- 10x faster initialization with Q8-only
- Reduced memory footprint by 75%
- Better scaling for millions of items

Co-Authored-By: Recovery checkpoint system
2025-09-11 16:23:32 -07:00

13 KiB
Raw Permalink Blame History

Augmentation Configuration System

Version: 2.0.0
Status: Production Ready

Overview

The Brainy Augmentation Configuration System provides a VSCode-style extension architecture with multiple configuration sources, schema validation, and tool discovery. This system maintains Brainy's zero-config philosophy while enabling sophisticated enterprise configuration management.

Table of Contents

Quick Start

Using an Augmentation with Configuration

import { BrainyData } from '@soulcraft/brainy'

// Zero-config (uses defaults)
const brain = new BrainyData()

// With custom configuration
  immediateWrites: true,
  checkpointInterval: 300000 // 5 minutes
}))

Configuring via Environment Variables

export BRAINY_AUG_CACHE_TTL=600000

Configuring via Files

Create a .brainyrc file in your project root:

{
  "augmentations": {
    "wal": {
      "enabled": true,
      "immediateWrites": true,
      "maxSize": 20971520
    },
    "cache": {
      "ttl": 600000,
      "maxSize": 2000
    }
  }
}

Configuration Sources

Configuration is resolved in the following priority order (highest to lowest):

  1. Runtime Updates - Dynamic configuration changes via API
  2. Constructor Parameters - Code-time configuration
  3. Environment Variables - BRAINY_AUG_<NAME>_<KEY>
  4. Configuration Files - .brainyrc, brainy.config.json
  5. Schema Defaults - Default values from manifest

Resolution Example

// Schema default
{ maxSize: 10485760 }

// File configuration (.brainyrc)
{ maxSize: 20971520 }

// Environment variable

// Constructor parameter

// Final resolved value: 41943040 (constructor wins)

Creating Configurable Augmentations

Step 1: Extend ConfigurableAugmentation

import { ConfigurableAugmentation, AugmentationManifest } from '@soulcraft/brainy'

export class MyAugmentation extends ConfigurableAugmentation {
  name = 'my-augmentation'
  timing = 'around' as const
  metadata = 'none' as const
  operations = ['search', 'add']
  priority = 50
  
  constructor(config?: MyConfig) {
    super(config) // Handles configuration resolution
  }
  
  // Required: Provide manifest for discovery
  getManifest(): AugmentationManifest {
    return {
      id: 'my-augmentation',
      name: 'My Augmentation',
      version: '1.0.0',
      description: 'Does something amazing',
      category: 'performance',
      configSchema: {
        type: 'object',
        properties: {
          enabled: {
            type: 'boolean',
            default: true,
            description: 'Enable this augmentation'
          },
          threshold: {
            type: 'number',
            default: 100,
            minimum: 1,
            maximum: 1000,
            description: 'Processing threshold'
          }
        }
      }
    }
  }
  
  // Optional: Handle runtime configuration changes
  protected async onConfigChange(newConfig: MyConfig, oldConfig: MyConfig): Promise<void> {
    if (newConfig.threshold !== oldConfig.threshold) {
      // React to threshold change
      this.updateThreshold(newConfig.threshold)
    }
  }
  
  async execute<T>(operation: string, params: any, next: () => Promise<T>): Promise<T> {
    if (!this.config.enabled) {
      return next()
    }
    
    // Your augmentation logic here
    return next()
  }
}

Step 2: Define Configuration Interface

interface MyConfig {
  enabled?: boolean
  threshold?: number
  mode?: 'fast' | 'balanced' | 'thorough'
}

Step 3: Add JSON Schema in Manifest

configSchema: {
  type: 'object',
  properties: {
    enabled: {
      type: 'boolean',
      default: true,
      description: 'Enable augmentation'
    },
    threshold: {
      type: 'number',
      default: 100,
      minimum: 1,
      maximum: 1000,
      description: 'Processing threshold'
    },
    mode: {
      type: 'string',
      default: 'balanced',
      enum: ['fast', 'balanced', 'thorough'],
      description: 'Processing mode'
    }
  },
  required: [],
  additionalProperties: false
}

Configuration Discovery

The Discovery API allows tools to discover and configure augmentations dynamically:

import { AugmentationDiscovery } from '@soulcraft/brainy'

const discovery = new AugmentationDiscovery(brain.augmentations)

// Discover all augmentations with manifests
const listings = await discovery.discover({
  includeConfig: true,
  includeSchema: true
})

// Get configuration schema
const schema = await discovery.getConfigSchema('wal')

// Validate configuration
const validation = await discovery.validateConfig('wal', {
  enabled: true,
  maxSize: 'invalid' // Will fail validation
})

// Update configuration at runtime
await discovery.updateConfig('wal', {
  checkpointInterval: 120000
})

Runtime Configuration

Update Configuration Dynamically

// Get augmentation
const wal = brain.augmentations.get('wal')

// Update configuration
await wal.updateConfig({
  checkpointInterval: 300000
})

// Get current configuration
const config = wal.getConfig()

React to Configuration Changes

class MyAugmentation extends ConfigurableAugmentation {
  protected async onConfigChange(newConfig: any, oldConfig: any): Promise<void> {
    // Stop old processes
    if (oldConfig.enabled && !newConfig.enabled) {
      await this.stop()
    }
    
    // Start new processes
    if (!oldConfig.enabled && newConfig.enabled) {
      await this.start()
    }
    
    // Update settings
    if (newConfig.interval !== oldConfig.interval) {
      this.rescheduleTimer(newConfig.interval)
    }
  }
}

Environment Variables

Naming Convention

BRAINY_AUG_<AUGMENTATION_ID>_<CONFIG_KEY>=value

Examples


# Cache augmentation
BRAINY_AUG_CACHE_ENABLED=true
BRAINY_AUG_CACHE_MAX_SIZE=2000
BRAINY_AUG_CACHE_TTL=600000

# Complex values (JSON)
BRAINY_AUG_MYAUG_FILTERS='["*.js","*.ts"]'
BRAINY_AUG_MYAUG_OPTIONS='{"deep":true,"follow":false}'

Docker Example

ENV BRAINY_AUG_CACHE_TTL=600000

Configuration Files

File Locations (Priority Order)

  1. .brainyrc (current directory)
  2. .brainyrc.json (current directory)
  3. brainy.config.json (current directory)
  4. ~/.brainy/config.json (user home)
  5. ~/.brainyrc (user home)

File Format

{
  "augmentations": {
    "wal": {
      "enabled": true,
      "immediateWrites": true,
      "maxSize": 20971520,
      "checkpointInterval": 300000
    },
    "cache": {
      "enabled": true,
      "maxSize": 2000,
      "ttl": 600000
    },
    "metrics": {
      "enabled": false
    }
  }
}

Per-Environment Configuration

{
  "augmentations": {
    "wal": {
      "development": {
        "enabled": true,
        "immediateWrites": true,
        "maxSize": 5242880
      },
      "production": {
        "enabled": true,
        "immediateWrites": false,
        "maxSize": 104857600,
        "checkpointInterval": 60000
      }
    }
  }
}

CLI Commands

List Augmentations with Configuration

# Show all augmentations with config status
brainy augment list --detailed

# Show configuration for specific augmentation
brainy augment config wal

# Set configuration value
brainy augment config wal --set immediateWrites=true

# Show environment variable names
brainy augment config wal --env

# Export configuration schema
brainy augment schema wal > wal-schema.json

# Validate configuration file
brainy augment validate --file config.json

Interactive Configuration

# Interactive configuration wizard
brainy augment configure wal

? Operation mode? 
   Performance (immediate writes)
    Durability (synchronous writes)
    Custom
? Maximum log size? (10MB) 20MB
? Checkpoint interval? (1 minute) 5 minutes

Configuration saved to .brainyrc

Tool Integration

Brain-Cloud Explorer UI

// Auto-generate configuration form from schema
const ConfigurationUI = ({ augmentationId }) => {
  const [manifest, setManifest] = useState(null)
  const [config, setConfig] = useState({})
  
  useEffect(() => {
    // Fetch manifest with schema
    fetch(`/api/augmentations/${augmentationId}/manifest`)
      .then(res => res.json())
      .then(setManifest)
    
    // Get current configuration
    discovery.getConfig(augmentationId)
      .then(setConfig)
  }, [augmentationId])
  
  const handleSave = async (newConfig) => {
    // Validate configuration
    const validation = await fetch(`/api/augmentations/${augmentationId}/validate`, {
      method: 'POST',
      body: JSON.stringify(newConfig)
    }).then(res => res.json())
    
    if (validation.valid) {
      // Apply configuration
      await discovery.updateConfig(augmentationId, newConfig)
    }
  }
  
  // Render form based on schema
  return <SchemaForm 
    schema={manifest?.configSchema}
    values={config}
    onSubmit={handleSave}
  />
}

VS Code Extension

// package.json contribution points
{
  "contributes": {
    "configuration": {
      "title": "Brainy Augmentations",
      "properties": {
        "brainy.augmentations.wal.enabled": {
          "type": "boolean",
          "default": true,
        },
        "brainy.augmentations.wal.maxSize": {
          "type": "number",
          "default": 10485760,
        }
      }
    }
  }
}

Migration Guide

Migrating from BaseAugmentation

Before:

export class MyAugmentation extends BaseAugmentation {
  constructor(config: MyConfig = {}) {
    super()
    this.config = {
      enabled: config.enabled ?? true,
      threshold: config.threshold ?? 100
    }
  }
  
  // No manifest
  // No config discovery
  // No runtime updates
}

After:

export class MyAugmentation extends ConfigurableAugmentation {
  constructor(config?: MyConfig) {
    super(config) // Config resolution handled automatically
  }
  
  getManifest(): AugmentationManifest {
    return {
      id: 'my-augmentation',
      name: 'My Augmentation',
      version: '1.0.0',
      description: 'Does something amazing',
      category: 'performance',
      configSchema: {
        type: 'object',
        properties: {
          enabled: { type: 'boolean', default: true },
          threshold: { type: 'number', default: 100 }
        }
      }
    }
  }
  
  // Optional: Handle config changes
  protected async onConfigChange(newConfig: MyConfig, oldConfig: MyConfig): Promise<void> {
    // React to changes
  }
}

Backwards Compatibility

The system maintains full backwards compatibility:

  1. BaseAugmentation still works - Existing augmentations continue to function
  2. Constructor config still works - Existing configuration patterns preserved
  3. Zero-config still works - Defaults are applied automatically
  4. Progressive enhancement - Add features as needed

Best Practices

1. Always Provide Defaults

configSchema: {
  properties: {
    enabled: {
      type: 'boolean',
      default: true, // Always provide defaults
      description: 'Enable this feature'
    }
  }
}

2. Use Descriptive Configuration Keys

// Good
checkpointInterval: 60000

// Bad
ci: 60000

3. Validate Configuration

protected async onConfigChange(newConfig: any, oldConfig: any): Promise<void> {
  // Validate before applying
  if (newConfig.maxSize < 1048576) {
    throw new Error('maxSize must be at least 1MB')
  }
  
  // Apply changes
  this.maxSize = newConfig.maxSize
}

4. Document Environment Variables

/**
 * Environment Variables:
 * - BRAINY_AUG_MYAUG_ENABLED: Enable augmentation (boolean)
 * - BRAINY_AUG_MYAUG_THRESHOLD: Processing threshold (number)
 * - BRAINY_AUG_MYAUG_MODE: Processing mode (fast|balanced|thorough)
 */

5. Provide Configuration Examples

configExamples: [
  {
    name: 'Production',
    description: 'Optimized for production use',
    config: {
      enabled: true,
      mode: 'thorough',
      threshold: 500
    }
  },
  {
    name: 'Development',
    description: 'Lightweight for development',
    config: {
      enabled: true,
      mode: 'fast',
      threshold: 10
    }
  }
]

Troubleshooting

Configuration Not Loading

  1. Check file locations and names
  2. Verify JSON syntax in config files
  3. Check environment variable names (case-sensitive)
  4. Use brainy augment config <name> --debug to see resolution

Validation Errors

  1. Check schema requirements
  2. Verify data types match schema
  3. Check minimum/maximum constraints
  4. Use discovery API to validate before applying

Runtime Updates Not Working

  1. Ensure augmentation extends ConfigurableAugmentation
  2. Implement onConfigChange if needed
  3. Check for validation errors
  4. Verify augmentation is initialized

API Reference

See the Discovery API Documentation for complete API details.

Examples

See the examples directory for complete working examples.