brainy/docs/augmentations/api-server.md
David Snelling 292a9f9c42 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™
MAJOR RELEASE: Complete evolution of Brainy with groundbreaking features and performance.

🎯 KEY FEATURES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Triple Intelligence™ Engine
  - Unified Vector + Metadata + Graph search
  - O(log n) performance on all operations
  - 3ms average search latency at any scale

 API Consolidation
  - 15+ search methods → 2 clean APIs
  - search() for vector similarity
  - find() for natural language queries

 Natural Language Processing
  - 220+ pre-computed NLP patterns
  - Instant context understanding
  - "Show me recent React components with tests"

 Zero Configuration
  - Works instantly, no setup required
  - Built-in embedding models (no API keys)
  - Smart defaults for everything
  - Automatic optimization

 Enterprise Features (Free for Everyone)
  - Scales to 10M+ items
  - Write-Ahead Logging (WAL) for durability
  - Distributed architecture with sharding
  - Read/write separation
  - Connection pooling & request deduplication
  - Built-in monitoring & health checks

 Universal Compatibility
  - Node.js, Browser, Edge Workers
  - 4 Storage Adapters (Memory, FileSystem, OPFS, S3)
  - TypeScript with full type safety
  - Worker-based embeddings

📦 WHAT'S INCLUDED:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Core AI Database with HNSW indexing
• 19 Production-ready augmentations
• Universal Memory Manager
• Complete CLI with all commands
• Brain Cloud integration (soulcraft.com)
• Comprehensive documentation
• 52 test files with 400+ tests
• Migration guide from 1.x

📊 PERFORMANCE:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Initialize: 450ms (24MB memory)
• Search: 3ms average (up to 10M items)
• Metadata Filter: 0.8ms (O(log n))
• Bulk Import: 2.3s per 1000 items
• Production Scale: 5.8ms at 10M items

🔧 TECHNICAL IMPROVEMENTS:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• TypeScript compilation: 153 errors → 0
• Memory usage: 200MB → 24MB baseline
• Circular dependencies resolved
• Worker thread communication fixed
• Storage adapter consistency
• Request coalescing for 3x performance

🛠️ CLI FEATURES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• brainy add - Smart data ingestion
• brainy find - Natural language search
• brainy search - Vector similarity
• brainy chat - AI conversation mode
• brainy cloud - Brain Cloud integration
• brainy augment - Manage extensions
• 100% API compatibility

📚 DOCUMENTATION:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Professional README with examples
• Quick Start guide (5 minutes)
• Enterprise Features guide
• Migration guide from 1.x
• API reference
• Architecture documentation

🌟 USE CASES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• AI memory layer for chatbots
• Semantic document search
• Code intelligence platforms
• Knowledge management systems
• Real-time recommendation engines
• Customer support automation

MIT License - Enterprise features included free for everyone.
No premium tiers, no paywalls, no limits.

Built with ❤️ by the Brainy community.
Visit https://soulcraft.com for Brain Cloud integration.
2025-08-26 12:32:21 -07:00

8.6 KiB

API Server Augmentation

Overview

The APIServerAugmentation is a powerful augmentation that exposes your Brainy instance through REST, WebSocket, and MCP (Model Context Protocol) APIs. It transforms Brainy into a full-featured API server with zero configuration required.

Features

🌐 REST API

Complete CRUD operations and advanced queries through HTTP endpoints.

🔌 WebSocket Server

Real-time bidirectional communication with automatic operation broadcasting.

🧠 MCP Integration

Built-in Model Context Protocol support for AI agent communication.

📊 Operation Broadcasting

Automatically broadcasts all Brainy operations to subscribed WebSocket clients.

🔒 Optional Security

Built-in authentication and rate limiting when needed.

Installation

The APIServerAugmentation is included in Brainy core. No additional installation required.

For Node.js environments, you may want to install optional dependencies:

npm install express cors ws

Zero-Config Usage

import { BrainyData } from 'brainy'
import { APIServerAugmentation } from 'brainy/augmentations'

const brain = new BrainyData()

// Register the API server augmentation
brain.augmentations.register(new APIServerAugmentation())

await brain.init()

// Server is now running at http://localhost:3000
console.log('API Server ready!')
console.log('REST: http://localhost:3000/api/*')
console.log('WebSocket: ws://localhost:3000/ws')
console.log('MCP: http://localhost:3000/api/mcp')

Configuration Options

While zero-config works great, you can customize the server:

const apiServer = new APIServerAugmentation({
  enabled: true,           // Enable/disable the server
  port: 3000,             // HTTP port
  host: '0.0.0.0',        // Bind address
  
  cors: {
    origin: '*',          // CORS allowed origins
    credentials: true     // Allow credentials
  },
  
  auth: {
    required: false,      // Require authentication
    apiKeys: [],         // Valid API keys
    bearerTokens: []     // Valid bearer tokens
  },
  
  rateLimit: {
    windowMs: 60000,     // Rate limit window (ms)
    max: 100            // Max requests per window
  }
})

REST API Endpoints

Health Check

GET /health

Returns server status and basic metrics.

POST /api/search
Content-Type: application/json

{
  "query": "search text",
  "limit": 10,
  "options": {}
}

Add Data

POST /api/add
Content-Type: application/json

{
  "content": "data to add",
  "metadata": {
    "key": "value"
  }
}

Get by ID

GET /api/get/:id

Delete

DELETE /api/delete/:id

Create Relationship

POST /api/relate
Content-Type: application/json

{
  "source": "id1",
  "target": "id2",
  "verb": "relates_to",
  "metadata": {}
}

Complex Queries

POST /api/find
Content-Type: application/json

{
  "where": { "type": "document" },
  "like": "machine learning",
  "limit": 10
}

Clustering

POST /api/cluster
Content-Type: application/json

{
  "algorithm": "kmeans",
  "options": {
    "k": 5
  }
}

Statistics

GET /api/stats

Operation History

GET /api/history

WebSocket API

Connection

const ws = new WebSocket('ws://localhost:3000/ws')

ws.onopen = () => {
  console.log('Connected to Brainy WebSocket')
}

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data)
  console.log('Received:', msg)
}

Subscribe to Operations

ws.send(JSON.stringify({
  type: 'subscribe',
  operations: ['all']  // or specific: ['add', 'search', 'delete']
}))

Search via WebSocket

ws.send(JSON.stringify({
  type: 'search',
  query: 'your search',
  limit: 10,
  requestId: 'unique-id'
}))

Add Data via WebSocket

ws.send(JSON.stringify({
  type: 'add',
  content: 'data to add',
  metadata: {},
  requestId: 'unique-id'
}))

Operation Broadcasts

When subscribed, you'll receive real-time updates:

{
  "type": "operation",
  "operation": "add",
  "params": { /* sanitized parameters */ },
  "timestamp": 1234567890,
  "duration": 15
}

MCP (Model Context Protocol)

The MCP endpoint allows AI agents to interact with Brainy:

POST /api/mcp
Content-Type: application/json

{
  "method": "search",
  "params": {
    "query": "find documents about AI"
  }
}

Authentication

When authentication is enabled:

API Key

GET /api/stats
X-API-Key: your-api-key

Bearer Token

GET /api/stats
Authorization: Bearer your-token

Environment Support

Node.js

Full support with Express, WebSocket, and all features.

Deno 🚧

Planned support using Deno.serve() or oak framework.

Browser/Service Worker 🚧

Planned support for intercepting fetch() calls locally.

How It Works

The APIServerAugmentation hooks into Brainy's augmentation pipeline:

  1. Timing: Executes after operations complete
  2. Operations: Monitors all operations
  3. Broadcasting: Sends operation details to subscribed clients
  4. History: Maintains operation history (last 1000 operations)

Example: Multi-Client Sync

// Server
const brain = new BrainyData()
brain.augmentations.register(new APIServerAugmentation())
await brain.init()

// Client 1 - WebSocket subscriber
const ws1 = new WebSocket('ws://localhost:3000/ws')
ws1.onopen = () => {
  ws1.send(JSON.stringify({
    type: 'subscribe',
    operations: ['add', 'delete']
  }))
}
ws1.onmessage = (e) => {
  console.log('Client 1 received update:', JSON.parse(e.data))
}

// Client 2 - REST API user
fetch('http://localhost:3000/api/add', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    content: 'New data',
    metadata: { source: 'client2' }
  })
})
// Client 1 automatically receives notification!

Performance Considerations

  • Operation History: Limited to last 1000 operations
  • WebSocket Heartbeat: Every 30 seconds
  • Client Timeout: 60 seconds of inactivity
  • Parameter Sanitization: Sensitive fields removed, large content truncated
  • Rate Limiting: In-memory tracking (use Redis in production)

Security Notes

  1. Default Configuration: No auth, open CORS - suitable for development
  2. Production: Enable auth, configure CORS, use HTTPS
  3. Sensitive Data: Parameters are sanitized before broadcasting
  4. Rate Limiting: Basic in-memory implementation included

Comparison with Previous Implementations

The APIServerAugmentation unifies and replaces:

  • BrainyMCPBroadcast - Node-specific WebSocket/HTTP server
  • WebSocketConduitAugmentation - WebSocket client functionality
  • ServerSearchAugmentations - Remote Brainy connections

Benefits of the unified approach:

  • Single augmentation for all API needs
  • Consistent interface across protocols
  • Automatic operation broadcasting
  • Environment-aware implementation
  • Zero-configuration philosophy

Advanced Usage

Custom Operation Filtering

class FilteredAPIServer extends APIServerAugmentation {
  shouldExecute(operation: string, params: any): boolean {
    // Don't broadcast sensitive operations
    if (operation === 'delete' && params.sensitive) {
      return false
    }
    return true
  }
}

Integration with Other Augmentations

const brain = new BrainyData()

// Stack augmentations for complete system
brain.augmentations.register(new WALAugmentation())           // Durability
brain.augmentations.register(new EntityRegistryAugmentation()) // Dedup
brain.augmentations.register(new APIServerAugmentation())      // API

await brain.init()
// All augmentations work together seamlessly!

Troubleshooting

Server won't start

  • Check if port is already in use
  • Verify Node.js dependencies are installed: npm install express cors ws
  • Check console for error messages

WebSocket connections drop

  • Ensure heartbeat responses are handled
  • Check for proxy/firewall issues
  • Verify CORS configuration

Authentication not working

  • Ensure auth.required is set to true
  • Verify API keys or bearer tokens are correctly configured
  • Check request headers are properly formatted

Future Enhancements

  • Deno server implementation
  • Service Worker implementation
  • GraphQL endpoint
  • gRPC support
  • Built-in SSL/TLS
  • Redis-based rate limiting
  • Prometheus metrics endpoint
  • OpenAPI/Swagger documentation