brainy/docs/augmentations/api-server.md

403 lines
8.5 KiB
Markdown
Raw Permalink Normal View History

🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ MAJOR RELEASE: Complete evolution of Brainy with groundbreaking features and performance. 🎯 KEY FEATURES: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ✨ Triple Intelligence™ Engine - Unified Vector + Metadata + Graph search - O(log n) performance on all operations - 3ms average search latency at any scale ✨ API Consolidation - 15+ search methods → 2 clean APIs - search() for vector similarity - find() for natural language queries ✨ Natural Language Processing - 220+ pre-computed NLP patterns - Instant context understanding - "Show me recent React components with tests" ✨ Zero Configuration - Works instantly, no setup required - Built-in embedding models (no API keys) - Smart defaults for everything - Automatic optimization ✨ Enterprise Features (Free for Everyone) - Scales to 10M+ items - Write-Ahead Logging (WAL) for durability - Distributed architecture with sharding - Read/write separation - Connection pooling & request deduplication - Built-in monitoring & health checks ✨ Universal Compatibility - Node.js, Browser, Edge Workers - 4 Storage Adapters (Memory, FileSystem, OPFS, S3) - TypeScript with full type safety - Worker-based embeddings 📦 WHAT'S INCLUDED: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • Core AI Database with HNSW indexing • 19 Production-ready augmentations • Universal Memory Manager • Complete CLI with all commands • Brain Cloud integration (soulcraft.com) • Comprehensive documentation • 52 test files with 400+ tests • Migration guide from 1.x 📊 PERFORMANCE: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • Initialize: 450ms (24MB memory) • Search: 3ms average (up to 10M items) • Metadata Filter: 0.8ms (O(log n)) • Bulk Import: 2.3s per 1000 items • Production Scale: 5.8ms at 10M items 🔧 TECHNICAL IMPROVEMENTS: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • TypeScript compilation: 153 errors → 0 • Memory usage: 200MB → 24MB baseline • Circular dependencies resolved • Worker thread communication fixed • Storage adapter consistency • Request coalescing for 3x performance 🛠️ CLI FEATURES: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • brainy add - Smart data ingestion • brainy find - Natural language search • brainy search - Vector similarity • brainy chat - AI conversation mode • brainy cloud - Brain Cloud integration • brainy augment - Manage extensions • 100% API compatibility 📚 DOCUMENTATION: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • Professional README with examples • Quick Start guide (5 minutes) • Enterprise Features guide • Migration guide from 1.x • API reference • Architecture documentation 🌟 USE CASES: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • AI memory layer for chatbots • Semantic document search • Code intelligence platforms • Knowledge management systems • Real-time recommendation engines • Customer support automation MIT License - Enterprise features included free for everyone. No premium tiers, no paywalls, no limits. Built with ❤️ by the Brainy community. Visit https://soulcraft.com for Brain Cloud integration.
2025-08-26 12:32:21 -07:00
# API Server Augmentation
## Overview
The `APIServerAugmentation` is a powerful augmentation that exposes your Brainy instance through REST, WebSocket, and MCP (Model Context Protocol) APIs. It transforms Brainy into a full-featured API server with zero configuration required.
## Features
### 🌐 REST API
Complete CRUD operations and advanced queries through HTTP endpoints.
### 🔌 WebSocket Server
Real-time bidirectional communication with automatic operation broadcasting.
### 🧠 MCP Integration
Built-in Model Context Protocol support for AI agent communication.
### 📊 Operation Broadcasting
Automatically broadcasts all Brainy operations to subscribed WebSocket clients.
### 🔒 Optional Security
Built-in authentication and rate limiting when needed.
## Installation
The APIServerAugmentation is included in Brainy core. No additional installation required.
For Node.js environments, you may want to install optional dependencies:
```bash
npm install express cors ws
```
## Zero-Config Usage
```typescript
import { BrainyData } from 'brainy'
import { APIServerAugmentation } from 'brainy/augmentations'
const brain = new BrainyData()
// Register the API server augmentation
brain.augmentations.register(new APIServerAugmentation())
await brain.init()
// Server is now running at http://localhost:3000
console.log('API Server ready!')
console.log('REST: http://localhost:3000/api/*')
console.log('WebSocket: ws://localhost:3000/ws')
console.log('MCP: http://localhost:3000/api/mcp')
```
## Configuration Options
While zero-config works great, you can customize the server:
```typescript
const apiServer = new APIServerAugmentation({
enabled: true, // Enable/disable the server
port: 3000, // HTTP port
host: '0.0.0.0', // Bind address
cors: {
origin: '*', // CORS allowed origins
credentials: true // Allow credentials
},
auth: {
required: false, // Require authentication
apiKeys: [], // Valid API keys
bearerTokens: [] // Valid bearer tokens
},
rateLimit: {
windowMs: 60000, // Rate limit window (ms)
max: 100 // Max requests per window
}
})
```
## REST API Endpoints
### Health Check
```http
GET /health
```
Returns server status and basic metrics.
### Search
```http
POST /api/search
Content-Type: application/json
{
"query": "search text",
"limit": 10,
"options": {}
}
```
### Add Data
```http
POST /api/add
Content-Type: application/json
{
"content": "data to add",
"metadata": {
"key": "value"
}
}
```
### Get by ID
```http
GET /api/get/:id
```
### Delete
```http
DELETE /api/delete/:id
```
### Create Relationship
```http
POST /api/relate
Content-Type: application/json
{
"source": "id1",
"target": "id2",
"verb": "relates_to",
"metadata": {}
}
```
### Complex Queries
```http
POST /api/find
Content-Type: application/json
{
"where": { "type": "document" },
"like": "machine learning",
"limit": 10
}
```
### Clustering
```http
POST /api/cluster
Content-Type: application/json
{
"algorithm": "kmeans",
"options": {
"k": 5
}
}
```
### Statistics
```http
GET /api/stats
```
### Operation History
```http
GET /api/history
```
## WebSocket API
### Connection
```javascript
const ws = new WebSocket('ws://localhost:3000/ws')
ws.onopen = () => {
console.log('Connected to Brainy WebSocket')
}
ws.onmessage = (event) => {
const msg = JSON.parse(event.data)
console.log('Received:', msg)
}
```
### Subscribe to Operations
```javascript
ws.send(JSON.stringify({
type: 'subscribe',
operations: ['all'] // or specific: ['add', 'search', 'delete']
}))
```
### Search via WebSocket
```javascript
ws.send(JSON.stringify({
type: 'search',
query: 'your search',
limit: 10,
requestId: 'unique-id'
}))
```
### Add Data via WebSocket
```javascript
ws.send(JSON.stringify({
type: 'add',
content: 'data to add',
metadata: {},
requestId: 'unique-id'
}))
```
### Operation Broadcasts
When subscribed, you'll receive real-time updates:
```javascript
{
"type": "operation",
"operation": "add",
"params": { /* sanitized parameters */ },
"timestamp": 1234567890,
"duration": 15
}
```
## MCP (Model Context Protocol)
The MCP endpoint allows AI agents to interact with Brainy:
```http
POST /api/mcp
Content-Type: application/json
{
"method": "search",
"params": {
"query": "find documents about AI"
}
}
```
## Authentication
When authentication is enabled:
### API Key
```http
GET /api/stats
X-API-Key: your-api-key
```
### Bearer Token
```http
GET /api/stats
Authorization: Bearer your-token
```
## Environment Support
### Node.js ✅
Full support with Express, WebSocket, and all features.
### Deno 🚧
Planned support using Deno.serve() or oak framework.
### Browser/Service Worker 🚧
Planned support for intercepting fetch() calls locally.
## How It Works
The APIServerAugmentation hooks into Brainy's augmentation pipeline:
1. **Timing**: Executes `after` operations complete
2. **Operations**: Monitors `all` operations
3. **Broadcasting**: Sends operation details to subscribed clients
4. **History**: Maintains operation history (last 1000 operations)
## Example: Multi-Client Sync
```typescript
// Server
const brain = new BrainyData()
brain.augmentations.register(new APIServerAugmentation())
await brain.init()
// Client 1 - WebSocket subscriber
const ws1 = new WebSocket('ws://localhost:3000/ws')
ws1.onopen = () => {
ws1.send(JSON.stringify({
type: 'subscribe',
operations: ['add', 'delete']
}))
}
ws1.onmessage = (e) => {
console.log('Client 1 received update:', JSON.parse(e.data))
}
// Client 2 - REST API user
fetch('http://localhost:3000/api/add', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
content: 'New data',
metadata: { source: 'client2' }
})
})
// Client 1 automatically receives notification!
```
## Performance Considerations
- **Operation History**: Limited to last 1000 operations
- **WebSocket Heartbeat**: Every 30 seconds
- **Client Timeout**: 60 seconds of inactivity
- **Parameter Sanitization**: Sensitive fields removed, large content truncated
- **Rate Limiting**: In-memory tracking (use Redis in production)
## Security Notes
1. **Default Configuration**: No auth, open CORS - suitable for development
2. **Production**: Enable auth, configure CORS, use HTTPS
3. **Sensitive Data**: Parameters are sanitized before broadcasting
4. **Rate Limiting**: Basic in-memory implementation included
## Comparison with Previous Implementations
The APIServerAugmentation unifies and replaces:
- `BrainyMCPBroadcast` - Node-specific WebSocket/HTTP server
- `WebSocketConduitAugmentation` - WebSocket client functionality
- `ServerSearchAugmentations` - Remote Brainy connections
Benefits of the unified approach:
- Single augmentation for all API needs
- Consistent interface across protocols
- Automatic operation broadcasting
- Environment-aware implementation
- Zero-configuration philosophy
## Advanced Usage
### Custom Operation Filtering
```typescript
class FilteredAPIServer extends APIServerAugmentation {
shouldExecute(operation: string, params: any): boolean {
// Don't broadcast sensitive operations
if (operation === 'delete' && params.sensitive) {
return false
}
return true
}
}
```
### Integration with Other Augmentations
```typescript
const brain = new BrainyData()
// Stack augmentations for complete system
brain.augmentations.register(new EntityRegistryAugmentation()) // Dedup
brain.augmentations.register(new APIServerAugmentation()) // API
await brain.init()
// All augmentations work together seamlessly!
```
## Troubleshooting
### Server won't start
- Check if port is already in use
- Verify Node.js dependencies are installed: `npm install express cors ws`
- Check console for error messages
### WebSocket connections drop
- Ensure heartbeat responses are handled
- Check for proxy/firewall issues
- Verify CORS configuration
### Authentication not working
- Ensure `auth.required` is set to `true`
- Verify API keys or bearer tokens are correctly configured
- Check request headers are properly formatted
## Future Enhancements
- [ ] Deno server implementation
- [ ] Service Worker implementation
- [ ] GraphQL endpoint
- [ ] gRPC support
- [ ] Built-in SSL/TLS
- [ ] Redis-based rate limiting
- [ ] Prometheus metrics endpoint
- [ ] OpenAPI/Swagger documentation
## Related Documentation
- [Augmentation System Overview](../AUGMENTATION-SYSTEM.md)
- [BrainyAugmentation Interface](./brainy-augmentation.md)
- [MCP Integration](../mcp/README.md)
- [Zero-Config Philosophy](../ZERO-CONFIG.md)