docs: consolidate and archive redundant documentation
- Archived 13 API design iterations to docs/api-design-archive/ - Consolidated augmentation docs to docs/augmentations-archive/ - Maintained ONE definitive API doc at docs/api/README.md - Cleaned up documentation structure for 2.0 release - Preserved all historical documents for reference
This commit is contained in:
parent
ef8f35ec2a
commit
994276f09f
35 changed files with 444 additions and 239 deletions
|
|
@ -1,549 +0,0 @@
|
|||
# 🌐 Brainy API Exposure Architecture
|
||||
|
||||
## 📡 Current State: Built-in MCP Support
|
||||
|
||||
Brainy **already has** Model Context Protocol (MCP) support built-in:
|
||||
|
||||
```typescript
|
||||
// Already exists in Brainy!
|
||||
import { BrainyMCPService } from 'brainy/mcp'
|
||||
|
||||
const brain = new BrainyData()
|
||||
const mcpService = new BrainyMCPService(brain)
|
||||
|
||||
// Handles MCP requests
|
||||
const response = await mcpService.handleRequest({
|
||||
type: 'data_access',
|
||||
operation: 'search',
|
||||
parameters: { query: 'find documents' }
|
||||
})
|
||||
```
|
||||
|
||||
### What's Already Built:
|
||||
- **BrainyMCPAdapter** - Exposes data operations
|
||||
- **MCPAugmentationToolset** - Exposes augmentations as MCP tools
|
||||
- **BrainyMCPService** - Unified service layer
|
||||
- **ServerSearchConduitAugmentation** - Connect to remote Brainy instances
|
||||
|
||||
## 🚀 The Missing Piece: API Server Augmentation
|
||||
|
||||
What's **NOT** built yet is a unified API server that exposes REST, WebSocket, and MCP over network. This SHOULD be an augmentation!
|
||||
|
||||
```typescript
|
||||
/**
|
||||
* Universal API Server Augmentation
|
||||
* Exposes Brainy through REST, WebSocket, MCP, and GraphQL
|
||||
*/
|
||||
export class APIServerAugmentation extends BaseAugmentation {
|
||||
readonly name = 'api-server'
|
||||
readonly timing = 'after' as const
|
||||
readonly operations = ['all'] as const // Monitor all operations
|
||||
readonly priority = 5 // Low priority, runs last
|
||||
|
||||
private httpServer?: any
|
||||
private wsServer?: any
|
||||
private mcpService?: BrainyMCPService
|
||||
private clients = new Set<any>()
|
||||
private apiKeys = new Map<string, any>()
|
||||
|
||||
protected async onInitialize(): Promise<void> {
|
||||
const config = this.context.config.apiServer || {}
|
||||
|
||||
if (!config.enabled) {
|
||||
this.log('API Server disabled in config')
|
||||
return
|
||||
}
|
||||
|
||||
// Initialize MCP service
|
||||
this.mcpService = new BrainyMCPService(this.context.brain, {
|
||||
enableAuth: config.requireAuth
|
||||
})
|
||||
|
||||
// Start servers based on environment
|
||||
if (typeof process !== 'undefined' && process.versions?.node) {
|
||||
await this.startNodeServers(config)
|
||||
} else if (typeof Deno !== 'undefined') {
|
||||
await this.startDenoServer(config)
|
||||
} else if (typeof self !== 'undefined') {
|
||||
await this.startServiceWorker(config)
|
||||
}
|
||||
}
|
||||
|
||||
private async startNodeServers(config: any) {
|
||||
const express = await import('express')
|
||||
const { WebSocketServer } = await import('ws')
|
||||
const cors = await import('cors')
|
||||
|
||||
const app = express.default()
|
||||
|
||||
// Middleware
|
||||
app.use(cors.default(config.cors))
|
||||
app.use(express.json())
|
||||
app.use(this.authMiddleware.bind(this))
|
||||
app.use(this.rateLimitMiddleware.bind(this))
|
||||
|
||||
// REST API Routes
|
||||
this.setupRESTRoutes(app)
|
||||
|
||||
// Start HTTP server
|
||||
this.httpServer = app.listen(config.port || 3000, () => {
|
||||
this.log(`REST API listening on port ${config.port || 3000}`)
|
||||
})
|
||||
|
||||
// WebSocket server for real-time
|
||||
this.wsServer = new WebSocketServer({
|
||||
server: this.httpServer,
|
||||
path: '/ws'
|
||||
})
|
||||
|
||||
this.setupWebSocketServer()
|
||||
|
||||
// MCP over WebSocket
|
||||
this.setupMCPWebSocket()
|
||||
}
|
||||
|
||||
private setupRESTRoutes(app: any) {
|
||||
// Health check
|
||||
app.get('/health', (req: any, res: any) => {
|
||||
res.json({ status: 'healthy', version: '2.0.0' })
|
||||
})
|
||||
|
||||
// Search endpoint
|
||||
app.post('/api/search', async (req: any, res: any) => {
|
||||
try {
|
||||
const { query, limit = 10, options = {} } = req.body
|
||||
const results = await this.context.brain.search(query, limit, options)
|
||||
res.json({ success: true, results })
|
||||
} catch (error) {
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: error.message
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// Add data endpoint
|
||||
app.post('/api/add', async (req: any, res: any) => {
|
||||
try {
|
||||
const { content, metadata } = req.body
|
||||
const id = await this.context.brain.add(content, metadata)
|
||||
res.json({ success: true, id })
|
||||
} catch (error) {
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: error.message
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// Get endpoint
|
||||
app.get('/api/get/:id', async (req: any, res: any) => {
|
||||
try {
|
||||
const data = await this.context.brain.get(req.params.id)
|
||||
res.json({ success: true, data })
|
||||
} catch (error) {
|
||||
res.status(404).json({
|
||||
success: false,
|
||||
error: 'Not found'
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// Delete endpoint
|
||||
app.delete('/api/delete/:id', async (req: any, res: any) => {
|
||||
try {
|
||||
await this.context.brain.delete(req.params.id)
|
||||
res.json({ success: true })
|
||||
} catch (error) {
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: error.message
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// Relate endpoint
|
||||
app.post('/api/relate', async (req: any, res: any) => {
|
||||
try {
|
||||
const { source, target, verb, metadata } = req.body
|
||||
await this.context.brain.relate(source, target, verb, metadata)
|
||||
res.json({ success: true })
|
||||
} catch (error) {
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: error.message
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// Find endpoint (complex queries)
|
||||
app.post('/api/find', async (req: any, res: any) => {
|
||||
try {
|
||||
const results = await this.context.brain.find(req.body)
|
||||
res.json({ success: true, results })
|
||||
} catch (error) {
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: error.message
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// Cluster endpoint
|
||||
app.post('/api/cluster', async (req: any, res: any) => {
|
||||
try {
|
||||
const { algorithm = 'kmeans', options = {} } = req.body
|
||||
const clusters = await this.context.brain.cluster(algorithm, options)
|
||||
res.json({ success: true, clusters })
|
||||
} catch (error) {
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: error.message
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// MCP endpoint (for non-WebSocket MCP)
|
||||
app.post('/api/mcp', async (req: any, res: any) => {
|
||||
try {
|
||||
const response = await this.mcpService.handleRequest(req.body)
|
||||
res.json(response)
|
||||
} catch (error) {
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: error.message
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// GraphQL endpoint (optional)
|
||||
if (this.context.config.apiServer?.enableGraphQL) {
|
||||
this.setupGraphQL(app)
|
||||
}
|
||||
}
|
||||
|
||||
private setupWebSocketServer() {
|
||||
this.wsServer.on('connection', (ws: any) => {
|
||||
this.clients.add(ws)
|
||||
|
||||
ws.on('message', async (message: string) => {
|
||||
try {
|
||||
const msg = JSON.parse(message)
|
||||
await this.handleWebSocketMessage(msg, ws)
|
||||
} catch (error) {
|
||||
ws.send(JSON.stringify({
|
||||
type: 'error',
|
||||
error: error.message
|
||||
}))
|
||||
}
|
||||
})
|
||||
|
||||
ws.on('close', () => {
|
||||
this.clients.delete(ws)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
private async handleWebSocketMessage(msg: any, ws: any) {
|
||||
switch (msg.type) {
|
||||
case 'subscribe':
|
||||
// Subscribe to operations
|
||||
ws.subscriptions = msg.operations || ['all']
|
||||
ws.send(JSON.stringify({
|
||||
type: 'subscribed',
|
||||
operations: ws.subscriptions
|
||||
}))
|
||||
break
|
||||
|
||||
case 'search':
|
||||
const results = await this.context.brain.search(msg.query, msg.limit)
|
||||
ws.send(JSON.stringify({
|
||||
type: 'searchResults',
|
||||
results
|
||||
}))
|
||||
break
|
||||
|
||||
case 'mcp':
|
||||
// Handle MCP over WebSocket
|
||||
const response = await this.mcpService.handleRequest(msg.request)
|
||||
ws.send(JSON.stringify({
|
||||
type: 'mcpResponse',
|
||||
response
|
||||
}))
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
private setupMCPWebSocket() {
|
||||
// Dedicated MCP WebSocket endpoint
|
||||
const { WebSocketServer } = require('ws')
|
||||
const mcpWs = new WebSocketServer({
|
||||
port: (this.context.config.apiServer?.mcpPort || 3001),
|
||||
path: '/mcp'
|
||||
})
|
||||
|
||||
mcpWs.on('connection', (ws: any) => {
|
||||
ws.on('message', async (message: string) => {
|
||||
try {
|
||||
const request = JSON.parse(message)
|
||||
const response = await this.mcpService.handleRequest(request)
|
||||
ws.send(JSON.stringify(response))
|
||||
} catch (error) {
|
||||
ws.send(JSON.stringify({
|
||||
error: error.message,
|
||||
type: 'error'
|
||||
}))
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
this.log(`MCP WebSocket listening on port ${this.context.config.apiServer?.mcpPort || 3001}`)
|
||||
}
|
||||
|
||||
// Broadcast changes to all connected clients
|
||||
async execute<T>(operation: string, params: any, next: () => Promise<T>): Promise<T> {
|
||||
const result = await next()
|
||||
|
||||
// Broadcast to WebSocket clients
|
||||
if (this.clients.size > 0) {
|
||||
const message = JSON.stringify({
|
||||
type: 'operation',
|
||||
operation,
|
||||
params: this.sanitizeParams(params),
|
||||
timestamp: Date.now()
|
||||
})
|
||||
|
||||
for (const client of this.clients) {
|
||||
if (client.subscriptions?.includes('all') ||
|
||||
client.subscriptions?.includes(operation)) {
|
||||
client.send(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
private authMiddleware(req: any, res: any, next: any) {
|
||||
if (!this.context.config.apiServer?.requireAuth) {
|
||||
return next()
|
||||
}
|
||||
|
||||
const apiKey = req.headers['x-api-key']
|
||||
if (!apiKey || !this.apiKeys.has(apiKey)) {
|
||||
return res.status(401).json({ error: 'Unauthorized' })
|
||||
}
|
||||
|
||||
req.user = this.apiKeys.get(apiKey)
|
||||
next()
|
||||
}
|
||||
|
||||
private rateLimitMiddleware(req: any, res: any, next: any) {
|
||||
// Simple rate limiting
|
||||
const ip = req.ip
|
||||
const limit = this.context.config.apiServer?.rateLimit || 100
|
||||
|
||||
// Implementation details...
|
||||
next()
|
||||
}
|
||||
|
||||
private sanitizeParams(params: any) {
|
||||
// Remove sensitive data before broadcasting
|
||||
const safe = { ...params }
|
||||
delete safe.apiKey
|
||||
delete safe.password
|
||||
return safe
|
||||
}
|
||||
|
||||
protected async onShutdown() {
|
||||
// Close all connections
|
||||
for (const client of this.clients) {
|
||||
client.close()
|
||||
}
|
||||
|
||||
// Close servers
|
||||
if (this.httpServer) {
|
||||
await new Promise(resolve => this.httpServer.close(resolve))
|
||||
}
|
||||
if (this.wsServer) {
|
||||
this.wsServer.close()
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🎯 Usage: Deploy Brainy as a Server
|
||||
|
||||
```typescript
|
||||
import { BrainyData } from 'brainy'
|
||||
import { APIServerAugmentation } from 'brainy/augmentations'
|
||||
|
||||
const brain = new BrainyData({
|
||||
apiServer: {
|
||||
enabled: true,
|
||||
port: 3000,
|
||||
mcpPort: 3001,
|
||||
requireAuth: true,
|
||||
rateLimit: 100,
|
||||
cors: { origin: '*' },
|
||||
enableGraphQL: false
|
||||
}
|
||||
})
|
||||
|
||||
// Register the API server augmentation
|
||||
brain.augmentations.register(new APIServerAugmentation())
|
||||
|
||||
await brain.init()
|
||||
|
||||
console.log('Brainy API Server running!')
|
||||
console.log('REST API: http://localhost:3000')
|
||||
console.log('WebSocket: ws://localhost:3000/ws')
|
||||
console.log('MCP: ws://localhost:3001/mcp')
|
||||
```
|
||||
|
||||
## 🔌 Client Usage
|
||||
|
||||
### REST API
|
||||
```javascript
|
||||
// Search
|
||||
const response = await fetch('http://localhost:3000/api/search', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'X-API-Key': 'your-key'
|
||||
},
|
||||
body: JSON.stringify({
|
||||
query: 'find documents about AI',
|
||||
limit: 10
|
||||
})
|
||||
})
|
||||
const { results } = await response.json()
|
||||
```
|
||||
|
||||
### WebSocket (Real-time)
|
||||
```javascript
|
||||
const ws = new WebSocket('ws://localhost:3000/ws')
|
||||
|
||||
ws.onopen = () => {
|
||||
// Subscribe to operations
|
||||
ws.send(JSON.stringify({
|
||||
type: 'subscribe',
|
||||
operations: ['add', 'delete', 'relate']
|
||||
}))
|
||||
}
|
||||
|
||||
ws.onmessage = (event) => {
|
||||
const msg = JSON.parse(event.data)
|
||||
console.log('Operation:', msg.operation, msg.params)
|
||||
}
|
||||
```
|
||||
|
||||
### MCP (AI Agents)
|
||||
```javascript
|
||||
const mcpWs = new WebSocket('ws://localhost:3001/mcp')
|
||||
|
||||
mcpWs.send(JSON.stringify({
|
||||
type: 'data_access',
|
||||
operation: 'search',
|
||||
requestId: '123',
|
||||
parameters: { query: 'test' }
|
||||
}))
|
||||
|
||||
mcpWs.onmessage = (event) => {
|
||||
const response = JSON.parse(event.data)
|
||||
console.log('MCP Response:', response)
|
||||
}
|
||||
```
|
||||
|
||||
## 🏗️ Architecture Benefits
|
||||
|
||||
### Why as an Augmentation?
|
||||
|
||||
1. **Optional** - Not everyone needs a server
|
||||
2. **Configurable** - Easy to enable/disable
|
||||
3. **Extensible** - Add custom endpoints
|
||||
4. **Integrated** - Hooks into all operations
|
||||
5. **Real-time** - Broadcasts changes automatically
|
||||
|
||||
### Security Features
|
||||
|
||||
- **API Key Authentication**
|
||||
- **Rate Limiting**
|
||||
- **CORS Configuration**
|
||||
- **Parameter Sanitization**
|
||||
- **SSL/TLS Support** (with proper certs)
|
||||
|
||||
## 🌍 Deployment Options
|
||||
|
||||
### Local Development
|
||||
```bash
|
||||
npm install brainy
|
||||
node server.js # Your server file with APIServerAugmentation
|
||||
```
|
||||
|
||||
### Docker Container
|
||||
```dockerfile
|
||||
FROM node:18-alpine
|
||||
WORKDIR /app
|
||||
COPY package*.json ./
|
||||
RUN npm install brainy
|
||||
COPY server.js .
|
||||
EXPOSE 3000 3001
|
||||
CMD ["node", "server.js"]
|
||||
```
|
||||
|
||||
### Cloud Deployment
|
||||
Deploy to any Node.js hosting:
|
||||
- Vercel Edge Functions
|
||||
- Cloudflare Workers (with adapter)
|
||||
- AWS Lambda (with adapter)
|
||||
- Google Cloud Run
|
||||
- Traditional VPS
|
||||
|
||||
### Browser Service Worker
|
||||
```javascript
|
||||
// In browser, use Service Worker for local API
|
||||
if ('serviceWorker' in navigator) {
|
||||
// APIServerAugmentation can create a Service Worker
|
||||
// that intercepts fetch() calls and handles them locally
|
||||
}
|
||||
```
|
||||
|
||||
## 🎯 The Complete Picture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Client Applications │
|
||||
├─────────────┬───────────┬──────────────────┤
|
||||
│ REST │ WebSocket │ MCP │
|
||||
└──────┬──────┴─────┬─────┴──────┬───────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ APIServerAugmentation │
|
||||
│ (Unified API exposure as augmentation) │
|
||||
└─────────────────┬───────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ BrainyData Core │
|
||||
│ (with all augmentations in pipeline) │
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Storage Layer │
|
||||
│ (FileSystem, S3, OPFS, Memory) │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 🚀 Summary
|
||||
|
||||
1. **MCP is built-in** - Already in Brainy core
|
||||
2. **API Server should be an augmentation** - Optional, configurable
|
||||
3. **Exposes everything** - REST, WebSocket, MCP, GraphQL
|
||||
4. **Real-time by default** - Broadcasts all operations
|
||||
5. **Secure** - Auth, rate limiting, CORS
|
||||
6. **Deploy anywhere** - Node, Deno, Browser, Cloud
|
||||
|
||||
The beauty is that the API server is just another augmentation - it hooks into the pipeline like everything else and exposes Brainy's capabilities to the world!
|
||||
Loading…
Add table
Add a link
Reference in a new issue