From fe8eb38e1f40cb199f05d8961ab35e25392c8c6e Mon Sep 17 00:00:00 2001 From: David Snelling Date: Fri, 20 Jun 2025 10:56:59 -0700 Subject: [PATCH] **feat: enhance environment detection, adaptive storage, and introduce MCP service details** ### Changes: - **README Enhancements**: - Added detection capabilities for browser, Node.js, and serverless environments with optimized configuration. - Expanded details on adaptive storage strategies based on the detected environment. - Included new information on the Model Control Protocol (MCP), including components like `BrainyMCPAdapter` and `MCPAugmentationToolset`. - **Terminology Updates**: - Replaced "restore sparse data" terminology with "import sparse data" in code samples and CLI commands for clarity (e.g., `importSparseData` method and `brainy import-sparse` command). - Updated related examples for consistency with new term usage. - **Cross-Environment Compatibility**: - Highlighted dynamic imports and storage migration for seamless operation in different environments. - Described augmented browser support and streamlined development practices. - **MCP Documentation**: - Detailed REST and WebSocket APIs for interacting with Brainy through MCP. - Outlined configuration options like rate limiting and authentication for MCP services. - **cloud-wrapper/README.md**: - Added configuration details for MCP services, including specific usage of ports, API keys, and CORS settings. - Expanded API scopes by introducing MCP-related endpoints for real-time and batch interactions. ### Purpose: Enhanced documentation to reflect recent updates to environment detection, storage optimization, and MCP services. The changes improve clarity, usability, and developer understanding across environments and interaction protocols. --- README.md | 62 +++++++++++++++++++++++++++++++++++++---- cloud-wrapper/README.md | 47 ++++++++++++++++++++++++++++++- 2 files changed, 102 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 9c579a5f..23689b7e 100644 --- a/README.md +++ b/README.md @@ -151,6 +151,7 @@ Brainy combines four key technologies to create its adaptive intelligence: 2. **HNSW Algorithm** - Enables fast similarity search through a hierarchical graph structure (like a super-efficient treasure map for your data) 3. **Adaptive Environment Detection** - Automatically senses your platform and optimizes accordingly: + - Detects browser, Node.js, and serverless environments automatically - Adjusts performance parameters based on available resources - Learns from query patterns to optimize future searches - Tunes itself for your specific use cases the more you use it @@ -159,6 +160,7 @@ Brainy combines four key technologies to create its adaptive intelligence: - Browser: Origin Private File System (OPFS) - Node.js: File system - Server: S3-compatible storage (optional) + - Serverless: In-memory storage with optional cloud persistence - Fallback: In-memory storage - Automatically migrates between storage types as needed! @@ -731,13 +733,13 @@ const restoreResult = await db.restore(backupData, { clearExisting: true // Whether to clear existing data before restore }) -// Restore sparse data (without vectors) +// Import sparse data (without vectors) // Vectors will be automatically created using the embedding function const sparseData = { nouns: [ { id: '123', - // No vector field - will be created during restore + // No vector field - will be created during import metadata: { noun: 'Thing', text: 'This text will be used to generate a vector' @@ -748,7 +750,7 @@ const sparseData = { version: '1.0.0' } -const sparseRestoreResult = await db.restore(sparseData) +const sparseImportResult = await db.importSparseData(sparseData) ``` ### CLI Backup/Restore @@ -760,8 +762,8 @@ brainy backup --output brainy-backup.json # Restore data from a file brainy restore --input brainy-backup.json --clear-existing -# Restore sparse data (without vectors) -brainy restore --input sparse-data.json +# Import sparse data (without vectors) +brainy import-sparse --input sparse-data.json ``` ## 🔋 Embedding @@ -782,7 +784,54 @@ Brainy includes an augmentation system for extending functionality: - **Perception Augmentations**: Data interpretation and visualization - **Activation Augmentations**: Trigger actions -## 🌐 Browser Compatibility +### Model Control Protocol (MCP) + +Brainy includes a Model Control Protocol (MCP) implementation that allows external models to access Brainy data and use the augmentation pipeline as tools: + +- **BrainyMCPAdapter**: Provides access to Brainy data through MCP +- **MCPAugmentationToolset**: Exposes the augmentation pipeline as tools +- **BrainyMCPService**: Integrates the adapter and toolset, providing WebSocket and REST server implementations + +The MCP components have different environment compatibility: + +- **BrainyMCPAdapter** and **MCPAugmentationToolset** can run in any environment (browser, Node.js, server) +- **BrainyMCPService** has core functionality that works in any environment, but its server functionality (WebSocket/REST) has been moved to the cloud-wrapper project to avoid including Node.js-specific dependencies in the browser bundle + +For detailed documentation and usage examples, see the [MCP documentation](src/mcp/README.md). + +## 🌐 Cross-Environment Compatibility + +Brainy is designed to run seamlessly in any environment, from browsers to Node.js to serverless functions and containers. All Brainy data, functions, and augmentations are environment-agnostic, allowing you to use the same code everywhere. + +### Environment Detection + +Brainy automatically detects the environment it's running in: + +```typescript +import { environment } from '@soulcraft/brainy' + +// Check which environment we're running in +console.log(`Running in ${ + environment.isBrowser ? 'browser' : + environment.isNode ? 'Node.js' : + 'serverless/unknown' +} environment`) +``` + +### Adaptive Storage + +Storage adapters are automatically selected based on the environment: + +- **Browser**: Uses Origin Private File System (OPFS) when available, falls back to in-memory storage +- **Node.js**: Uses file system storage by default, with options for S3-compatible cloud storage +- **Serverless**: Uses in-memory storage with options for cloud persistence +- **Container**: Automatically detects and uses the appropriate storage based on available capabilities + +### Dynamic Imports + +Brainy uses dynamic imports to load environment-specific dependencies only when needed, keeping the bundle size small and ensuring compatibility across environments. + +### Browser Support Works in all modern browsers: @@ -808,6 +857,7 @@ Key features of the cloud wrapper: - RESTful API for standard CRUD operations - WebSocket API for real-time updates and subscriptions +- Model Control Protocol (MCP) service for external model access - Support for multiple storage backends (Memory, FileSystem, S3) - Configurable via environment variables - Deployment scripts for AWS, Google Cloud, and Cloudflare diff --git a/cloud-wrapper/README.md b/cloud-wrapper/README.md index 7cd72f6a..436691cb 100644 --- a/cloud-wrapper/README.md +++ b/cloud-wrapper/README.md @@ -9,6 +9,8 @@ A standalone web service wrapper for the [Brainy](https://github.com/soulcraft/b ## Features - RESTful API for all Brainy operations +- WebSocket API for real-time updates and subscriptions +- Model Control Protocol (MCP) service for external model access - Support for multiple storage backends (Memory, FileSystem, S3) - Configurable via environment variables - Deployment scripts for AWS, Google Cloud, and Cloudflare @@ -66,6 +68,18 @@ When using `STORAGE_TYPE=s3`, the following environment variables are required: - `S3_REGION`: The S3 region (default: `us-east-1`) - `S3_ENDPOINT` (optional): Custom endpoint for S3-compatible services +### MCP Service Configuration + +The Model Control Protocol (MCP) service can be configured using the following environment variables: + +- `MCP_WS_PORT`: Port for the MCP WebSocket server (if not set, WebSocket server is disabled) +- `MCP_REST_PORT`: Port for the MCP REST server (if not set, REST server is disabled) +- `MCP_ENABLE_AUTH`: Enable authentication for MCP requests (`true` or `false`) +- `MCP_API_KEYS`: Comma-separated list of API keys for authentication +- `MCP_RATE_LIMIT_REQUESTS`: Maximum number of requests per time window +- `MCP_RATE_LIMIT_WINDOW_MS`: Time window for rate limiting in milliseconds (default: `60000`) +- `MCP_ENABLE_CORS`: Enable CORS for MCP REST server (`true` or `false`) + ## Local Development 1. Build the project: @@ -82,7 +96,7 @@ When using `STORAGE_TYPE=s3`, the following environment variables are required: ## API Endpoints -Brainy Cloud Wrapper provides both REST API and WebSocket API for interacting with the database. +Brainy Cloud Wrapper provides REST API, WebSocket API, and Model Control Protocol (MCP) API for interacting with the database. ### REST API @@ -222,6 +236,37 @@ When subscribed, you'll receive messages when relevant events occur: } ``` +### MCP API + +The Model Control Protocol (MCP) API provides a standardized interface for external models to access Brainy data and use the augmentation pipeline as tools. The MCP API is available through both WebSocket and REST endpoints. + +#### MCP REST API Endpoints + +- `POST /mcp/data`: Access Brainy data (search, get, add, etc.) +- `POST /mcp/tools`: Execute augmentation pipeline tools +- `POST /mcp/system`: Get system information +- `POST /mcp/auth`: Authenticate with the MCP service +- `GET /mcp/tools`: Get available tools + +#### MCP WebSocket + +Connect to the MCP WebSocket server at `ws://your-server:MCP_WS_PORT` and send JSON messages in the MCP format: + +```json +{ + "type": "DATA_ACCESS", + "requestId": "unique-request-id", + "version": "1.0", + "operation": "search", + "parameters": { + "query": "Your search query", + "k": 5 + } +} +``` + +For detailed documentation on the MCP API, see the [MCP documentation](../src/mcp/README.md). + ## Cloud Deployment ### AWS Lambda and API Gateway