diff --git a/README.md b/README.md index c76bd319..a9fb2573 100644 --- a/README.md +++ b/README.md @@ -559,6 +559,25 @@ Works in all modern browsers: For browsers without OPFS support, falls back to in-memory storage. +## ☁️ Cloud Deployment + +Brainy can be deployed as a standalone web service on various cloud platforms using the included cloud wrapper: + +- **AWS Lambda and API Gateway**: Deploy as a serverless function with API Gateway +- **Google Cloud Run**: Deploy as a containerized service +- **Cloudflare Workers**: Deploy as a serverless function on the edge + +The cloud wrapper provides both RESTful and WebSocket APIs for all Brainy operations, enabling both request-response and real-time communication patterns. It supports multiple storage backends and can be configured via environment variables. + +Key features of the cloud wrapper: +- RESTful API for standard CRUD operations +- WebSocket API for real-time updates and subscriptions +- Support for multiple storage backends (Memory, FileSystem, S3) +- Configurable via environment variables +- Deployment scripts for AWS, Google Cloud, and Cloudflare + +To get started with cloud deployment, see the [Cloud Wrapper README](cloud-wrapper/README.md). + ## 📚 Examples The repository includes several examples: @@ -700,6 +719,67 @@ if (connectionResult[0] && (await connectionResult[0]).success) { - Node.js >= 23.11.0 +## 🤝 Contributing + +### Code Style Guidelines + +Brainy follows a specific code style to maintain consistency throughout the codebase: + +1. **No Semicolons**: All code in the project should avoid using semicolons wherever possible. This applies to: + - Source code files (.ts, .js) + - Generated code + - Code examples in documentation + - Templates and snippets + +2. **Formatting**: The project uses Prettier for code formatting with the following settings: + - No semicolons (`"semi": false`) + - Single quotes for strings (`"singleQuote": true`) + - 2-space indentation (`"tabWidth": 2`) + - Always use parentheses around arrow function parameters (`"arrowParens": "always"`) + - Place the closing bracket of JSX elements on the same line (`"bracketSameLine": true`) + - Add spaces between brackets and object literals (`"bracketSpacing": true`) + - Line width of 80 characters (`"printWidth": 80`) + - No trailing commas (`"trailingComma": "none"`) + - Use spaces instead of tabs (`"useTabs": false`) + +3. **Linting**: ESLint is configured with the following rules: + - No semicolons: + ```json + "semi": ["error", "never"], + "@typescript-eslint/semi": ["error", "never"] + ``` + - Unused variables handling: + ```json + "no-unused-vars": "off", + "@typescript-eslint/no-unused-vars": ["warn", { + "args": "after-used", + "argsIgnorePattern": "^_" + }] + ``` + - Extends recommended ESLint and TypeScript ESLint configurations + +4. **TypeScript Configuration**: + - Strict type checking enabled (`"strict": true`) + - Consistent casing in file names enforced (`"forceConsistentCasingInFileNames": true`) + - ES2020 target with Node.js module system + - Source maps generated for debugging + +5. **Commit Messages**: + - Use the imperative mood ("Add feature" not "Added feature") + - Keep the first line concise (under 50 characters) + - Reference issues and pull requests where appropriate + - Provide detailed explanations in the commit body when necessary + +When contributing to the project, please ensure your code follows these guidelines. The project's CI/CD pipeline will automatically check for style violations. + +### Development Workflow + +1. Fork the repository +2. Create a feature branch +3. Make your changes +4. Run tests with `npm test` +5. Submit a pull request + ## 📄 License [MIT](LICENSE) diff --git a/cloud-wrapper/README.md b/cloud-wrapper/README.md new file mode 100644 index 00000000..46434397 --- /dev/null +++ b/cloud-wrapper/README.md @@ -0,0 +1,292 @@ +# Brainy Cloud Wrapper + +A standalone web service wrapper for the [Brainy](https://github.com/soulcraft/brainy) vector graph database. This wrapper allows you to deploy Brainy as a RESTful API service on various cloud platforms including AWS, Google Cloud, and Cloudflare. + +## Features + +- RESTful API for all Brainy operations +- Support for multiple storage backends (Memory, FileSystem, S3) +- Configurable via environment variables +- Deployment scripts for AWS, Google Cloud, and Cloudflare +- Secure by default with Helmet middleware +- Cross-origin resource sharing (CORS) support + +## Prerequisites + +- Node.js 23.11.0 or higher +- npm or yarn +- For cloud deployments: + - AWS: AWS CLI installed and configured + - Google Cloud: Google Cloud SDK installed and configured + - Cloudflare: Wrangler CLI installed and configured + +## Installation + +1. Clone the repository: + ```bash + git clone https://github.com/soulcraft/brainy.git + cd brainy/cloud-wrapper + ``` + +2. Install dependencies: + ```bash + npm install --legacy-peer-deps + ``` + +3. Create a `.env` file based on the example: + ```bash + cp .env.example .env + ``` + +4. Edit the `.env` file to configure your environment. + +## Configuration + +The cloud wrapper can be configured using environment variables. See the `.env.example` file for available options. + +### Storage Options + +- `STORAGE_TYPE`: The type of storage to use. Options: + - `memory`: In-memory storage (default for Cloudflare) + - `filesystem`: File system storage (default for local and AWS/GCP) + - `s3`: S3-compatible storage (AWS S3, MinIO, etc.) + - `r2`: Cloudflare R2 storage (Cloudflare only) + +### S3 Storage Configuration + +When using `STORAGE_TYPE=s3`, the following environment variables are required: + +- `S3_BUCKET_NAME`: The name of the S3 bucket +- `S3_ACCESS_KEY_ID`: Your S3 access key ID +- `S3_SECRET_ACCESS_KEY`: Your S3 secret access key +- `S3_REGION`: The S3 region (default: `us-east-1`) +- `S3_ENDPOINT` (optional): Custom endpoint for S3-compatible services + +## Local Development + +1. Build the project: + ```bash + npm run build + ``` + +2. Start the development server: + ```bash + npm run dev + ``` + +3. The API will be available at `http://localhost:3000`. + +## API Endpoints + +Brainy Cloud Wrapper provides both REST API and WebSocket API for interacting with the database. + +### REST API + +#### Status + +- `GET /api/status`: Get database status + +#### Nouns (Entities) + +- `POST /api/nouns`: Add a new noun + - Body: `{ "text": "Your text", "metadata": { ... } }` +- `GET /api/nouns/:id`: Get a noun by ID +- `PUT /api/nouns/:id`: Update noun metadata + - Body: `{ "metadata": { ... } }` +- `DELETE /api/nouns/:id`: Delete a noun + +#### Search + +- `POST /api/search`: Search for similar nouns + - Body: `{ "query": "Your search query", "limit": 10 }` + +#### Verbs (Relationships) + +- `POST /api/verbs`: Add a relationship between nouns + - Body: `{ "sourceId": "...", "targetId": "...", "metadata": { ... } }` +- `GET /api/verbs`: Get all relationships +- `GET /api/verbs/source/:id`: Get relationships by source +- `GET /api/verbs/target/:id`: Get relationships by target +- `DELETE /api/verbs/:id`: Delete a relationship + +#### Database Management + +- `DELETE /api/clear`: Clear all data + +### WebSocket API + +The WebSocket API provides real-time communication with the Brainy database. Connect to the WebSocket server at `ws://your-server:port`. + +#### Message Format + +All WebSocket messages follow this format: + +```json +{ + "type": "messageType", + "id": "unique-message-id", + "payload": { + // Message-specific data + } +} +``` + +#### Available Message Types + +##### Status +- Request: `{ "type": "status" }` +- Response: `{ "type": "status", "id": "...", "payload": { /* database status */ } }` + +##### Nouns (Entities) +- Add a noun: + - Request: `{ "type": "addNoun", "payload": { "text": "Your text", "metadata": { ... } } }` + - Response: `{ "type": "addNoun", "id": "...", "payload": { "id": "new-noun-id" } }` + +- Get a noun: + - Request: `{ "type": "getNoun", "payload": { "id": "noun-id" } }` + - Response: `{ "type": "getNoun", "id": "...", "payload": { /* noun data */ } }` + +- Update a noun: + - Request: `{ "type": "updateNoun", "payload": { "id": "noun-id", "metadata": { ... } } }` + - Response: `{ "type": "updateNoun", "id": "...", "payload": { "success": true } }` + +- Delete a noun: + - Request: `{ "type": "deleteNoun", "payload": { "id": "noun-id" } }` + - Response: `{ "type": "deleteNoun", "id": "...", "payload": { "success": true } }` + +##### Search +- Search for similar nouns: + - Request: `{ "type": "search", "payload": { "query": "Your search query", "limit": 10 } }` + - Response: `{ "type": "search", "id": "...", "payload": [ /* search results */ ] }` + +##### Verbs (Relationships) +- Add a verb: + - Request: `{ "type": "addVerb", "payload": { "sourceId": "...", "targetId": "...", "metadata": { ... } } }` + - Response: `{ "type": "addVerb", "id": "...", "payload": { "success": true } }` + +- Get all verbs: + - Request: `{ "type": "getVerbs" }` + - Response: `{ "type": "getVerbs", "id": "...", "payload": [ /* all verbs */ ] }` + +- Get verbs by source: + - Request: `{ "type": "getVerbsBySource", "payload": { "id": "source-id" } }` + - Response: `{ "type": "getVerbsBySource", "id": "...", "payload": [ /* verbs */ ] }` + +- Get verbs by target: + - Request: `{ "type": "getVerbsByTarget", "payload": { "id": "target-id" } }` + - Response: `{ "type": "getVerbsByTarget", "id": "...", "payload": [ /* verbs */ ] }` + +- Delete a verb: + - Request: `{ "type": "deleteVerb", "payload": { "id": "verb-id" } }` + - Response: `{ "type": "deleteVerb", "id": "...", "payload": { "success": true } }` + +##### Database Management +- Clear all data: + - Request: `{ "type": "clear" }` + - Response: `{ "type": "clear", "id": "...", "payload": { "success": true } }` + +#### Real-time Subscriptions + +The WebSocket API supports subscribing to real-time updates: + +- Subscribe to updates: + - Request: `{ "type": "subscribe", "payload": { "type": "nouns" } }` + - Response: `{ "type": "subscribe", "id": "...", "payload": { "success": true, "type": "nouns" } }` + +- Unsubscribe from updates: + - Request: `{ "type": "unsubscribe", "payload": { "type": "nouns" } }` + - Response: `{ "type": "unsubscribe", "id": "...", "payload": { "success": true, "type": "nouns" } }` + +Available subscription types: +- `nouns`: Updates about nouns (added, updated, deleted) +- `verbs`: Updates about verbs (added, deleted) +- `searchResults`: Updates about search results + +When subscribed, you'll receive messages when relevant events occur: + +```json +{ + "type": "subscribe", + "payload": { + "type": "nouns", + "data": { + "type": "added", + "id": "noun-id", + "data": { /* noun data */ } + } + } +} +``` + +## Cloud Deployment + +### AWS Lambda and API Gateway + +1. Configure AWS-specific environment variables: + ``` + AWS_REGION=us-east-1 + AWS_FUNCTION_NAME=brainy-cloud-service + AWS_API_GATEWAY_NAME=brainy-api + AWS_STAGE_NAME=prod + AWS_ACCOUNT_ID=your-account-id + ``` + +2. Deploy to AWS: + ```bash + npm run deploy:aws + ``` + +### Google Cloud Run + +1. Configure GCP-specific environment variables: + ``` + GCP_PROJECT_ID=your-project-id + GCP_REGION=us-central1 + GCP_SERVICE_NAME=brainy-cloud-service + GCP_IMAGE_NAME=brainy-cloud-service + GCP_MEMORY=512Mi + GCP_CPU=1 + GCP_MAX_INSTANCES=10 + GCP_MIN_INSTANCES=0 + ``` + +2. Deploy to Google Cloud: + ```bash + npm run deploy:gcp + ``` + +### Cloudflare Workers + +1. Configure Cloudflare-specific environment variables: + ``` + CF_ACCOUNT_ID=your-account-id + CF_WORKER_NAME=brainy-cloud-service + CF_KV_NAMESPACE=BRAINY_STORAGE + CF_R2_BUCKET=brainy-storage + ``` + +2. Deploy to Cloudflare: + ```bash + npm run deploy:cloudflare + ``` + +## Storage Considerations + +### AWS Lambda + +When deploying to AWS Lambda, it's recommended to use S3 storage for persistence. The filesystem storage option will work but data will be lost when the Lambda function is recycled. + +### Google Cloud Run + +For Google Cloud Run, you can use either filesystem storage (for ephemeral storage) or S3-compatible storage (like Google Cloud Storage with an S3 compatibility layer). + +### Cloudflare Workers + +Cloudflare Workers have limited storage options. The recommended approach is to use: +- Cloudflare KV for small datasets +- Cloudflare R2 for larger datasets +- Memory storage for temporary data + +## License + +MIT diff --git a/cloud-wrapper/package.json b/cloud-wrapper/package.json new file mode 100644 index 00000000..16dc6475 --- /dev/null +++ b/cloud-wrapper/package.json @@ -0,0 +1,85 @@ +{ + "name": "@soulcraft/brainy-cloud", + "version": "0.1.0", + "description": "Cloud deployment wrapper for Brainy vector graph database", + "main": "dist/index.js", + "type": "module", + "scripts": { + "build": "tsc", + "start": "node dist/index.js", + "dev": "nodemon --exec node --experimental-specifier-resolution=node --loader ts-node/esm src/index.ts", + "deploy:aws": "node scripts/deploy-aws.js", + "deploy:gcp": "node scripts/deploy-gcp.js", + "deploy:cloudflare": "node scripts/deploy-cloudflare.js" + }, + "keywords": [ + "brainy", + "vector-database", + "cloud", + "aws", + "google-cloud", + "cloudflare" + ], + "author": "David Snelling (david@soulcraft.com)", + "license": "MIT", + "dependencies": { + "@soulcraft/brainy": "^0.7.5", + "cors": "^2.8.5", + "dotenv": "^16.3.1", + "express": "^4.18.2", + "helmet": "^7.1.0", + "morgan": "^1.10.0", + "uuid": "^9.0.1", + "ws": "^8.16.0" + }, + "devDependencies": { + "@types/cors": "^2.8.17", + "@types/express": "^4.17.21", + "@types/morgan": "^1.9.9", + "@types/node": "^20.10.5", + "@types/uuid": "^9.0.7", + "@types/ws": "^8.5.10", + "nodemon": "^3.0.2", + "ts-node": "^10.9.2", + "typescript": "^5.3.3" + }, + "engines": { + "node": ">=23.11.0" + }, + "prettier": { + "arrowParens": "always", + "bracketSameLine": true, + "bracketSpacing": true, + "htmlWhitespaceSensitivity": "css", + "printWidth": 80, + "proseWrap": "preserve", + "semi": false, + "singleQuote": true, + "tabWidth": 2, + "trailingComma": "none", + "useTabs": false + }, + "eslintConfig": { + "root": true, + "extends": [ + "eslint:recommended", + "plugin:@typescript-eslint/recommended" + ], + "parser": "@typescript-eslint/parser", + "plugins": [ + "@typescript-eslint" + ], + "rules": { + "no-unused-vars": "off", + "@typescript-eslint/no-unused-vars": [ + "warn", + { + "args": "after-used", + "argsIgnorePattern": "^_" + } + ], + "semi": ["error", "never"], + "@typescript-eslint/semi": ["error", "never"] + } + } +} diff --git a/cloud-wrapper/tsconfig.json b/cloud-wrapper/tsconfig.json new file mode 100644 index 00000000..371697c0 --- /dev/null +++ b/cloud-wrapper/tsconfig.json @@ -0,0 +1,17 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "esModuleInterop": true, + "outDir": "./dist", + "rootDir": "./src", + "strict": true, + "declaration": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true, + "resolveJsonModule": true + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist"] +} diff --git a/package.json b/package.json index 3f90e1ef..260f9827 100644 --- a/package.json +++ b/package.json @@ -110,7 +110,9 @@ "args": "after-used", "argsIgnorePattern": "^_" } - ] + ], + "semi": ["error", "never"], + "@typescript-eslint/semi": ["error", "never"] } } }