soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

feat: add cloud deployment support and contribution guidelines

Browse source 
Introduced a cloud deployment wrapper for Brainy supporting AWS Lambda, Google Cloud Run, and Cloudflare Workers. Added configuration options, deployment scripts, and API reference. Updated `README.md` with contribution guidelines and cloud deployment documentation.
This commit is contained in:
David Snelling 2025-06-06 09:08:33 -07:00
parent 1635cc229c
commit 6feab06f34
5 changed files with 477 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

80
README.md
View file

@ -559,6 +559,25 @@ Works in all modern browsers:
For browsers without OPFS support, falls back to in-memory storage. For browsers without OPFS support, falls back to in-memory storage.
## ☁️ Cloud Deployment
Brainy can be deployed as a standalone web service on various cloud platforms using the included cloud wrapper:
- **AWS Lambda and API Gateway**: Deploy as a serverless function with API Gateway
- **Google Cloud Run**: Deploy as a containerized service
- **Cloudflare Workers**: Deploy as a serverless function on the edge
The cloud wrapper provides both RESTful and WebSocket APIs for all Brainy operations, enabling both request-response and real-time communication patterns. It supports multiple storage backends and can be configured via environment variables.
Key features of the cloud wrapper:
- RESTful API for standard CRUD operations
- WebSocket API for real-time updates and subscriptions
- Support for multiple storage backends (Memory, FileSystem, S3)
- Configurable via environment variables
- Deployment scripts for AWS, Google Cloud, and Cloudflare
To get started with cloud deployment, see the [Cloud Wrapper README](cloud-wrapper/README.md).
## 📚 Examples ## 📚 Examples
The repository includes several examples: The repository includes several examples:
@ -700,6 +719,67 @@ if (connectionResult[0] && (await connectionResult[0]).success) {
- Node.js >= 23.11.0 - Node.js >= 23.11.0
## 🤝 Contributing
### Code Style Guidelines
Brainy follows a specific code style to maintain consistency throughout the codebase:
1. **No Semicolons**: All code in the project should avoid using semicolons wherever possible. This applies to:
- Source code files (.ts, .js)
- Generated code
- Code examples in documentation
- Templates and snippets
2. **Formatting**: The project uses Prettier for code formatting with the following settings:
- No semicolons (`"semi": false`)
- Single quotes for strings (`"singleQuote": true`)
- 2-space indentation (`"tabWidth": 2`)
- Always use parentheses around arrow function parameters (`"arrowParens": "always"`)
- Place the closing bracket of JSX elements on the same line (`"bracketSameLine": true`)
- Add spaces between brackets and object literals (`"bracketSpacing": true`)
- Line width of 80 characters (`"printWidth": 80`)
- No trailing commas (`"trailingComma": "none"`)
- Use spaces instead of tabs (`"useTabs": false`)
3. **Linting**: ESLint is configured with the following rules:
- No semicolons:
```json
"semi": ["error", "never"],
"@typescript-eslint/semi": ["error", "never"]
```
- Unused variables handling:
```json
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": ["warn", {
"args": "after-used",
"argsIgnorePattern": "^_"
}]
```
- Extends recommended ESLint and TypeScript ESLint configurations
4. **TypeScript Configuration**:
- Strict type checking enabled (`"strict": true`)
- Consistent casing in file names enforced (`"forceConsistentCasingInFileNames": true`)
- ES2020 target with Node.js module system
- Source maps generated for debugging
5. **Commit Messages**:
- Use the imperative mood ("Add feature" not "Added feature")
- Keep the first line concise (under 50 characters)
- Reference issues and pull requests where appropriate
- Provide detailed explanations in the commit body when necessary
When contributing to the project, please ensure your code follows these guidelines. The project's CI/CD pipeline will automatically check for style violations.
### Development Workflow
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests with `npm test`
5. Submit a pull request
## 📄 License ## 📄 License
[MIT](LICENSE) [MIT](LICENSE)

292
cloud-wrapper/README.md Normal file
View file

@ -0,0 +1,292 @@
# Brainy Cloud Wrapper
A standalone web service wrapper for the [Brainy](https://github.com/soulcraft/brainy) vector graph database. This wrapper allows you to deploy Brainy as a RESTful API service on various cloud platforms including AWS, Google Cloud, and Cloudflare.
## Features
- RESTful API for all Brainy operations
- Support for multiple storage backends (Memory, FileSystem, S3)
- Configurable via environment variables
- Deployment scripts for AWS, Google Cloud, and Cloudflare
- Secure by default with Helmet middleware
- Cross-origin resource sharing (CORS) support
## Prerequisites
- Node.js 23.11.0 or higher
- npm or yarn
- For cloud deployments:
- AWS: AWS CLI installed and configured
- Google Cloud: Google Cloud SDK installed and configured
- Cloudflare: Wrangler CLI installed and configured
## Installation
1. Clone the repository:
```bash
git clone https://github.com/soulcraft/brainy.git
cd brainy/cloud-wrapper
```
2. Install dependencies:
```bash
npm install --legacy-peer-deps
```
3. Create a `.env` file based on the example:
```bash
cp .env.example .env
```
4. Edit the `.env` file to configure your environment.
## Configuration
The cloud wrapper can be configured using environment variables. See the `.env.example` file for available options.
### Storage Options
- `STORAGE_TYPE`: The type of storage to use. Options:
- `memory`: In-memory storage (default for Cloudflare)
- `filesystem`: File system storage (default for local and AWS/GCP)
- `s3`: S3-compatible storage (AWS S3, MinIO, etc.)
- `r2`: Cloudflare R2 storage (Cloudflare only)
### S3 Storage Configuration
When using `STORAGE_TYPE=s3`, the following environment variables are required:
- `S3_BUCKET_NAME`: The name of the S3 bucket
- `S3_ACCESS_KEY_ID`: Your S3 access key ID
- `S3_SECRET_ACCESS_KEY`: Your S3 secret access key
- `S3_REGION`: The S3 region (default: `us-east-1`)
- `S3_ENDPOINT` (optional): Custom endpoint for S3-compatible services
## Local Development
1. Build the project:
```bash
npm run build
```
2. Start the development server:
```bash
npm run dev
```
3. The API will be available at `http://localhost:3000`.
## API Endpoints
Brainy Cloud Wrapper provides both REST API and WebSocket API for interacting with the database.
### REST API
#### Status
- `GET /api/status`: Get database status
#### Nouns (Entities)
- `POST /api/nouns`: Add a new noun
- Body: `{ "text": "Your text", "metadata": { ... } }`
- `GET /api/nouns/:id`: Get a noun by ID
- `PUT /api/nouns/:id`: Update noun metadata
- Body: `{ "metadata": { ... } }`
- `DELETE /api/nouns/:id`: Delete a noun
#### Search
- `POST /api/search`: Search for similar nouns
- Body: `{ "query": "Your search query", "limit": 10 }`
#### Verbs (Relationships)
- `POST /api/verbs`: Add a relationship between nouns
- Body: `{ "sourceId": "...", "targetId": "...", "metadata": { ... } }`
- `GET /api/verbs`: Get all relationships
- `GET /api/verbs/source/:id`: Get relationships by source
- `GET /api/verbs/target/:id`: Get relationships by target
- `DELETE /api/verbs/:id`: Delete a relationship
#### Database Management
- `DELETE /api/clear`: Clear all data
### WebSocket API
The WebSocket API provides real-time communication with the Brainy database. Connect to the WebSocket server at `ws://your-server:port`.
#### Message Format
All WebSocket messages follow this format:
```json
{
"type": "messageType",
"id": "unique-message-id",
"payload": {
// Message-specific data
}
}
```
#### Available Message Types
##### Status
- Request: `{ "type": "status" }`
- Response: `{ "type": "status", "id": "...", "payload": { /* database status */ } }`
##### Nouns (Entities)
- Add a noun:
- Request: `{ "type": "addNoun", "payload": { "text": "Your text", "metadata": { ... } } }`
- Response: `{ "type": "addNoun", "id": "...", "payload": { "id": "new-noun-id" } }`
- Get a noun:
- Request: `{ "type": "getNoun", "payload": { "id": "noun-id" } }`
- Response: `{ "type": "getNoun", "id": "...", "payload": { /* noun data */ } }`
- Update a noun:
- Request: `{ "type": "updateNoun", "payload": { "id": "noun-id", "metadata": { ... } } }`
- Response: `{ "type": "updateNoun", "id": "...", "payload": { "success": true } }`
- Delete a noun:
- Request: `{ "type": "deleteNoun", "payload": { "id": "noun-id" } }`
- Response: `{ "type": "deleteNoun", "id": "...", "payload": { "success": true } }`
##### Search
- Search for similar nouns:
- Request: `{ "type": "search", "payload": { "query": "Your search query", "limit": 10 } }`
- Response: `{ "type": "search", "id": "...", "payload": [ /* search results */ ] }`
##### Verbs (Relationships)
- Add a verb:
- Request: `{ "type": "addVerb", "payload": { "sourceId": "...", "targetId": "...", "metadata": { ... } } }`
- Response: `{ "type": "addVerb", "id": "...", "payload": { "success": true } }`
- Get all verbs:
- Request: `{ "type": "getVerbs" }`
- Response: `{ "type": "getVerbs", "id": "...", "payload": [ /* all verbs */ ] }`
- Get verbs by source:
- Request: `{ "type": "getVerbsBySource", "payload": { "id": "source-id" } }`
- Response: `{ "type": "getVerbsBySource", "id": "...", "payload": [ /* verbs */ ] }`
- Get verbs by target:
- Request: `{ "type": "getVerbsByTarget", "payload": { "id": "target-id" } }`
- Response: `{ "type": "getVerbsByTarget", "id": "...", "payload": [ /* verbs */ ] }`
- Delete a verb:
- Request: `{ "type": "deleteVerb", "payload": { "id": "verb-id" } }`
- Response: `{ "type": "deleteVerb", "id": "...", "payload": { "success": true } }`
##### Database Management
- Clear all data:
- Request: `{ "type": "clear" }`
- Response: `{ "type": "clear", "id": "...", "payload": { "success": true } }`
#### Real-time Subscriptions
The WebSocket API supports subscribing to real-time updates:
- Subscribe to updates:
- Request: `{ "type": "subscribe", "payload": { "type": "nouns" } }`
- Response: `{ "type": "subscribe", "id": "...", "payload": { "success": true, "type": "nouns" } }`
- Unsubscribe from updates:
- Request: `{ "type": "unsubscribe", "payload": { "type": "nouns" } }`
- Response: `{ "type": "unsubscribe", "id": "...", "payload": { "success": true, "type": "nouns" } }`
Available subscription types:
- `nouns`: Updates about nouns (added, updated, deleted)
- `verbs`: Updates about verbs (added, deleted)
- `searchResults`: Updates about search results
When subscribed, you'll receive messages when relevant events occur:
```json
{
"type": "subscribe",
"payload": {
"type": "nouns",
"data": {
"type": "added",
"id": "noun-id",
"data": { /* noun data */ }
}
}
}
```
## Cloud Deployment
### AWS Lambda and API Gateway
1. Configure AWS-specific environment variables:
```
AWS_REGION=us-east-1
AWS_FUNCTION_NAME=brainy-cloud-service
AWS_API_GATEWAY_NAME=brainy-api
AWS_STAGE_NAME=prod
AWS_ACCOUNT_ID=your-account-id
```
2. Deploy to AWS:
```bash
npm run deploy:aws
```
### Google Cloud Run
1. Configure GCP-specific environment variables:
```
GCP_PROJECT_ID=your-project-id
GCP_REGION=us-central1
GCP_SERVICE_NAME=brainy-cloud-service
GCP_IMAGE_NAME=brainy-cloud-service
GCP_MEMORY=512Mi
GCP_CPU=1
GCP_MAX_INSTANCES=10
GCP_MIN_INSTANCES=0
```
2. Deploy to Google Cloud:
```bash
npm run deploy:gcp
```
### Cloudflare Workers
1. Configure Cloudflare-specific environment variables:
```
CF_ACCOUNT_ID=your-account-id
CF_WORKER_NAME=brainy-cloud-service
CF_KV_NAMESPACE=BRAINY_STORAGE
CF_R2_BUCKET=brainy-storage
```
2. Deploy to Cloudflare:
```bash
npm run deploy:cloudflare
```
## Storage Considerations
### AWS Lambda
When deploying to AWS Lambda, it's recommended to use S3 storage for persistence. The filesystem storage option will work but data will be lost when the Lambda function is recycled.
### Google Cloud Run
For Google Cloud Run, you can use either filesystem storage (for ephemeral storage) or S3-compatible storage (like Google Cloud Storage with an S3 compatibility layer).
### Cloudflare Workers
Cloudflare Workers have limited storage options. The recommended approach is to use:
- Cloudflare KV for small datasets
- Cloudflare R2 for larger datasets
- Memory storage for temporary data
## License
MIT

85
cloud-wrapper/package.json Normal file
View file

@ -0,0 +1,85 @@
{
"name": "@soulcraft/brainy-cloud",
"version": "0.1.0",
"description": "Cloud deployment wrapper for Brainy vector graph database",
"main": "dist/index.js",
"type": "module",
"scripts": {
"build": "tsc",
"start": "node dist/index.js",
"dev": "nodemon --exec node --experimental-specifier-resolution=node --loader ts-node/esm src/index.ts",
"deploy:aws": "node scripts/deploy-aws.js",
"deploy:gcp": "node scripts/deploy-gcp.js",
"deploy:cloudflare": "node scripts/deploy-cloudflare.js"
},
"keywords": [
"brainy",
"vector-database",
"cloud",
"aws",
"google-cloud",
"cloudflare"
],
"author": "David Snelling (david@soulcraft.com)",
"license": "MIT",
"dependencies": {
"@soulcraft/brainy": "^0.7.5",
"cors": "^2.8.5",
"dotenv": "^16.3.1",
"express": "^4.18.2",
"helmet": "^7.1.0",
"morgan": "^1.10.0",
"uuid": "^9.0.1",
"ws": "^8.16.0"
},
"devDependencies": {
"@types/cors": "^2.8.17",
"@types/express": "^4.17.21",
"@types/morgan": "^1.9.9",
"@types/node": "^20.10.5",
"@types/uuid": "^9.0.7",
"@types/ws": "^8.5.10",
"nodemon": "^3.0.2",
"ts-node": "^10.9.2",
"typescript": "^5.3.3"
},
"engines": {
"node": ">=23.11.0"
},
"prettier": {
"arrowParens": "always",
"bracketSameLine": true,
"bracketSpacing": true,
"htmlWhitespaceSensitivity": "css",
"printWidth": 80,
"proseWrap": "preserve",
"semi": false,
"singleQuote": true,
"tabWidth": 2,
"trailingComma": "none",
"useTabs": false
},
"eslintConfig": {
"root": true,
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/recommended"
],
"parser": "@typescript-eslint/parser",
"plugins": [
"@typescript-eslint"
],
"rules": {
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": [
"warn",
{
"args": "after-used",
"argsIgnorePattern": "^_"
}
],
"semi": ["error", "never"],
"@typescript-eslint/semi": ["error", "never"]
}
}
}

17
cloud-wrapper/tsconfig.json Normal file
View file

@ -0,0 +1,17 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"esModuleInterop": true,
"outDir": "./dist",
"rootDir": "./src",
"strict": true,
"declaration": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}

4
package.json
View file

@ -110,7 +110,9 @@
"args": "after-used", "args": "after-used",
"argsIgnorePattern": "^_" "argsIgnorePattern": "^_"
} }
] ],
"semi": ["error", "never"],
"@typescript-eslint/semi": ["error", "never"]
} }
} }
} }