brainy/docs/deployment/aws-deployment.md

AWS Deployment Guide for Brainy

Overview

Deploy Brainy on AWS with automatic scaling, high availability, and zero-config dynamic adaptation. Brainy automatically adapts to your AWS environment without manual configuration.

Quick Start (Zero-Config)

Option 1: AWS Lambda (Serverless)

# Install Brainy
npm install @soulcraft/brainy

# Create handler.js
cat > handler.js << 'EOF'
const { Brainy } = require('@soulcraft/brainy')

let brain

exports.handler = async (event) => {
 // Brainy auto-detects Lambda environment and configures accordingly
 if (!brain) {
 brain = new Brainy() // Zero config - auto-adapts to Lambda
 await brain.init()
 }

 const { method, ...params } = JSON.parse(event.body)

 switch(method) {
 case 'add':
 const id = await brain.add(params)
 return { statusCode: 200, body: JSON.stringify({ id }) }
 case 'find':
 const results = await brain.find(params)
 return { statusCode: 200, body: JSON.stringify({ results }) }
 default:
 return { statusCode: 400, body: 'Unknown method' }
 }
}
EOF

# Deploy with AWS SAM
sam init --runtime nodejs20.x --name brainy-app
sam deploy --guided

Option 2: ECS Fargate (Container)

# Build and push Docker image
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin $ECR_URI
docker build -t brainy .
docker tag brainy:latest $ECR_URI/brainy:latest
docker push $ECR_URI/brainy:latest

# Deploy with minimal ECS task definition
cat > task-definition.json << 'EOF'
{
 "family": "brainy",
 "networkMode": "awsvpc",
 "requiresCompatibilities": ["FARGATE"],
 "cpu": "256",
 "memory": "512",
 "containerDefinitions": [{
 "name": "brainy",
 "image": "$ECR_URI/brainy:latest",
 "environment": [
 {"name": "NODE_ENV", "value": "production"}
 ],
 "logConfiguration": {
 "logDriver": "awslogs",
 "options": {
 "awslogs-group": "/ecs/brainy",
 "awslogs-region": "us-east-1",
 "awslogs-stream-prefix": "ecs"
 }
 }
 }]
}
EOF

# Brainy auto-detects ECS environment and uses S3 for storage
aws ecs register-task-definition --cli-input-json file://task-definition.json
aws ecs create-service --cluster default --service-name brainy --task-definition brainy --desired-count 2

Option 3: EC2 Auto-Scaling

# User data script for EC2 instances
#!/bin/bash
curl -fsSL https://rpm.nodesource.com/setup_20.x | sudo bash -
sudo yum install -y nodejs git

# Clone and setup (or use your deployment method)
git clone https://github.com/yourorg/brainy-app.git /app
cd /app
npm install --production

# Create systemd service
cat > /etc/systemd/system/brainy.service << 'EOF'
[Unit]
Description=Brainy
After=network.target

[Service]
Type=simple
User=ec2-user
WorkingDirectory=/app
ExecStart=/usr/bin/node index.js
Restart=on-failure
Environment=NODE_ENV=production

[Install]
WantedBy=multi-user.target
EOF

systemctl start brainy
systemctl enable brainy

Zero-Config Storage (Automatic)

Brainy automatically detects and uses the best available storage:

// No configuration needed - Brainy auto-detects:
const brain = new Brainy()

// Auto-detection priority:
// 1. S3 (if IAM role has permissions)
// 2. EFS (if mounted at /mnt/efs)
// 3. EBS volume (if available)
// 4. Instance storage (fallback)

Manual S3 Configuration (Optional)

const brain = new Brainy({
 storage: {
 type: 's3',
 options: {
 bucket: process.env.S3_BUCKET || 'auto', // 'auto' creates bucket
 region: process.env.AWS_REGION || 'auto', // 'auto' detects region
 // IAM role provides credentials automatically
 }
 }
})

Scaling Strategies

1. Horizontal Scaling (Recommended)

# Auto-scaling policy
Resources:
 AutoScalingTarget:
 Type: AWS::ApplicationAutoScaling::ScalableTarget
 Properties:
 ServiceNamespace: ecs
 ResourceId: service/default/brainy
 ScalableDimension: ecs:service:DesiredCount
 MinCapacity: 2
 MaxCapacity: 100

 AutoScalingPolicy:
 Type: AWS::ApplicationAutoScaling::ScalingPolicy
 Properties:
 PolicyType: TargetTrackingScaling
 TargetTrackingScalingPolicyConfiguration:
 TargetValue: 70.0
 PredefinedMetricSpecification:
 PredefinedMetricType: ECSServiceAverageCPUUtilization

2. Vertical Scaling

Brainy automatically adapts to available memory:

• 256MB: Minimal mode, optimized caching
• 512MB: Standard mode, balanced performance
• 1GB+: Full mode, maximum performance

High Availability Setup

Multi-AZ Deployment

// Brainy automatically handles multi-AZ with S3
const brain = new Brainy({
 distributed: {
 enabled: true, // Auto-enables with S3 storage
 coordinationMethod: 'auto' // Uses S3 for coordination
 }
})

Load Balancing

# Application Load Balancer with health checks
aws elbv2 create-load-balancer \
 --name brainy-alb \
 --subnets subnet-xxx subnet-yyy \
 --security-groups sg-xxx

aws elbv2 create-target-group \
 --name brainy-targets \
 --protocol HTTP \
 --port 3000 \
 --vpc-id vpc-xxx \
 --health-check-path /health \
 --health-check-interval-seconds 30

Monitoring & Observability

CloudWatch Integration

Brainy automatically sends metrics when running on AWS:

// Automatic CloudWatch metrics (no config needed)
// - Request count
// - Response time
// - Error rate
// - Storage usage
// - Memory usage

Custom Metrics

const brain = new Brainy({
 monitoring: {
 enabled: true,
 customMetrics: {
 namespace: 'Brainy/Production',
 dimensions: [
 { Name: 'Environment', Value: 'production' },
 { Name: 'Service', Value: 'api' }
 ]
 }
 }
})

Security Best Practices

1. IAM Role (Recommended)

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "s3:GetObject",
 "s3:PutObject",
 "s3:DeleteObject",
 "s3:ListBucket"
 ],
 "Resource": [
 "arn:aws:s3:::brainy-*/*",
 "arn:aws:s3:::brainy-*"
 ]
 }
 ]
}

2. VPC Configuration

# Private subnets with NAT Gateway
aws ec2 create-vpc --cidr-block 10.0.0.0/16
aws ec2 create-subnet --vpc-id vpc-xxx --cidr-block 10.0.1.0/24 --availability-zone us-east-1a
aws ec2 create-subnet --vpc-id vpc-xxx --cidr-block 10.0.2.0/24 --availability-zone us-east-1b

3. Encryption

// Automatic encryption with S3
const brain = new Brainy({
 storage: {
 type: 's3',
 options: {
 encryption: 'auto' // Uses S3 SSE-S3 by default
 }
 }
})

Cost Optimization

1. Spot Instances (70% savings)

aws ec2 request-spot-fleet --spot-fleet-request-config '{
 "IamFleetRole": "arn:aws:iam::xxx:role/fleet-role",
 "TargetCapacity": 2,
 "SpotPrice": "0.05",
 "LaunchSpecifications": [{
 "ImageId": "ami-xxx",
 "InstanceType": "t3.medium",
 "UserData": "BASE64_ENCODED_STARTUP_SCRIPT"
 }]
}'

2. S3 Intelligent-Tiering

// Brainy automatically uses S3 Intelligent-Tiering
const brain = new Brainy({
 storage: {
 type: 's3',
 options: {
 storageClass: 'INTELLIGENT_TIERING' // Automatic cost optimization
 }
 }
})

3. Lambda Reserved Concurrency

aws lambda put-function-concurrency \
 --function-name brainy-handler \
 --reserved-concurrent-executions 10

Deployment Automation

GitHub Actions CI/CD

name: Deploy to AWS
on:
 push:
 branches: [main]

jobs:
 deploy:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v2

 - name: Configure AWS credentials
 uses: aws-actions/configure-aws-credentials@v1
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
 aws-region: us-east-1

 - name: Deploy to ECS
 run: |
 docker build -t brainy .
 docker tag brainy:latest $ECR_URI/brainy:latest
 docker push $ECR_URI/brainy:latest
 aws ecs update-service --cluster default --service brainy --force-new-deployment

Troubleshooting

Common Issues

1. Storage Auto-Detection Fails

// Explicitly specify storage
const brain = new Brainy({
storage: { type: 's3', options: { bucket: 'my-bucket' } }
})

2. Memory Issues

// Optimize for low memory
const brain = new Brainy({
cache: { maxSize: 100 }, // Reduce cache size
index: { M: 8 } // Reduce HNSW connections
})

3. Cold Starts (Lambda)

Progressive Initialization (Zero-Config)

Brainy automatically detects Lambda environments (AWS_LAMBDA_FUNCTION_NAME) and uses progressive initialization for <200ms cold starts:

// Zero-config - Brainy auto-detects Lambda
const brain = new Brainy({
storage: {
type: 's3',
s3Storage: {
bucketName: 'my-bucket',
region: 'us-east-1',
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
}
}
})
await brain.init() // Returns in <200ms

// First write validates bucket (lazy validation)
await brain.add('noun', { name: 'test' }) // Validates here

Manual Override (if needed)

const brain = new Brainy({
storage: {
type: 's3',
s3Storage: {
bucketName: 'my-bucket',
// Force specific mode
initMode: 'progressive' // 'auto' | 'progressive' | 'strict'
}
}
})

Mode	Cold Start	Best For
auto (default)	<200ms in Lambda	Zero-config, auto-detects
progressive	<200ms always	Force fast init everywhere
strict	100-500ms+	Local dev, tests, debugging

Warm-up (Alternative)

exports.warmup = async () => {
if (!brain) {
brain = new Brainy({ warmup: true })
await brain.init()
}
}

Readiness Detection

Use the brain.ready Promise to ensure Brainy is initialized before handling requests:

let brain

exports.handler = async (event) => {
if (!brain) {
brain = new Brainy({ storage: { type: 's3', ... } })
brain.init() // Fire and forget
}

// Wait for initialization to complete
await brain.ready

// Now safe to use brain methods
const results = await brain.find({ query: event.queryStringParameters.q })
return { statusCode: 200, body: JSON.stringify(results) }
}

For health checks, use isFullyInitialized() to verify all background tasks are complete:

exports.healthCheck = async () => {
try {
await brain.ready
return {
statusCode: 200,
body: JSON.stringify({
status: 'ready',
fullyInitialized: brain.isFullyInitialized()
})
}
} catch (error) {
return {
statusCode: 503,
body: JSON.stringify({ status: 'initializing' })
}
}
}

Production Checklist

• IAM roles configured with minimal permissions
• VPC with private subnets
• Auto-scaling configured
• CloudWatch alarms set up
• Backup strategy (S3 versioning enabled)
• SSL/TLS certificates configured
• Rate limiting enabled
• Health checks configured
• Monitoring dashboard created
• Cost alerts configured

Support

• Documentation: https://brainy.soulcraft.ai/docs
• Issues: https://github.com/soulcraft/brainy/issues
• Community: https://discord.gg/brainy