brainy/docs/SCALING.md

Brainy Scaling Guide

One Line Summary: Single-node by design — Brainy scales by getting the most out of one machine plus operator-layer backup.

Table of Contents

• Quick Start
• How Brainy Scales
• Storage Configurations
• Scaling Patterns
• Real World Examples

Quick Start

In-Memory

import Brainy from '@soulcraft/brainy'
const brain = new Brainy({ storage: { type: 'memory' } })

On-Disk (Default for Node)

const brain = new Brainy({
  storage: { type: 'filesystem', rootDirectory: './brainy-data' }
})

How Brainy Scales

Brainy 8.0 is a single-node library. There is no cluster, no peer discovery, no S3 coordination. Scaling means:

• Up: give the process more RAM, CPU, and IOPS
• Out: stand up multiple independent Brainy instances behind your own service layer
• Cold storage: snapshot the on-disk artifact off-site so you can rehydrate elsewhere

The three knobs that matter most:

1. config.vector.recall — 'fast', 'balanced', or 'accurate' (default 'balanced')
2. config.vector.quantization — { bits: 4 | 8 } for memory savings on the open-core JS vector index
3. config.vector.persistMode — 'immediate' for durability, 'deferred' for throughput

The native vector provider (via the optional @soulcraft/cortex package) extends this with a higher-performing index when installed.

Storage Configurations

Filesystem (Recommended for Production)

const brain = new Brainy({
  storage: {
    type: 'filesystem',
    rootDirectory: '/var/lib/brainy'
  }
})

• Stores everything in a sharded JSON tree under rootDirectory
• Atomic writes via rename
• Survives process restarts
• Snapshot it off-site with gsutil rsync, aws s3 sync, rclone, or tar from your scheduler

Memory

const brain = new Brainy({ storage: { type: 'memory' } })

• Zero I/O, fastest possible
• No persistence — process exit discards everything
• Use for tests and ephemeral caches

Auto

const brain = new Brainy({
  storage: { type: 'auto', rootDirectory: './data' }
})

• Picks filesystem when running on Node with a writable rootDirectory
• Falls back to memory otherwise

Scaling Patterns

Stage 1: Prototype (Memory)

const brain = new Brainy({ storage: { type: 'memory' } })
// Development, tests, <100K items

Stage 2: Production (Filesystem)

const brain = new Brainy({
  storage: { type: 'filesystem', rootDirectory: '/var/lib/brainy' }
})
// Most production workloads up to ~10M entities on a single host

Stage 3: Higher Throughput (Tune the Vector Index)

const brain = new Brainy({
  storage: { type: 'filesystem', rootDirectory: '/var/lib/brainy' },
  vector: {
    recall: 'fast',                // Trade recall for latency
    quantization: { bits: 8 },     // SQ8 quantization
    persistMode: 'deferred'        // Batch persistence
  }
})

Stage 4: Multi-Instance (Operator-Layer)

Run multiple Brainy processes behind your own routing/service layer. Each process owns its own rootDirectory. Sync each artifact off-site independently. Brainy itself does not coordinate between processes.

Real World Examples

Example 1: Single-Node App With Backup

const brain = new Brainy({
  storage: { type: 'filesystem', rootDirectory: '/var/lib/brainy' }
})

Schedule (cron / systemd timer):

*/15 * * * *  rclone sync /var/lib/brainy remote:brainy-backup

Example 2: Tests

const brain = new Brainy({ storage: { type: 'memory' } })
// Fast, no cleanup needed between runs

Example 3: Multi-Tenant Service

Spin up one Brainy instance per tenant, each in its own directory:

function brainForTenant(tenantId: string) {
  return new Brainy({
    storage: {
      type: 'filesystem',
      rootDirectory: `/var/lib/brainy/${tenantId}`
    }
  })
}

Your service layer handles routing and isolation; Brainy stays simple.

Example 4: Higher Recall at Scale

const brain = new Brainy({
  storage: { type: 'filesystem', rootDirectory: '/var/lib/brainy' },
  vector: {
    recall: 'accurate',
    quantization: { bits: 8 }
  }
})

Tuning Knobs Summary

Setting	Values	When to change
vector.recall	'fast' / 'balanced' / 'accurate'	Trade recall for latency
vector.quantization.bits	4 / 8	Smaller index, lower memory
vector.persistMode	'immediate' / 'deferred'	Throughput vs. durability
storage.cache.maxSize	integer	Hot-path read cache size
storage.cache.ttl	ms	Cache freshness

Monitoring & Observability

const stats = await brain.stats()
// {
//   nounCount: 50000,
//   verbCount: 80000,
//   vectorIndex: { ... },
//   storage: { used: '45GB' }
// }

Troubleshooting

Issue: Slow queries

1. Switch to vector.recall: 'fast'
2. Enable SQ8 quantization
3. Increase the read cache (storage.cache.maxSize)
4. Consider the optional native vector provider via @soulcraft/cortex

Issue: Memory pressure

1. Enable vector.quantization: { bits: 4 } or { bits: 8 }
2. Reduce storage.cache.maxSize
3. Move to vector.persistMode: 'deferred' to batch writes

Issue: Slow startup after a crash

1. Use vector.persistMode: 'immediate' so the index file stays in sync with storage
2. Verify backup integrity periodically

Best Practices

1. One process = one rootDirectory — never share a directory between processes
2. Snapshot from your scheduler — Brainy doesn't ship cloud SDKs; use rclone / aws s3 sync / gsutil
3. Profile before tuning — recall: 'balanced' is right for most workloads
4. Install the native vector provider only when measured profiling shows it pays off

Summary

• Brainy 8.0 is a library, not a cluster
• Storage adapters: filesystem, memory, auto
• Vector tuning: recall, quantization, persistMode
• Backup is an operator-layer concern — snapshot rootDirectory