brainy/docs/SCALING.md

Brainy Scaling Guide

One Line Summary: Single-node by design — Brainy scales by getting the most out of one machine plus operator-layer backup.

Table of Contents

• Quick Start
• How Brainy Scales
• Storage Configurations
• Scaling Patterns
• Real World Examples

Quick Start

In-Memory

import Brainy from '@soulcraft/brainy'
const brain = new Brainy({ storage: { type: 'memory' } })

On-Disk (Default for Node)

const brain = new Brainy({
  storage: { type: 'filesystem', path: './brainy-data' }
})

How Brainy Scales

Brainy 8.0 is a single-node library. There is no cluster, no peer discovery, no S3 coordination. Scaling means:

• Up: give the process more RAM, CPU, and IOPS
• Out: stand up multiple independent Brainy instances behind your own service layer
• Cold storage: snapshot the on-disk artifact off-site so you can rehydrate elsewhere

The three knobs that matter most:

1. config.vector.recall — 'fast', 'balanced', or 'accurate' (default 'balanced')
2. config.vector.persistMode — 'immediate' for durability, 'deferred' for throughput

The native vector provider (via the optional @soulcraft/cor package) extends this with a higher-performing index — and its own at-scale acceleration such as on-disk compressed indexing — when installed.

Measured Performance

Numbers below are measured by tests/benchmarks/find-composition-scale.js (a single Node 22 process, in-memory storage, 384-dim vectors, balanced recall). They are the open-core (pure-TypeScript) path — what you get from @soulcraft/brainy with no native provider installed. Run it yourself: node --max-old-space-size=8192 tests/benchmarks/find-composition-scale.js 100000.

find() query latency, p50 / p95 (200 queries each):

Query	5,000 entities	100,000 entities
Vector similarity ({ vector })	0.8 / 1.3 ms	1.4 / 4.7 ms
Graph 1-hop ({ connected })	0.5 / 0.7 ms	0.7 / 0.8 ms
Metadata filter ({ where }, low-selectivity)	0.7 / 1.2 ms	23.5 / 30.1 ms
Vector + metadata	7.7 / 8.3 ms	78.8 / 93.8 ms

What the shape tells you:

• Vector and graph lookups scale ~logarithmically — they barely move from 5k to 100k, because HNSW search is ~O(ef·log n) and graph adjacency is O(degree).
• Metadata-filtered paths scale with the size of the match set, not the database. The benchmark's category filter matches ~10% of rows (10,000 at 100k); the cost is materializing that candidate set and running the vector search inside it (find() does metadata-first hard filtering, then ranks within the candidates — see How find works). A high-selectivity filter (few matches) is far cheaper; a 10%-of-everything filter is the worst case. This candidate-restricted search is precisely the path the native provider accelerates (Rust roaring-bitmap candidate intersection).
• Composition is correct, not lossy. Combining vector + metadata + graph returns exactly the entities satisfying all constraints — verified by tests/integration/find-triple-composition.test.ts.

Memory: ~62 KB resident per entity at 100k (6.2 GB RSS for 100k × 384-dim including the HNSW graph, metadata index, and 100k edges).

Scale ceiling (open-core). The pure-JS HNSW build cost (~100 inserts/s at 384-dim on one core) makes the in-process open-core path most appropriate up to ~10⁵–10⁶ entities. Query latency stays low well beyond that, but for the 10⁸–10¹⁰ regime install the native provider (@soulcraft/cor, on-disk DiskANN) — same API, no code change. Projected from the two measured points, vector p50 at 1M is ~2 ms; metadata-heavy composition grows with match-set size and is the path to move onto the native provider first.

Storage Configurations

Filesystem (Recommended for Production)

const brain = new Brainy({
  storage: {
    type: 'filesystem',
    path: '/var/lib/brainy'
  }
})

• Stores everything in a sharded JSON tree under path
• Atomic writes via rename
• Survives process restarts
• Snapshot it off-site with gsutil rsync, aws s3 sync, rclone, or tar from your scheduler

Memory

const brain = new Brainy({ storage: { type: 'memory' } })

• Zero I/O, fastest possible
• No persistence — process exit discards everything
• Use for tests and ephemeral caches

Auto

const brain = new Brainy({
  storage: { type: 'auto', path: './data' }
})

• Picks filesystem when running on Node with a writable path
• Falls back to memory otherwise

Scaling Patterns

Stage 1: Prototype (Memory)

const brain = new Brainy({ storage: { type: 'memory' } })
// Development, tests, <100K items

Stage 2: Production (Filesystem)

const brain = new Brainy({
  storage: { type: 'filesystem', path: '/var/lib/brainy' }
})
// Most production workloads up to ~10M entities on a single host

Stage 3: Higher Throughput (Tune the Vector Index)

const brain = new Brainy({
  storage: { type: 'filesystem', path: '/var/lib/brainy' },
  vector: {
    recall: 'fast',                // Trade recall for latency
    persistMode: 'deferred'        // Batch persistence
  }
})

Stage 4: Multi-Instance (Operator-Layer)

Run multiple Brainy processes behind your own routing/service layer. Each process owns its own path. Sync each artifact off-site independently. Brainy itself does not coordinate between processes.

Real World Examples

Example 1: Single-Node App With Backup

const brain = new Brainy({
  storage: { type: 'filesystem', path: '/var/lib/brainy' }
})

Schedule (cron / systemd timer):

*/15 * * * *  rclone sync /var/lib/brainy remote:brainy-backup

Example 2: Tests

const brain = new Brainy({ storage: { type: 'memory' } })
// Fast, no cleanup needed between runs

Example 3: Multi-Tenant Service

Spin up one Brainy instance per tenant, each in its own directory:

function brainForTenant(tenantId: string) {
  return new Brainy({
    storage: {
      type: 'filesystem',
      path: `/var/lib/brainy/${tenantId}`
    }
  })
}

Your service layer handles routing and isolation; Brainy stays simple.

Example 4: Higher Recall at Scale

const brain = new Brainy({
  storage: { type: 'filesystem', path: '/var/lib/brainy' },
  vector: {
    recall: 'accurate'
  }
})

Tuning Knobs Summary

Setting	Values	When to change
vector.recall	'fast' / 'balanced' / 'accurate'	Trade recall for latency
vector.persistMode	'immediate' / 'deferred'	Throughput vs. durability
storage.cache.maxSize	integer	Hot-path read cache size
storage.cache.ttl	ms	Cache freshness

Monitoring & Observability

const stats = await brain.stats()
// {
//   nounCount: 50000,
//   verbCount: 80000,
//   vectorIndex: { ... },
//   storage: { used: '45GB' }
// }

Troubleshooting

Issue: Slow queries

1. Switch to vector.recall: 'fast'
2. Increase the read cache (storage.cache.maxSize)
3. Consider the optional native vector provider via @soulcraft/cor

Issue: Memory pressure

1. Reduce storage.cache.maxSize
2. Move to vector.persistMode: 'deferred' to batch writes
3. Consider the optional native vector provider via @soulcraft/cor for at-scale index acceleration

Issue: Slow startup after a crash

1. Use vector.persistMode: 'immediate' so the index file stays in sync with storage
2. Verify backup integrity periodically

Best Practices

1. One process = one path — never share a directory between processes
2. Snapshot from your scheduler — Brainy doesn't ship cloud SDKs; use rclone / aws s3 sync / gsutil
3. Profile before tuning — recall: 'balanced' is right for most workloads
4. Install the native vector provider only when measured profiling shows it pays off

Summary

• Brainy 8.0 is a library, not a cluster
• Storage adapters: filesystem, memory, auto
• Vector tuning: recall, persistMode
• Backup is an operator-layer concern — snapshot path