brainy/docs/SCALING.md

# Brainy Scaling Guide

> **One Line Summary**: Single-node by design — Brainy scales by getting the most out of one machine plus operator-layer backup.

## Table of Contents
- [Quick Start](#quick-start)
- [How Brainy Scales](#how-brainy-scales)
- [Storage Configurations](#storage-configurations)
- [Scaling Patterns](#scaling-patterns)
- [Real World Examples](#real-world-examples)

## Quick Start

### In-Memory
```typescript
import Brainy from '@soulcraft/brainy'
const brain = new Brainy({ storage: { type: 'memory' } })
```

### On-Disk (Default for Node)
```typescript
const brain = new Brainy({
  storage: { type: 'filesystem', path: './brainy-data' }
})
```

## How Brainy Scales

Brainy 8.0 is a **single-node library**. There is no cluster, no peer discovery, no S3 coordination. Scaling means:

- **Up**: give the process more RAM, CPU, and IOPS
- **Out**: stand up multiple independent Brainy instances behind your own service layer
- **Cold storage**: snapshot the on-disk artifact off-site so you can rehydrate elsewhere

The three knobs that matter most:

1. **`config.vector.recall`** — `'fast'`, `'balanced'`, or `'accurate'` (default `'balanced'`)
2. **`config.vector.persistMode`** — `'immediate'` for durability, `'deferred'` for throughput

The native vector provider (via the optional `@soulcraft/cortex` package) extends this with a higher-performing index — and its own at-scale acceleration such as on-disk compressed indexing — when installed.

## Measured Performance

Numbers below are **measured** by `tests/benchmarks/find-composition-scale.js` (a single
Node 22 process, in-memory storage, 384-dim vectors, `balanced` recall). They are the
open-core (pure-TypeScript) path — what you get from `@soulcraft/brainy` with no native
provider installed. Run it yourself: `node --max-old-space-size=8192 tests/benchmarks/find-composition-scale.js 100000`.

`find()` query latency, p50 / p95 (200 queries each):

| Query | 5,000 entities | 100,000 entities |
|---|---|---|
| Vector similarity (`{ vector }`) | 0.8 / 1.3 ms | 1.4 / 4.7 ms |
| Graph 1-hop (`{ connected }`) | 0.5 / 0.7 ms | 0.7 / 0.8 ms |
| Metadata filter (`{ where }`, low-selectivity) | 0.7 / 1.2 ms | 23.5 / 30.1 ms |
| Vector + metadata | 7.7 / 8.3 ms | 78.8 / 93.8 ms |

What the shape tells you:

- **Vector and graph lookups scale ~logarithmically** — they barely move from 5k to 100k,
  because HNSW search is ~O(ef·log n) and graph adjacency is O(degree).
- **Metadata-filtered paths scale with the size of the match set, not the database.** The
  benchmark's `category` filter matches ~10% of rows (10,000 at 100k); the cost is
  materializing that candidate set and running the vector search *inside* it (`find()` does
  metadata-first hard filtering, then ranks within the candidates — see
  [How find works](./FIND_SYSTEM.md)). A **high-selectivity** filter (few matches) is far
  cheaper; a 10%-of-everything filter is the worst case. This candidate-restricted search is
  precisely the path the native provider accelerates (Rust roaring-bitmap candidate
  intersection).
- **Composition is correct, not lossy.** Combining vector + metadata + graph returns exactly
  the entities satisfying all constraints — verified by
  `tests/integration/find-triple-composition.test.ts`.

Memory: ~62 KB resident per entity at 100k (6.2 GB RSS for 100k × 384-dim including the HNSW
graph, metadata index, and 100k edges).

**Scale ceiling (open-core).** The pure-JS HNSW *build* cost (~100 inserts/s at 384-dim on
one core) makes the in-process open-core path most appropriate up to ~10⁵–10⁶ entities.
*Query* latency stays low well beyond that, but for the 10⁸–10¹⁰ regime install the native
provider (`@soulcraft/cortex`, on-disk DiskANN) — same API, no code change. _Projected from
the two measured points, vector p50 at 1M is ~2 ms; metadata-heavy composition grows with
match-set size and is the path to move onto the native provider first._

## Storage Configurations

### Filesystem (Recommended for Production)
```typescript
const brain = new Brainy({
  storage: {
    type: 'filesystem',
    path: '/var/lib/brainy'
  }
})
```
- Stores everything in a sharded JSON tree under `path`
- Atomic writes via rename
- Survives process restarts
- Snapshot it off-site with `gsutil rsync`, `aws s3 sync`, `rclone`, or `tar` from your scheduler

### Memory
```typescript
const brain = new Brainy({ storage: { type: 'memory' } })
```
- Zero I/O, fastest possible
- No persistence — process exit discards everything
- Use for tests and ephemeral caches

### Auto
```typescript
const brain = new Brainy({
  storage: { type: 'auto', path: './data' }
})
```
- Picks `filesystem` when running on Node with a writable `path`
- Falls back to `memory` otherwise

## Scaling Patterns

### Stage 1: Prototype (Memory)
```typescript
const brain = new Brainy({ storage: { type: 'memory' } })
// Development, tests, <100K items
```

### Stage 2: Production (Filesystem)
```typescript
const brain = new Brainy({
  storage: { type: 'filesystem', path: '/var/lib/brainy' }
})
// Most production workloads up to ~10M entities on a single host
```

### Stage 3: Higher Throughput (Tune the Vector Index)
```typescript
const brain = new Brainy({
  storage: { type: 'filesystem', path: '/var/lib/brainy' },
  vector: {
    recall: 'fast',                // Trade recall for latency
    persistMode: 'deferred'        // Batch persistence
  }
})
```

### Stage 4: Multi-Instance (Operator-Layer)
Run multiple Brainy processes behind your own routing/service layer. Each process owns its own `path`. Sync each artifact off-site independently. Brainy itself does not coordinate between processes.

## Real World Examples

### Example 1: Single-Node App With Backup
```typescript
const brain = new Brainy({
  storage: { type: 'filesystem', path: '/var/lib/brainy' }
})
```
Schedule (cron / systemd timer):
```bash
*/15 * * * *  rclone sync /var/lib/brainy remote:brainy-backup
```

### Example 2: Tests
```typescript
const brain = new Brainy({ storage: { type: 'memory' } })
// Fast, no cleanup needed between runs
```

### Example 3: Multi-Tenant Service
Spin up one Brainy instance per tenant, each in its own directory:
```typescript
function brainForTenant(tenantId: string) {
  return new Brainy({
    storage: {
      type: 'filesystem',
      path: `/var/lib/brainy/${tenantId}`
    }
  })
}
```
Your service layer handles routing and isolation; Brainy stays simple.

### Example 4: Higher Recall at Scale
```typescript
const brain = new Brainy({
  storage: { type: 'filesystem', path: '/var/lib/brainy' },
  vector: {
    recall: 'accurate'
  }
})
```

## Tuning Knobs Summary

| Setting | Values | When to change |
|---------|--------|----------------|
| `vector.recall` | `'fast'` / `'balanced'` / `'accurate'` | Trade recall for latency |
| `vector.persistMode` | `'immediate'` / `'deferred'` | Throughput vs. durability |
| `storage.cache.maxSize` | integer | Hot-path read cache size |
| `storage.cache.ttl` | ms | Cache freshness |

## Monitoring & Observability

```typescript
const stats = await brain.stats()
// {
//   nounCount: 50000,
//   verbCount: 80000,
//   vectorIndex: { ... },
//   storage: { used: '45GB' }
// }
```

## Troubleshooting

### Issue: Slow queries
1. Switch to `vector.recall: 'fast'`
2. Increase the read cache (`storage.cache.maxSize`)
3. Consider the optional native vector provider via `@soulcraft/cortex`

### Issue: Memory pressure
1. Reduce `storage.cache.maxSize`
2. Move to `vector.persistMode: 'deferred'` to batch writes
3. Consider the optional native vector provider via `@soulcraft/cortex` for at-scale index acceleration

### Issue: Slow startup after a crash
1. Use `vector.persistMode: 'immediate'` so the index file stays in sync with storage
2. Verify backup integrity periodically

## Best Practices

1. **One process = one `path`** — never share a directory between processes
2. **Snapshot from your scheduler** — Brainy doesn't ship cloud SDKs; use `rclone` / `aws s3 sync` / `gsutil`
3. **Profile before tuning** — `recall: 'balanced'` is right for most workloads
4. **Install the native vector provider only when measured profiling shows it pays off**

## Summary

- Brainy 8.0 is a **library**, not a cluster
- Storage adapters: `filesystem`, `memory`, `auto`
- Vector tuning: `recall`, `persistMode`
- Backup is an operator-layer concern — snapshot `path`