brainy/README.md

Brainy

Three database paradigms. One API. Zero configuration.

Built because we were tired of stitching together Pinecone + Neo4j + MongoDB and spending weeks on configuration before writing a single line of business logic. Brainy unifies vector search, graph traversal, and metadata filtering so you don't have to choose.

New here? → What is Brainy? — plain-language overview, no jargon

---

Install

npm install @soulcraft/brainy

Quick Start

import { Brainy, NounType, VerbType } from '@soulcraft/brainy'

const brain = new Brainy()
await brain.init()

// Add knowledge — text auto-embeds, metadata auto-indexes
const reactId = await brain.add({
  data: 'React is a JavaScript library for building user interfaces',
  type: NounType.Concept,
  metadata: { category: 'frontend', year: 2013 }
})

const nextId = await brain.add({
  data: 'Next.js framework for React with server-side rendering',
  type: NounType.Concept,
  metadata: { category: 'framework', year: 2016 }
})

// Create a relationship
await brain.relate({ from: nextId, to: reactId, type: VerbType.BuiltOn })

// Query all three paradigms at once
const results = await brain.find({
  query: 'modern frontend frameworks',            // Vector similarity
  where: { year: { greaterThan: 2015 } },         // Metadata filtering
  connected: { to: reactId, depth: 2 }            // Graph traversal
})

Full API Reference | soulcraft.com/docs

---

Three Indexes, One Query

Every piece of knowledge lives in three indexes simultaneously:

• data → Vector index — Content for semantic search. Strings auto-embed into 384-dim vectors. Queried with find({ query: '...' }).
• metadata → Metadata index — Structured fields for filtering. O(1) lookups. Queried with find({ where: { ... } }).
• relate() → Graph index — Typed, directed relationships between entities. Traversed with find({ connected: { ... } }).

// Data → vector index (semantic search)
const articleId = await brain.add({
  data: 'A deep dive into transformer architectures',
  type: NounType.Document,
  metadata: { author: 'Dr. Chen', year: 2024, tags: ['AI'] }  // → metadata index
})

// Relationships → graph index
await brain.relate({ from: authorId, to: articleId, type: VerbType.Authored })

// Query all three at once
brain.find({
  query: 'attention mechanisms',                  // Vector similarity
  where: { year: { greaterThan: 2023 } },         // Metadata filter
  connected: { from: authorId, depth: 1 }         // Graph traversal
})

Data Model Reference | Query Operators

---

Features

Triple Intelligence

Vector search + graph traversal + metadata filtering in every query. No stitching services together — one find() call combines all three.

const results = await brain.find({
  query: 'machine learning',
  where: { department: 'engineering', level: 'senior' },
  connected: { from: teamLeadId, via: VerbType.WorksWith, depth: 2 }
})

Hybrid Search

Automatically combines keyword (text) and semantic (vector) search. No configuration needed.

await brain.find({ query: 'David Smith' })             // Auto: text + semantic
await brain.find({ query: 'AI concepts', searchMode: 'semantic' })  // Semantic only
await brain.find({ query: 'exact id', searchMode: 'text' })         // Text only

Query Operators

Filter metadata with equality, comparison, array, existence, pattern, and logical operators:

await brain.find({
  where: {
    status: 'active',                          // Exact match
    score: { greaterThan: 90 },                // Comparison
    tags: { contains: 'ai' },                  // Array
    anyOf: [{ role: 'admin' }, { role: 'owner' }]  // Logical OR
  }
})

Query Operators Reference — all operators with indexed/in-memory matrix

Graph Relationships

Typed, directed edges between entities. Traverse connections at any depth.

await brain.relate({ from: personId, to: projectId, type: VerbType.WorksOn })

const results = await brain.find({
  connected: { from: personId, via: VerbType.WorksOn, depth: 3 }
})

Database as a Value

The whole database, pinned as an immutable value. Snapshot isolation, time travel, atomic transactions, instant hard-link snapshots.

const db = brain.now()                         // Pin current state — O(1)

// Atomic multi-write transaction (all-or-nothing, with CAS)
await brain.transact([
  { op: 'update', id: orderId, metadata: { status: 'paid' } },
  { op: 'relate', from: invoiceId, to: orderId, type: VerbType.References, subtype: 'billing' }
], { meta: { author: 'billing-service' }, ifAtGeneration: db.generation })

await db.get(orderId)                          // Still 'pending' — pinned, forever
await brain.get(orderId)                       // 'paid' — live

// Time travel: full query surface at any past state
const yesterday = await brain.asOf(new Date(Date.now() - 86_400_000))
const past = await yesterday.find({ query: 'unpaid orders' })

// What-if: speculative writes, nothing touches disk
const whatIf = await db.with([{ op: 'remove', id: orderId }])

// Instant backup: hard-link snapshot, opens read-only with Brainy.load()
await brain.now().persist('/backups/today')

Consistency Model | Snapshots & Time Travel

Virtual Filesystem

File operations with semantic search built in.

const vfs = brain.vfs

await vfs.writeFile('/docs/readme.md', 'Project documentation')
const content = await vfs.readFile('/docs/readme.md')
const tree = await vfs.getTreeStructure('/docs', { maxDepth: 3 })

// Semantic file search
const matches = await vfs.search('React components with hooks')

VFS Quick Start | Common Patterns

Import Anything

CSV, Excel, PDF, URLs — auto-detected format, auto-classified entities.

await brain.import('customers.csv')
await brain.import('sales-data.xlsx', { excelSheets: ['Q1', 'Q2'] })
await brain.import('research-paper.pdf', { pdfExtractTables: true })
await brain.import('https://api.example.com/data.json')

Import Guide

Entity Extraction

AI-powered named entity recognition with 4-signal ensemble scoring.

const entities = await brain.extractEntities('John Smith founded Acme Corp in New York')
// [
//   { text: 'John Smith', type: NounType.Person, confidence: 0.95 },
//   { text: 'Acme Corp', type: NounType.Organization, confidence: 0.92 },
//   { text: 'New York', type: NounType.Location, confidence: 0.88 }
// ]

Neural Extraction Guide

Plugin System

Optional native acceleration via @soulcraft/cortex — SIMD distance calculations, CRoaring bitmaps, Candle ML embeddings.

const brain = new Brainy({ plugins: ['@soulcraft/cortex'] })
await brain.init()

Plugins are opt-in. Brainy never auto-imports packages unless listed in plugins.

Plugin Documentation

---

Type System

42 noun types and 127 verb types form a universal knowledge protocol:

42 Nouns × 127 Verbs = 5,334 base relationship combinations

Model any domain — healthcare (Patient → diagnoses → Condition), finance (Account → transfers → Transaction), education (Student → completes → Course), or your own.

Subtypes — sub-classification within a NounType or VerbType

Both noun types and verb types are intentionally coarse. Use the top-level subtype field to sub-classify entities AND relationships within a type — flat string, no hierarchy, your choice of vocabulary:

// Nouns: sub-classify entities
await brain.add({
  data: 'Avery Brooks — runs the AI lab',
  type: NounType.Person,
  subtype: 'employee'                 // 'customer', 'vendor', 'contractor', …
})

// Verbs: sub-classify relationships
await brain.relate({
  from: ceoId,
  to: vpId,
  type: VerbType.ReportsTo,
  subtype: 'direct'                   // 'dotted-line', 'matrix', …
})

// Filter on the fast path — column-store hit, not metadata fallback:
const employees = await brain.find({ type: NounType.Person, subtype: 'employee' })
const directReports = await brain.getRelations({ from: ceoId, subtype: 'direct' })

// O(1) counts via the persisted rollups:
brain.counts.bySubtype(NounType.Person)
// → { employee: 12, customer: 847, vendor: 34 }

brain.counts.byRelationshipSubtype(VerbType.ReportsTo)
// → { direct: 12, 'dotted-line': 3 }

Enforce the pairing. Register a vocabulary per type or turn on brain-wide strict mode to ensure every entity AND relationship has both type AND subtype:

// Per-type rule with a closed vocabulary
brain.requireSubtype(NounType.Person, { values: ['employee', 'customer'], required: true })

// 8.0 default: every write requires a subtype. Exempt genuine catch-all types…
const brain = new Brainy({ requireSubtype: { except: [NounType.Thing] } })

// …or opt out while migrating pre-8.0 data, then audit and back-fill:
const legacy = new Brainy({ requireSubtype: false })
await legacy.audit()                                      // gaps, grouped by type
await legacy.fillSubtypes({ [NounType.Person]: 'unspecified' })  // close them

For other facets you want counted (status, source, role), register them with brain.trackField(name). Renaming an existing convention to subtype? Use brain.migrateField({from, to, entityKind: 'both'}) to walk nouns AND verbs in one pass. Full guide: Subtypes & Facets.

Noun-Verb Taxonomy | Stage 3 Canonical Reference

---

Storage: Memory and Filesystem

The same API at every scale. Change one config line to go from prototype to production.

Development — Zero Config

const brain = new Brainy()

Production — Filesystem (gzip compression on by default)

const brain = new Brainy({
  storage: { type: 'filesystem', rootDirectory: './data' }
})

Backups and Portability — Snapshots

const db = brain.now()
await db.persist('/backups/2026-06-11')   // instant hard-link snapshot
await db.release()

const snapshot = await Brainy.load('/backups/2026-06-11')  // read-only Db
const hits = await snapshot.search('quarterly invoices')
await snapshot.release()

A snapshot directory is self-contained — copy it to another machine, open it with Brainy.load(), or restore it wholesale with brain.restore(path, { confirm: true }).

Performance benchmarks and capacity planning in docs/PERFORMANCE.md.

Capacity Planning

---

Use Cases

• AI agents — Persistent memory with semantic recall and relationship tracking
• Knowledge bases — Auto-linking, semantic search, relationship-aware navigation
• Semantic search — Find by meaning across codebases, documents, or media
• Enterprise knowledge — CRM, product catalogs, institutional memory
• Interactive experiences — Game worlds, NPCs, and characters that remember
• Content platforms — Similarity-based discovery, intelligent tagging

---

Documentation

Start Here

• Brainy explained simply — Plain-language overview, no jargon, no code

Core

• API Reference — Every method with parameters, returns, and examples
• Data Model — Entity structure, data vs metadata
• Query Operators — All BFO operators with examples
• Find System — Natural language find() and hybrid search

Architecture

• Architecture Overview — System design and components
• Triple Intelligence — Vector + graph + metadata unified query
• Noun-Verb Taxonomy — Universal type system
• Data Storage Architecture — Type-aware indexing and HNSW

Virtual Filesystem

• VFS Quick Start — Build file explorers that never crash
• VFS Core — Full VFS API reference
• Semantic VFS — AI-powered file navigation

Guides

• Import Anything — CSV, Excel, PDF, URLs
• Framework Integration — React, Vue, Angular, Svelte
• Natural Language Queries — Master the find() method

Operations

• Capacity Planning — Memory, storage, and scaling
• Performance — Benchmarks and architecture details

---

Requirements

Bun 1.0+ (recommended) or Node.js 22 LTS

bun install @soulcraft/brainy    # Bun — best performance
npm install @soulcraft/brainy    # Node.js — fully supported

Brainy 8.0 is server-only. Browser support (OPFS storage, Web Workers, in-browser WASM embeddings) was removed in 8.0 — the 7.x line remains available on npm if you need it.

Single-Writer Model

Brainy is single-writer, many-reader on filesystem storage. One writer holds an exclusive lock on the data directory; any number of readers can inspect it concurrently. Opening a second writer throws with the PID of the existing one.

// Live application — writer mode is the default
const brain = new Brainy({ storage: { type: 'filesystem', rootDirectory: '/data/brain' } })
await brain.init()

// Out-of-band diagnostics from a separate process — safe to run while the
// writer is live
const reader = await Brainy.openReadOnly({
  storage: { type: 'filesystem', rootDirectory: '/data/brain' }
})
await reader.requestFlush({ timeoutMs: 5000 })
const stats = await reader.stats()

For incident debugging, use the brainy inspect CLI:

brainy inspect stats   /data/brain
brainy inspect find    /data/brain --where '{"entityType":"booking"}'
brainy inspect explain /data/brain --where '{"entityType":"booking"}'
brainy inspect health  /data/brain

See the multi-process model and the inspection guide for the full story, including stale-lock detection, the cross-process flush RPC, and what's not yet enforced on cloud storage backends.

Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

License

MIT © Brainy Contributors