[](LICENSE)
[](https://nodejs.org/)
[](https://www.typescriptlang.org/)
[](CONTRIBUTING.md)
**A powerful graph & vector data platform for AI applications across any environment**
## โจ What is Brainy?
Imagine a database that thinks like you do - connecting ideas, finding patterns, and getting smarter over time. Brainy
is the **AI-native database** that brings vector search and knowledge graphs together in one powerful, ridiculously
easy-to-use package.
### ๐ NEW: Distributed Mode (v0.38+)
**Scale horizontally with zero configuration!** Brainy now supports distributed deployments with automatic coordination:
- **๐ Multi-Instance Coordination** - Multiple readers and writers working in harmony
- **๐ท๏ธ Smart Domain Detection** - Automatically categorizes data (medical, legal, product, etc.)
- **๐ Real-Time Health Monitoring** - Track performance across all instances
- **๐ Automatic Role Optimization** - Readers optimize for cache, writers for throughput
- **๐๏ธ Intelligent Partitioning** - Hash-based partitioning for perfect load distribution
### ๐ Why Developers Love Brainy
- **๐ง Zero-to-Smartโข** - No config files, no tuning parameters, no DevOps headaches. Brainy auto-detects your
environment and optimizes itself
- **๐ True Write-Once, Run-Anywhere** - Same code runs in React, Angular, Vue, Node.js, Deno, Bun, serverless, edge
workers, and even vanilla HTML
- **โก Scary Fast** - Handles millions of vectors with sub-millisecond search. Built-in GPU acceleration when available
- **๐ฏ Self-Learning** - Like having a database that goes to the gym. Gets faster and smarter the more you use it
- **๐ฎ AI-First Design** - Built for the age of embeddings, RAG, and semantic search. Your LLMs will thank you
- **๐ฎ Actually Fun to Use** - Clean API, great DX, and it does the heavy lifting so you can build cool stuff
### ๐ NEW: Ultra-Fast Search Performance + Auto-Configuration
**Your searches just got 100x faster AND Brainy now configures itself!** Advanced performance with zero setup:
- **๐ค Intelligent Auto-Configuration** - Detects environment and usage patterns, optimizes automatically
- **โก Smart Result Caching** - Repeated queries return in <1ms with automatic cache invalidation
- **๐ Cursor-Based Pagination** - Navigate millions of results with constant O(k) performance
- **๐ Real-Time Data Sync** - Cache automatically updates when data changes, even in distributed scenarios
- **๐ Performance Monitoring** - Built-in hit rate and memory usage tracking with adaptive optimization
- **๐ฏ Zero Breaking Changes** - All existing code works unchanged, just faster and smarter
```javascript
// Zero configuration - everything optimized automatically!
const brainy = new BrainyData() // Auto-detects environment & optimizes
await brainy.init()
// Caching happens automatically - no setup needed!
const results1 = await brainy.search('query', 10) // ~50ms first time
const results2 = await brainy.search('query', 10) // <1ms cached hit!
// Advanced pagination works instantly
const page1 = await brainy.searchWithCursor('query', 100)
const page2 = await brainy.searchWithCursor('query', 100, {
cursor: page1.cursor // Constant time, no matter how deep!
})
// Monitor auto-optimized performance
const stats = brainy.getCacheStats()
console.log(`Auto-tuned cache hit rate: ${(stats.search.hitRate * 100).toFixed(1)}%`)
```
## ๐ Quick Start (30 seconds!)
### Node.js TLDR
```bash
# Install
npm install brainy
# Use it
```
```javascript
import { createAutoBrainy, NounType, VerbType } from 'brainy'
const brainy = createAutoBrainy()
// Add data with Nouns (entities)
const catId = await brainy.add("Siamese cats are elegant and vocal", {
noun: NounType.Thing,
breed: "Siamese",
category: "animal"
})
const ownerId = await brainy.add("John loves his pets", {
noun: NounType.Person,
name: "John Smith"
})
// Connect with Verbs (relationships)
await brainy.addVerb(ownerId, catId, {
verb: VerbType.Owns,
since: "2020-01-01"
})
// Search by meaning
const results = await brainy.searchText("feline companions", 5)
// Search JSON documents by specific fields
const docs = await brainy.searchDocuments("Siamese", {
fields: ['breed', 'category'], // Search these fields
weights: { breed: 2.0 }, // Prioritize breed matches
limit: 10
})
// Find relationships
const johnsPets = await brainy.getVerbsBySource(ownerId, VerbType.Owns)
```
That's it! No config, no setup, Zero-to-Smartโข
### ๐ Distributed Mode Example (NEW!)
```javascript
// Writer Instance - Ingests data from multiple sources
const writer = createAutoBrainy({
storage: { s3Storage: { bucketName: 'my-bucket' } },
distributed: { role: 'writer' } // Explicit role for safety
})
// Reader Instance - Optimized for search queries
const reader = createAutoBrainy({
storage: { s3Storage: { bucketName: 'my-bucket' } },
distributed: { role: 'reader' } // 80% memory for cache
})
// Data automatically gets domain tags
await writer.add("Patient shows symptoms of...", {
diagnosis: "flu" // Auto-tagged as 'medical' domain
})
// Domain-aware search across all partitions
const results = await reader.search("medical symptoms", 10, {
filter: { domain: 'medical' } // Only search medical data
})
// Monitor health across all instances
const health = reader.getHealthStatus()
console.log(`Instance ${health.instanceId}: ${health.status}`)
```
## ๐ญ Key Features
### Core Capabilities
- **Vector Search** - Find semantically similar content using embeddings
- **Graph Relationships** - Connect data with meaningful relationships
- **JSON Document Search** - Search within specific fields with prioritization
- **Distributed Mode** - Scale horizontally with automatic coordination between instances
- **Real-Time Syncing** - WebSocket and WebRTC for distributed instances
- **Streaming Pipeline** - Process data in real-time as it flows through
- **Model Control Protocol** - Let AI models access your data
### Smart Optimizations
- **๐ค Intelligent Auto-Configuration** - Detects environment, usage patterns, and optimizes everything automatically
- **โก Runtime Performance Adaptation** - Continuously monitors and self-tunes based on real usage
- **๐ Distributed Mode Detection** - Automatically enables real-time updates for shared storage scenarios
- **๐ Workload-Aware Optimization** - Adapts cache size and TTL based on read/write patterns
- **๐ง Adaptive Learning** - Gets smarter with usage, learns from your data access patterns
- **#๏ธโฃ Intelligent Partitioning** - Hash-based partitioning for perfect load distribution
- **๐ฏ Role-Based Optimization** - Readers maximize cache, writers optimize throughput
- **๐ท๏ธ Domain-Aware Indexing** - Automatic categorization improves search relevance
- **๐๏ธ Multi-Level Caching** - Hot/warm/cold caching with predictive prefetching
- **๐พ Memory Optimization** - 75% reduction with compression for large datasets
### Developer Experience
- **TypeScript Support** - Fully typed API with generics
- **Extensible Augmentations** - Customize and extend functionality
- **REST API** - Web service wrapper for HTTP endpoints
- **Auto-Complete** - IntelliSense for all APIs and types
## ๐ฆ Installation
### Main Package
```bash
npm install brainy
```
### Optional: Offline Models Package
```bash
npm install @soulcraft/brainy-models
```
The `@soulcraft/brainy-models` package provides **offline access** to the Universal Sentence Encoder model, eliminating
network dependencies and ensuring consistent performance. Perfect for:
- **Air-gapped environments** - No internet? No problem
- **Consistent performance** - No network latency or throttling
- **Privacy-focused apps** - Keep everything local
- **High-reliability systems** - No external dependencies
```javascript
import { createAutoBrainy } from 'brainy'
import { BundledUniversalSentenceEncoder } from '@soulcraft/brainy-models'
// Use the bundled model for offline operation
const brainy = createAutoBrainy({
embeddingModel: BundledUniversalSentenceEncoder
})
```
## ๐จ Build Amazing Things
**๐ค AI Chat Applications** - Build ChatGPT-like apps with long-term memory and context awareness
**๐ Semantic Search Engines** - Search by meaning, not keywords. Find "that thing that's like a cat but bigger" โ
returns "tiger"
**๐ฏ Recommendation Engines** - "Users who liked this also liked..." but actually good
**๐งฌ Knowledge Graphs** - Connect everything to everything. Wikipedia meets Neo4j meets magic
**๐๏ธ Computer Vision Apps** - Store and search image embeddings. "Find all photos with dogs wearing hats"
**๐ต Music Discovery** - Find songs that "feel" similar. Spotify's Discover Weekly in your app
**๐ Smart Documentation** - Docs that answer questions. "How do I deploy to production?" โ relevant guides
**๐ก๏ธ Fraud Detection** - Find patterns humans can't see. Anomaly detection on steroids
**๐ Real-Time Collaboration** - Sync vector data across devices. Figma for AI data
**๐ฅ Medical Diagnosis Tools** - Match symptoms to conditions using embedding similarity
## ๐งฌ The Power of Nouns & Verbs
Brainy uses a **graph-based data model** that mirrors how humans think - with **Nouns** (entities) connected by **Verbs
** (relationships). This isn't just vectors in a void; it's structured, meaningful data.
### ๐ Nouns (What Things Are)
Nouns are your entities - the "things" in your data. Each noun has:
- A unique ID
- A vector representation (for similarity search)
- A type (Person, Document, Concept, etc.)
- Custom metadata
**Available Noun Types:**
| Category | Types | Use For |
|---------------------|-------------------------------------------------------------------|-------------------------------------------------------|
| **Core Entities** | `Person`, `Organization`, `Location`, `Thing`, `Concept`, `Event` | People, companies, places, objects, ideas, happenings |
| **Digital Content** | `Document`, `Media`, `File`, `Message`, `Content` | PDFs, images, videos, emails, posts, generic content |
| **Collections** | `Collection`, `Dataset` | Groups of items, structured data sets |
| **Business** | `Product`, `Service`, `User`, `Task`, `Project` | E-commerce, SaaS, project management |
| **Descriptive** | `Process`, `State`, `Role` | Workflows, conditions, responsibilities |
### ๐ Verbs (How Things Connect)
Verbs are your relationships - they give meaning to connections. Not just "these vectors are similar" but "this OWNS
that" or "this CAUSES that".
**Available Verb Types:**
| Category | Types | Examples |
|----------------|----------------------------------------------------------------------|------------------------------------------|
| **Core** | `RelatedTo`, `Contains`, `PartOf`, `LocatedAt`, `References` | Generic relations, containment, location |
| **Temporal** | `Precedes`, `Succeeds`, `Causes`, `DependsOn`, `Requires` | Time sequences, causality, dependencies |
| **Creation** | `Creates`, `Transforms`, `Becomes`, `Modifies`, `Consumes` | Creation, change, consumption |
| **Ownership** | `Owns`, `AttributedTo`, `CreatedBy`, `BelongsTo` | Ownership, authorship, belonging |
| **Social** | `MemberOf`, `WorksWith`, `FriendOf`, `Follows`, `Likes`, `ReportsTo` | Social networks, organizations |
| **Functional** | `Describes`, `Implements`, `Validates`, `Triggers`, `Serves` | Functions, implementations, services |
### ๐ก Why This Matters
```javascript
// Traditional vector DB: Just similarity
const similar = await vectorDB.search(embedding, 10)
// Result: [vector1, vector2, ...] - What do these mean? ๐คท
// Brainy: Similarity + Meaning + Relationships
const catId = await brainy.add("Siamese cat", {
noun: NounType.Thing,
breed: "Siamese"
})
const ownerId = await brainy.add("John Smith", {
noun: NounType.Person
})
await brainy.addVerb(ownerId, catId, {
verb: VerbType.Owns,
since: "2020-01-01"
})
// Now you can search with context!
const johnsPets = await brainy.getVerbsBySource(ownerId, VerbType.Owns)
const catOwners = await brainy.getVerbsByTarget(catId, VerbType.Owns)
```
## ๐ Distributed Mode (New!)
Brainy now supports **distributed deployments** with multiple specialized instances sharing the same data. Perfect for
scaling your AI applications across multiple servers.
### Distributed Setup
```javascript
// Single instance (no change needed!)
const brainy = createAutoBrainy({
storage: { s3Storage: { bucketName: 'my-bucket' } }
})
// Distributed mode requires explicit role configuration
// Option 1: Via environment variable
process.env.BRAINY_ROLE = 'writer' // or 'reader' or 'hybrid'
const brainy = createAutoBrainy({
storage: { s3Storage: { bucketName: 'my-bucket' } },
distributed: true
})
// Option 2: Via configuration
const writer = createAutoBrainy({
storage: { s3Storage: { bucketName: 'my-bucket' } },
distributed: { role: 'writer' } // Handles data ingestion
})
const reader = createAutoBrainy({
storage: { s3Storage: { bucketName: 'my-bucket' } },
distributed: { role: 'reader' } // Optimized for queries
})
// Option 3: Via read/write mode (role auto-inferred)
const writer = createAutoBrainy({
storage: { s3Storage: { bucketName: 'my-bucket' } },
writeOnly: true, // Automatically becomes 'writer' role
distributed: true
})
const reader = createAutoBrainy({
storage: { s3Storage: { bucketName: 'my-bucket' } },
readOnly: true, // Automatically becomes 'reader' role
distributed: true
})
```
### Key Distributed Features
**๐ฏ Explicit Role Configuration**
- Roles must be explicitly set (no dangerous auto-assignment)
- Can use environment variables, config, or read/write modes
- Clear separation between writers and readers
**#๏ธโฃ Hash-Based Partitioning**
- Handles multiple writers with different data types
- Even distribution across partitions
- No semantic conflicts with mixed data
**๐ท๏ธ Domain Tagging**
- Automatic domain detection (medical, legal, product, etc.)
- Filter searches by domain
- Logical separation without complexity
```javascript
// Data is automatically tagged with domains
await brainy.add({
symptoms: "fever",
diagnosis: "flu"
}, metadata) // Auto-tagged as 'medical'
// Search within specific domains
const medicalResults = await brainy.search(query, 10, {
filter: { domain: 'medical' }
})
```
**๐ Health Monitoring**
- Real-time health metrics
- Automatic dead instance cleanup
- Performance tracking
```javascript
// Get health status
const health = brainy.getHealthStatus()
// {
// status: 'healthy',
// role: 'reader',
// vectorCount: 1000000,
// cacheHitRate: 0.95,
// requestsPerSecond: 150
// }
```
**โก Role-Optimized Performance**
- **Readers**: 80% memory for cache, aggressive prefetching
- **Writers**: Optimized write batching, minimal cache
- **Hybrid**: Adaptive based on workload
### Deployment Examples
**Docker Compose**
```yaml
services:
writer:
image: myapp
environment:
BRAINY_ROLE: writer # Optional - auto-detects
reader:
image: myapp
environment:
BRAINY_ROLE: reader # Optional - auto-detects
scale: 5
```
**Kubernetes**
```yaml
# Automatically detects role from deployment type
apiVersion: apps/v1
kind: Deployment
metadata:
name: brainy-readers
spec:
replicas: 10 # Multiple readers
template:
spec:
containers:
- name: app
image: myapp
# Role auto-detected as 'reader' (multiple replicas)
```
**Benefits**
- โ **50-70% faster searches** with parallel readers
- โ **No coordination complexity** - Shared JSON config in S3
- โ **Zero downtime scaling** - Add/remove instances anytime
- โ **Automatic failover** - Dead instances cleaned up automatically
## ๐ค Why Choose Brainy?
### vs. Traditional Databases
โ **PostgreSQL with pgvector** - Requires complex setup, tuning, and DevOps expertise
โ **Brainy** - Zero config, auto-optimizes, works everywhere from browser to cloud
### vs. Vector Databases
โ **Pinecone/Weaviate/Qdrant** - Cloud-only, expensive, vendor lock-in
โ **Brainy** - Run locally, in browser, or cloud. Your choice, your data
### vs. Graph Databases
โ **Neo4j** - Great for graphs, no vector support
โ **Brainy** - Vectors + graphs in one. Best of both worlds
### vs. DIY Solutions
โ **Building your own** - Months of work, optimization nightmares
โ **Brainy** - Production-ready in 30 seconds
## ๐ Getting Started in 30 Seconds
### React
```jsx
import { createAutoBrainy } from 'brainy'
import { useEffect, useState } from 'react'
function SemanticSearch() {
const [brainy] = useState(() => createAutoBrainy())
const [results, setResults] = useState([])
const search = async (query) => {
const items = await brainy.searchText(query, 10)
setResults(items)
}
return (
search(e.target.value)}
placeholder="Search by meaning..." />
)
}
```
### Angular
```typescript
import { Component, OnInit } from '@angular/core'
import { createAutoBrainy } from 'brainy'
@Component({
selector: 'app-search',
template: `
