Critical fix for metadata indexing that was creating 60+ chunk files per entity.
Root cause: Vector embeddings (384-dimensional arrays) were being indexed in
metadata, causing each dimension to create a separate chunk file with numeric
field names ("0", "1", "2", etc.).
Changes:
- Modified extractIndexableFields() to exclude vector/embedding fields
- Added NEVER_INDEX set: ['vector', 'embedding', 'embeddings', 'connections']
- Added safety check to skip arrays > 10 elements
- Preserves small array indexing (tags, categories, roles)
Impact:
- Reduces metadata files from 69,429 → ~1,200 (58x reduction)
- Fixes server initialization hangs
- Fixes metadata batch loading stalling at batch 23
- Fixes VFS getDescendants() hanging with large datasets
- Fixes Graph View UI not loading
Test Results:
- 7/7 integration tests passing
- Verified: 6 chunk files for 10 entities (was 7,210 before fix)
- 611/622 unit tests passing
Files Modified:
- src/utils/metadataIndex.ts - Core fix
- src/coreTypes.ts - HNSWVerb type enforcement with VerbType enum
- src/storage/adapters/* - Include core relational fields in HNSWVerb
- src/storage/adapters/baseStorageAdapter.ts - Type enforcement (HNSWNoun, GraphVerb)
- tests/integration/metadata-vector-exclusion.test.ts - Comprehensive test coverage
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
||
|---|---|---|
| bin | ||
| docs | ||
| examples | ||
| models-cache/Xenova/all-MiniLM-L6-v2 | ||
| scripts | ||
| src | ||
| tests | ||
| .aiignore | ||
| .dockerignore | ||
| .gitignore | ||
| .npmignore | ||
| .nvmrc | ||
| .versionrc.json | ||
| brainy.png | ||
| CHANGELOG.md | ||
| CONTRIBUTING.md | ||
| docker-compose.yml | ||
| Dockerfile | ||
| eslint.config.js | ||
| EXPLORATION_SUMMARY.md | ||
| LICENSE | ||
| package-lock.json | ||
| package.json | ||
| README.md | ||
| README_STORAGE_EXPLORATION.md | ||
| SECURITY.md | ||
| STORAGE_ADAPTER_QUICK_REFERENCE.md | ||
| STORAGE_FILES_REFERENCE.md | ||
| tsconfig.cli.json | ||
| tsconfig.json | ||
| vitest.config.memory.ts | ||
| vitest.config.ts | ||
Brainy
🧠 Brainy - The Knowledge Operating System
The world's first Knowledge Operating System where every piece of knowledge - files, concepts, entities, ideas - exists as living information that understands itself, evolves over time, and connects to everything related. Built on revolutionary Triple Intelligence™ that unifies vector similarity, graph relationships, and document filtering in one magical API.
Why Brainy Changes Everything: Traditional systems trap knowledge in files or database rows. Brainy liberates it. Your characters exist across stories. Your concepts span projects. Your APIs remember their evolution. Every piece of knowledge - whether it's code, prose, or pure ideas - lives, breathes, and connects in a unified intelligence layer where everything understands its meaning, remembers its history, and relates to everything else.
Framework-first design. Built for modern web development with zero configuration and automatic framework compatibility. O(log n) performance, <10ms search latency, production-ready.
🎉 Key Features
🚀 NEW in 3.47.0: Billion-Scale Type-Aware HNSW
87% memory reduction for billion-scale deployments with 10x faster queries:
-
🎯 Type-Aware Vector Index: Separate HNSW graphs per entity type for massive memory savings
- Memory @ 1B scale: 384GB → 50GB (-87% / -334GB)
- Single-type queries: 10x faster (search 100M nodes instead of 1B)
- Multi-type queries: 5-8x faster (search subset of types)
- All-types queries: ~3x faster (31 smaller graphs vs 1 large graph)
-
⚡ Optimized Rebuild: Type-filtered pagination for 31x faster index rebuilding
- Before: 31B reads (UNACCEPTABLE)
- After: 1B reads with type filtering (CORRECT)
- Parallel type rebuilds: 10-20 minutes for all types
- Lazy loading: 15 minutes for top 2 types only
-
📊 Production-Ready: Comprehensive testing and zero breaking changes
- 47 new tests (33 unit + 14 integration) - all passing
- Backward compatible - opt-in via configuration
- Works with all storage backends (FileSystem, S3, GCS, R2, Memory, OPFS)
⚡ NEW in 3.36.0: Production-Scale Memory & Performance
Enterprise-grade adaptive sizing and zero-overhead optimizations:
-
🎯 Adaptive Memory Sizing: Auto-scales from 2GB to 128GB+ based on available system resources
- Container-aware (Docker/K8s cgroups v1/v2 detection)
- Environment-smart (development 25%, container 40%, production 50% allocation)
- Model memory accounting (150MB Q8, 250MB FP32 reserved before cache)
-
⚡ Sync Fast Path: Zero async overhead when vectors are cached
- Intelligent sync/async branching - synchronous when data is in memory
- Falls back to async only when loading from storage
- Massive performance win for hot paths (vector search, distance calculations)
-
📊 Production Monitoring: Comprehensive diagnostics
getCacheStats()- UnifiedCache hit rates, fairness metrics, memory pressure- Actionable recommendations for tuning
- Tracks model memory, cache efficiency, and competition across indexes
-
🛡️ Zero Breaking Changes: All optimizations are internal - your code stays the same
- Public API unchanged
- Automatic memory detection and allocation
- Progressive enhancement for existing applications
📖 Operations Guide → | 🎯 Migration Guide →
🚀 NEW in 3.21.0: Enhanced Import & Neural Processing
- 📊 Progress Tracking: Unified progress reporting with automatic time estimation
- ⚡ Entity Caching: 10-100x speedup on repeated entity extraction
- 🔗 Relationship Confidence: Multi-factor confidence scoring (0-1 scale)
- 📝 Evidence Tracking: Understand why relationships were detected
- 🎯 Production Ready: Fully backward compatible, opt-in features
🧠 Triple Intelligence™ Engine
- Vector Search: HNSW-powered semantic similarity
- Graph Relationships: Navigate connected knowledge
- Document Filtering: MongoDB-style metadata queries
- Unified API: All three in a single query interface
🎯 Clean API Design
- Modern Syntax:
brain.add(),brain.find(),brain.relate() - Type Safety: Full TypeScript integration
- Zero Config: Works out of the box with intelligent storage auto-detection
- Consistent Parameters: Clean, predictable API surface
⚡ Performance & Reliability
- <10ms Search: Fast semantic queries
- 384D Vectors: Optimized embeddings (all-MiniLM-L6-v2)
- Built-in Caching: Intelligent result caching + new entity extraction cache
- Production Ready: Thoroughly tested core functionality
⚡ Quick Start - Zero Configuration
npm install @soulcraft/brainy
🎯 True Zero Configuration
import { Brainy, NounType } from '@soulcraft/brainy'
// Just this - auto-detects everything!
const brain = new Brainy()
await brain.init()
// Add entities with automatic embedding
const jsId = await brain.add({
data: "JavaScript is a programming language",
nounType: NounType.Concept,
metadata: {
type: "language",
year: 1995,
paradigm: "multi-paradigm"
}
})
const nodeId = await brain.add({
data: "Node.js runtime environment",
nounType: NounType.Concept,
metadata: {
type: "runtime",
year: 2009,
platform: "server-side"
}
})
// Create relationships between entities
await brain.relate({
from: nodeId,
to: jsId,
type: "executes",
metadata: {
since: 2009,
performance: "high"
}
})
// Natural language search with graph relationships
const results = await brain.find({query: "programming languages used by server runtimes"})
// Triple Intelligence: vector + metadata + relationships
const filtered = await brain.find({
query: "JavaScript", // Vector similarity
where: {type: "language"}, // Metadata filtering
connected: {from: nodeId, depth: 1} // Graph relationships
})
🌐 Framework Integration
Brainy is framework-first! Works seamlessly with any modern JavaScript framework:
⚛️ React & Next.js
import { Brainy } from '@soulcraft/brainy'
function SearchComponent() {
const [brain] = useState(() => new Brainy())
useEffect(() => {
brain.init()
}, [])
const handleSearch = async (query) => {
const results = await brain.find(query)
setResults(results)
}
}
🟢 Vue.js & Nuxt.js
import { Brainy } from '@soulcraft/brainy'
export default {
async mounted() {
this.brain = new Brainy()
await this.brain.init()
},
methods: {
async search(query) {
return await this.brain.find(query)
}
}
}
🅰️ Angular
import { Injectable } from '@angular/core'
import { Brainy } from '@soulcraft/brainy'
@Injectable({ providedIn: 'root' })
export class BrainyService {
private brain = new Brainy()
async init() {
await this.brain.init()
}
async search(query: string) {
return await this.brain.find(query)
}
}
🔥 Other Frameworks
Brainy works with any framework that supports ES6 imports: Svelte, Solid.js, Qwik, Fresh, and more!
Framework Compatibility:
- ✅ All modern bundlers (Webpack, Vite, Rollup, Parcel)
- ✅ SSR/SSG (Next.js, Nuxt, SvelteKit, Astro)
- ✅ Edge runtimes (Vercel Edge, Cloudflare Workers)
- ✅ Browser and Node.js environments
📋 System Requirements
Node.js Version: 22 LTS or later (recommended)
- ✅ Node.js 22 LTS - Fully supported and recommended for production
- ✅ Node.js 20 LTS - Compatible (maintenance mode)
- ❌ Node.js 24 - Not supported (known ONNX runtime compatibility issues)
Important: Brainy uses ONNX runtime for AI embeddings. Node.js 24 has known compatibility issues that cause crashes during inference operations. We recommend Node.js 22 LTS for maximum stability.
If using nvm: nvm use (we provide a .nvmrc file)
🚀 Key Features
World's First Triple Intelligence™ Engine
The breakthrough that enables The Knowledge Operating System:
- Vector Search: Semantic similarity with HNSW indexing
- Graph Relationships: Navigate connected knowledge like Neo4j
- Document Filtering: MongoDB-style queries with O(log n) performance
- Unified in ONE API: No separate queries, no complex joins
- First to solve this: Others do vector OR graph OR document—we do ALL
The Knowledge Operating System with Infinite Expressiveness
Enabled by Triple Intelligence, standardized for everyone:
- 31 Noun Types × 40 Verb Types: 1,240 base combinations
- ∞ Expressiveness: Unlimited metadata = model ANY data
- One Language: All tools, augmentations, AI models speak the same types
- Perfect Interoperability: Move data between any Brainy instance
- No Schema Lock-in: Evolve without migrations
Natural Language Understanding
// Ask questions naturally
await brain.find("Show me recent React components with tests")
await brain.find("Popular JavaScript libraries similar to Vue")
await brain.find("Documentation about authentication from last month")
🧠🌐 Virtual Filesystem - Intelligent File Management
Build file explorers, IDEs, and knowledge systems that never crash from infinite recursion.
- Tree-Aware Operations: Safe directory listing prevents recursive loops
- Semantic Search: Find files by content, not just filename
- Production Storage: Filesystem and cloud storage for real applications
- Zero-Config: Works out of the box with intelligent defaults
import { Brainy } from '@soulcraft/brainy'
// ✅ CORRECT: Use persistent storage for file systems
const brain = new Brainy({
storage: {
type: 'filesystem', // Persisted to disk
path: './brainy-data' // Your file storage
}
})
await brain.init()
const vfs = brain.vfs()
await vfs.init()
// ✅ Safe file operations
await vfs.writeFile('/projects/app/index.js', 'console.log("Hello")')
await vfs.mkdir('/docs')
await vfs.writeFile('/docs/README.md', '# My Project')
// ✅ NEVER crashes: Tree-aware directory listing
const children = await vfs.getDirectChildren('/projects')
// Returns only direct children, never the directory itself
// ✅ Build file explorers safely
const tree = await vfs.getTreeStructure('/projects', {
maxDepth: 3, // Prevent deep recursion
sort: 'name' // Organized results
})
// ✅ Semantic file search
const reactFiles = await vfs.search('React components with hooks')
const docs = await vfs.search('API documentation', {
path: '/docs' // Search within specific directory
})
// ✅ Connect related files
await vfs.addRelationship('/src/auth.js', '/tests/auth.test.js', 'tested-by')
// Perfect for: File explorers, IDEs, documentation systems, code analysis
🚨 Prevents Common Mistakes:
- ❌ No infinite recursion in file trees (like brain-cloud team experienced)
- ❌ No data loss from memory storage
- ❌ No performance issues with large directories
- ❌ No need for complex fallback patterns
📖 VFS Quick Start → | 🎯 Common Patterns →
Your knowledge isn't trapped anymore. Characters live beyond stories. APIs exist beyond code files. Concepts connect across domains. This is knowledge that happens to support files, not a filesystem that happens to store knowledge.
🚀 NEW: Enhanced Directory Import with Caching
Import large projects 10-100x faster with intelligent caching:
import { Brainy } from '@soulcraft/brainy'
import { ProgressTracker, formatProgress } from '@soulcraft/brainy/types'
import { detectRelationshipsWithConfidence } from '@soulcraft/brainy/neural'
const brain = new Brainy()
await brain.init()
// Progress tracking for long operations
const tracker = ProgressTracker.create(1000)
tracker.start()
for await (const progress of importer.importStream('./project', {
batchSize: 100,
generateEmbeddings: true
})) {
const p = tracker.update(progress.processed, progress.current)
console.log(formatProgress(p))
// [RUNNING] 45% (450/1000) - 23.5 items/s - 23s remaining
}
// Entity extraction with intelligent caching
const entities = await brain.neural.extractor.extract(text, {
types: ['person', 'organization', 'technology'],
confidence: 0.7,
cache: {
enabled: true,
ttl: 7 * 24 * 60 * 60 * 1000, // 7 days
invalidateOn: 'mtime' // Re-extract when file changes
}
})
// Relationship detection with confidence scores
const relationships = detectRelationshipsWithConfidence(entities, text, {
minConfidence: 0.7
})
// Create relationships with evidence tracking
await brain.relate({
from: sourceId,
to: targetId,
type: 'creates',
confidence: 0.85,
evidence: {
sourceText: 'John created the database',
method: 'pattern',
reasoning: 'Matches creation pattern; entities in same sentence'
}
})
// Monitor cache performance
const stats = brain.neural.extractor.getCacheStats()
console.log(`Cache hit rate: ${(stats.hitRate * 100).toFixed(1)}%`)
// Cache hit rate: 89.5%
🎯 Zero Configuration Philosophy
Brainy automatically configures everything:
import { Brainy } from '@soulcraft/brainy'
// 1. Pure zero-config - detects everything
const brain = new Brainy()
// 2. Custom configuration
const brain = new Brainy({
storage: { type: 'filesystem', path: './brainy-data' },
embeddings: { model: 'all-MiniLM-L6-v2' },
cache: { enabled: true, maxSize: 1000 }
})
// 3. Production configuration
const customBrain = new Brainy({
mode: 'production',
model: 'q8', // Optimized model (99% accuracy, 75% smaller)
storage: 'cloud', // or 'memory', 'disk', 'auto'
features: ['core', 'search', 'cache']
})
What's Auto-Detected:
- Storage: S3/GCS/R2 → Filesystem (priority order)
- Models: Always Q8 for optimal balance
- Features: Minimal → Default → Full based on environment
- Memory: Optimal cache sizes and batching
- Performance: Threading, chunking, indexing strategies
Production Performance
- 3ms average search - Lightning fast queries
- 24MB memory footprint - Efficient resource usage
- Worker-based embeddings - Non-blocking operations
- Automatic caching - Intelligent result caching
🎛️ Advanced Configuration (When Needed)
Most users never need this - zero-config handles everything. For advanced use cases:
// Model is always Q8 for optimal performance
const brain = new Brainy() // Uses Q8 automatically
// Storage control (auto-detected by default)
const diskBrain = new Brainy({storage: 'disk'}) // Local filesystem
const cloudBrain = new Brainy({storage: 'cloud'}) // S3/GCS/R2
// Legacy full config (still supported)
const legacyBrain = new Brainy({
storage: {type: 'filesystem', path: './data'}
})
Model Details:
- Q8: 33MB, 99% accuracy, 75% smaller than full precision
- Fast loading and optimal memory usage
- Perfect for all environments
Air-gap deployment:
npm run download-models # Download Q8 model
npm run download-models:q8 # Download Q8 model
🚀 Import Anything - Files, Data, URLs
Brainy's universal import intelligently handles any data format:
// Import CSV with auto-detection
await brain.import('customers.csv')
// ✨ Auto-detects: encoding, delimiter, types, creates entities!
// Import Excel workbooks with multi-sheet support
await brain.import('sales-data.xlsx', {
excelSheets: ['Q1', 'Q2'] // or 'all' for all sheets
})
// ✨ Processes all sheets, preserves structure, infers types!
// Import PDF documents with table extraction
await brain.import('research-paper.pdf', {
pdfExtractTables: true
})
// ✨ Extracts text, detects tables, preserves metadata!
// Import JSON/YAML data
await brain.import([
{ name: 'Alice', role: 'Engineer' },
{ name: 'Bob', role: 'Designer' }
])
// ✨ Automatically creates Person entities with relationships!
// Import from URLs (auto-fetched)
await brain.import('https://api.example.com/data.json')
// ✨ Auto-detects URL, fetches, parses, processes!
📖 Complete Import Guide → | Live Example →
📚 Core API
search() - Vector Similarity
const results = await brain.search("machine learning", {
limit: 10, // Number of results
metadata: {type: "article"}, // Filter by metadata
includeContent: true // Include full content
})
find() - Natural Language Queries
// Simple natural language
const results = await brain.find("recent important documents")
// Structured query with Triple Intelligence
const results = await brain.find({
like: "JavaScript", // Vector similarity
where: { // Metadata filters
year: {greaterThan: 2020},
important: true
},
related: {to: "React"} // Graph relationships
})
CRUD Operations
// Create entities (nouns)
const id = await brain.add(data, { nounType: nounType, ...metadata })
// Create relationships (verbs)
const verbId = await brain.relate(sourceId, targetId, "relationType", {
strength: 0.9,
bidirectional: false
})
// Read
const item = await brain.getNoun(id)
const verb = await brain.getVerb(verbId)
// Update
await brain.updateNoun(id, newData, newMetadata)
await brain.updateVerb(verbId, newMetadata)
// Delete
await brain.deleteNoun(id)
await brain.deleteVerb(verbId)
// Bulk operations
await brain.import(arrayOfData)
const exported = await brain.export({format: 'json'})
// Import from CSV, Excel, PDF files (auto-detected)
await brain.import('customers.csv') // CSV with encoding detection
await brain.import('sales-report.xlsx') // Excel with multi-sheet support
await brain.import('research.pdf') // PDF with table extraction
🌐 Distributed System (NEW!)
Zero-Config Distributed Setup
// Single node (default)
const brain = new Brainy({
storage: {
type: 's3',
s3Storage: {
bucketName: 'my-data',
region: 'us-east-1',
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
}
}
})
// Distributed cluster - just add one flag!
const brain = new Brainy({
storage: {
type: 's3',
s3Storage: {
bucketName: 'my-data',
region: 'us-east-1',
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
}
},
distributed: true // That's it! Everything else is automatic
})
How It Works
- Storage-Based Discovery: Nodes find each other via S3/GCS (no Consul/etcd!)
- Automatic Sharding: Data distributed by content hash
- Smart Query Planning: Queries routed to optimal shards
- Live Rebalancing: Handles node joins/leaves automatically
- Zero Downtime: Streaming shard migration
Real-World Example: Social Media Firehose
import { Brainy, NounType } from '@soulcraft/brainy'
// Ingestion nodes (optimized for writes)
const ingestionNode = new Brainy({
storage: {
type: 's3',
s3Storage: {
bucketName: 'social-data',
region: 'us-east-1',
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
}
},
distributed: true,
writeOnly: true // Optimized for high-throughput writes
})
// Process Bluesky firehose
blueskyStream.on('post', async (post) => {
await ingestionNode.add(post, {
nounType: NounType.Message,
platform: 'bluesky',
author: post.author,
timestamp: post.createdAt
})
})
// Search nodes (optimized for queries)
const searchNode = new Brainy({
storage: {
type: 's3',
s3Storage: {
bucketName: 'social-data',
region: 'us-east-1',
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
}
},
distributed: true,
readOnly: true // Optimized for fast queries
})
// Search across ALL data from ALL nodes
const trending = await searchNode.find('trending AI topics', {
where: {timestamp: {greaterThan: Date.now() - 3600000}},
limit: 100
})
Benefits Over Traditional Systems
| Feature | Traditional (Pinecone, Weaviate) | Brainy Distributed |
|---|---|---|
| Setup | Complex (k8s, operators) | One flag: distributed: true |
| Coordination | External (etcd, Consul) | Built-in (via storage) |
| Minimum Nodes | 3-5 for HA | 1 (scale as needed) |
| Sharding | Random | Domain-aware |
| Query Planning | Basic | Triple Intelligence |
| Cost | High (always-on clusters) | Low (scale to zero) |
🎯 Use Cases
Knowledge Management with Relationships
// Store documentation with rich relationships
const apiGuide = await brain.add("REST API Guide", {
nounType: NounType.Document,
title: "API Guide",
category: "documentation",
version: "2.0"
})
const author = await brain.add("Jane Developer", {
nounType: NounType.Person,
role: "tech-lead"
})
const project = await brain.add("E-commerce Platform", {
nounType: NounType.Project,
status: "active"
})
// Create knowledge graph
await brain.relate(author, apiGuide, "authored", {
date: "2024-03-15"
})
await brain.relate(apiGuide, project, "documents", {
coverage: "complete"
})
// Query the knowledge graph naturally
const docs = await brain.find("documentation authored by tech leads for active projects")
Semantic Search
// Find similar content
const similar = await brain.search(existingContent, {
limit: 5,
threshold: 0.8
})
AI Memory Layer with Context
// Store messages with relationships
const userId = await brain.add("User 123", {
nounType: NounType.User,
tier: "premium"
})
const messageId = await brain.add(userMessage, {
nounType: NounType.Message,
timestamp: Date.now(),
session: "abc"
})
const topicId = await brain.add("Product Support", {
nounType: NounType.Topic,
category: "support"
})
// Link message elements
await brain.relate(userId, messageId, "sent")
await brain.relate(messageId, topicId, "about")
// Retrieve context with relationships
const context = await brain.find({
where: {type: "message"},
connected: {from: userId, type: "sent"},
like: "previous product issues"
})
💾 Storage Options
Brainy supports multiple storage backends:
// FileSystem (Node.js - recommended for development)
const brain = new Brainy({
storage: {
type: 'filesystem',
path: './data'
}
})
// Browser Storage (OPFS) - Works with frameworks
const brain = new Brainy({
storage: {type: 'opfs'} // Framework handles browser polyfills
})
// S3 Compatible (Production)
const brain = new Brainy({
storage: {
type: 's3',
bucket: 'my-bucket',
region: 'us-east-1'
}
})
🛠️ CLI
Brainy includes a powerful CLI for testing and management:
# Install globally
npm install -g brainy
# Add data
brainy add "JavaScript is awesome" --metadata '{"type":"opinion"}'
# Search
brainy search "programming"
# Natural language find
brainy find "awesome programming languages"
# Interactive mode
brainy chat
# Export data
brainy export --format json > backup.json
🧠 Neural API - Advanced AI Features
Brainy includes a powerful Neural API for advanced semantic analysis:
Clustering & Analysis
// Access via brain.neural
const neural = brain.neural
// Automatic semantic clustering
const clusters = await neural.clusters()
// Returns groups of semantically similar items
// Cluster with options
const clusters = await neural.clusters({
algorithm: 'kmeans', // or 'hierarchical', 'sample'
maxClusters: 5, // Maximum number of clusters
threshold: 0.8 // Similarity threshold
})
// Calculate similarity between any items
const similarity = await neural.similar('item1', 'item2')
// Returns 0-1 score
// Find nearest neighbors
const neighbors = await neural.neighbors('item-id', 10)
// Build semantic hierarchy
const hierarchy = await neural.hierarchy('item-id')
// Detect outliers
const outliers = await neural.outliers(0.3)
// Generate visualization data for D3/Cytoscape
const vizData = await neural.visualize({
maxNodes: 100,
dimensions: 3,
algorithm: 'force'
})
Real-World Examples
// Group customer feedback into themes
const feedbackClusters = await neural.clusters()
for (const cluster of feedbackClusters) {
console.log(`Theme: ${cluster.label}`)
console.log(`Items: ${cluster.members.length}`)
}
// Find related documents
const docId = await brain.add("Machine learning guide", { nounType: NounType.Document })
const similar = await neural.neighbors(docId, 5)
// Returns 5 most similar documents
// Detect anomalies in data
const anomalies = await neural.outliers(0.2)
console.log(`Found ${anomalies.length} outliers`)
🔌 Augmentations
Extend Brainy with powerful augmentations:
# List available augmentations
brainy augment list
# Install an augmentation
brainy augment install explorer
# Connect to Brain Cloud
brainy cloud setup
🏢 Enterprise Features - Included for Everyone
Brainy includes enterprise-grade capabilities at no extra cost. No premium tiers, no paywalls.
- Scales to 10M+ items with consistent 3ms search latency
- Distributed architecture with sharding and replication
- Read/write separation for horizontal scaling
- Connection pooling and request deduplication
- Built-in monitoring with metrics and health checks
- Production ready with circuit breakers and backpressure
📖 More enterprise features coming soon - Stay tuned!
📊 Benchmarks
| Operation | Performance | Memory |
|---|---|---|
| Initialize | 450ms | 24MB |
| Add Item | 12ms | +0.1MB |
| Vector Search (1k items) | 3ms | - |
| Metadata Filter (10k items) | 0.8ms | - |
| Natural Language Query | 15ms | - |
| Bulk Import (1000 items) | 2.3s | +8MB |
| Production Scale (10M items) | 5.8ms | 12GB |
🔄 Migration from Previous Versions
Key changes in the latest version:
- Search methods consolidated into
search()andfind() - Result format now includes full objects with metadata
- Enhanced natural language capabilities
- Distributed architecture support
🤝 Contributing
We welcome contributions! See CONTRIBUTING.md for guidelines.
🧠 The Knowledge Operating System Explained
How We Achieved The Impossible
Triple Intelligence™ makes us the world's first to unify three database paradigms:
- Vector databases (Pinecone, Weaviate) - semantic similarity
- Graph databases (Neo4j, ArangoDB) - relationships
- Document databases (MongoDB, Elasticsearch) - metadata filtering
One API to rule them all. Others make you choose. We unified them.
The Math of Infinite Expressiveness
31 Nouns × 40 Verbs × ∞ Metadata × Triple Intelligence = Universal Protocol
- 1,240 base combinations from standardized types
- ∞ domain specificity via unlimited metadata
- ∞ relationship depth via graph traversal
- = Model ANYTHING: From quantum physics to social networks
Why This Changes Everything
Like HTTP for the web, Brainy for knowledge:
- All augmentations compose perfectly - same noun-verb language
- All AI models share knowledge - GPT, Claude, Llama all understand
- All tools integrate seamlessly - no translation layers
- All data flows freely - perfect portability
The Vision: One protocol. All knowledge. Every tool. Any AI.
Proven across industries: Healthcare, Finance, Manufacturing, Education, Legal, Retail, Government, and beyond.
→ See the Mathematical Proof & Full Taxonomy
📖 Documentation
Framework Integration
- Framework Integration Guide - Complete framework setup guide
- Next.js Integration - React and Next.js examples
- Vue.js Integration - Vue and Nuxt examples
Virtual Filesystem (Semantic VFS) 🧠📁
- VFS Core Documentation - Complete filesystem architecture and API
- Semantic VFS Guide - Multi-dimensional file access (6 semantic dimensions)
- Neural Extraction API - AI-powered concept and entity extraction
- Examples & Scenarios - Real-world use cases and code
- VFS API Guide - Complete API reference
Core Documentation
- Getting Started Guide
- API Reference
- Architecture Overview
- Data Storage Architecture - Deep dive into storage, indexing, and sharding
- Natural Language Guide
- Triple Intelligence
- Noun-Verb Taxonomy
🏢 Enterprise & Cloud
Brain Cloud - Managed Brainy with team sync, persistent memory, and enterprise connectors.
# Get started with free trial
brainy cloud setup
Visit soulcraft.com for more information.
📄 License
MIT © Brainy Contributors
Built with ❤️ by the Brainy community
Zero-Configuration AI Database with Triple Intelligence™