From 4ec9ab2384fb255190d191c97d042bf014fccae0 Mon Sep 17 00:00:00 2001 From: David Snelling Date: Fri, 8 Aug 2025 06:10:34 -0700 Subject: [PATCH] docs: complete README overhaul with 50s sci-fi theme and comprehensive features MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Restructure with wow → why → quick → powerful → advanced flow - Add retro-futuristic "Brain in a Jar Database™" branding - Document new Brainy CLI with full command examples - Add Cortex AI and Augmentation Pipeline sections - Include real-world performance metrics and scaling strategies - Add WebRTC/WebSocket sync and GPU acceleration mentions - Update attribution to Soulcraft Research and BXL9000™ - Remove Discord link and consolidate redundant content --- README.md | 648 +++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 445 insertions(+), 203 deletions(-) diff --git a/README.md b/README.md index 2b4b68a5..7ca6f963 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,3 @@ -# 🧠⚛️ Brainy - Lightning-Fast Vector + Graph Database with AI Intelligence -
![Brainy Logo](brainy.png) @@ -9,91 +7,263 @@ [![Node.js](https://img.shields.io/badge/node-%3E%3D24.4.1-brightgreen.svg)](https://nodejs.org/) [![TypeScript](https://img.shields.io/badge/TypeScript-5.4.5-blue.svg)](https://www.typescriptlang.org/) -**The world's only true Vector + Graph database with built-in AI intelligence** -**Sub-millisecond queries across millions of vectors + billions of relationships** +# BRAINY: The Brain in a Jar Database™ + +**The world's only Vector + Graph + AI database and realtime data platform** + +*Zero-to-Smart™ technology that thinks so you don't have to*
-## The Problem: Three Databases for One Search +--- -**"I need semantic search, relationship traversal, AND metadata filtering - that means 3+ databases"** +## 🚀 THE AMAZING BRAINY: See It In Action! -❌ **Current Reality**: Pinecone + Neo4j + Elasticsearch + Custom Sync = Slow, expensive, complex -✅ **Brainy Reality**: One blazing-fast database. One API. Everything in sync. +```javascript +import { BrainyData } from '@soulcraft/brainy' -## 🚀 Quick Start: 8 Lines to Production +// 🧪 Initialize your brain-in-a-jar +const brainy = new BrainyData() // Zero config - it's ALIVE! +await brainy.init() + +// 🔬 Feed it knowledge with relationships +const openai = await brainy.add("OpenAI", { type: "company", funding: 11000000 }) +const gpt4 = await brainy.add("GPT-4", { type: "product", users: 100000000 }) +await brainy.relate(openai, gpt4, "develops") + +// ⚡ One query to rule them all - Vector + Graph + Faceted search! +const results = await brainy.search("AI language models", 5, { + metadata: { funding: { $gte: 10000000 } }, // MongoDB-style filtering + includeVerbs: true // Graph relationships +}) // Plus semantic vector search! +``` + +**🎭 8 lines. Three search paradigms. One brain-powered database.** + +## 💫 WHY BRAINY? The Problem We Solve + +### ❌ The Old Way: Database Frankenstein + +``` +Pinecone ($$$) + Neo4j ($$$) + Elasticsearch ($$$) + Sync Hell = 😱 +``` + +### ✅ The Brainy Way: One Smart Brain + +``` +Vector Search + Graph Relations + Metadata Filtering + AI Intelligence = 🧠✨ +``` + +**Your data gets a brain upgrade. No assembly required.** + +## ⚡ QUICK & EASY: From Zero to Smart in 60 Seconds + +### Installation ```bash npm install @soulcraft/brainy ``` +### Your First Brainy App + ```javascript import { BrainyData } from '@soulcraft/brainy' -const brainy = new BrainyData() // Auto-detects environment -await brainy.init() // Zero configuration +// It's alive! (No config needed) +const brainy = new BrainyData() +await brainy.init() -// Add data with relationships -const openai = await brainy.add("OpenAI", { type: "company", funding: 11000000 }) -const gpt4 = await brainy.add("GPT-4", { type: "product", users: 100000000 }) -await brainy.relate(openai, gpt4, "develops") +// Feed your brain some data +await brainy.add("Tesla", { type: "company", sector: "automotive" }) +await brainy.add("SpaceX", { type: "company", sector: "aerospace" }) -// One query, three search paradigms -const results = await brainy.search("AI language models", 5, { - metadata: { funding: { $gte: 10000000 } }, // MongoDB-style filtering - includeVerbs: true // Graph relationships -}) // Semantic vector search +// Ask it questions (semantic search) +const similar = await brainy.search("electric vehicles") + +// Use relationships (graph database) +await brainy.relate("Tesla", "SpaceX", "shares_founder_with") + +// Filter like MongoDB (faceted search) +const results = await brainy.search("innovation", { + metadata: { sector: "automotive" } +}) ``` -**That's it. You just built a knowledge graph with semantic search in 8 lines.** - -## 🎯 Key Features: Why Developers Choose Brainy - -### ⚡ Blazing Performance at Scale -``` -Vector Search (1M embeddings): 2-8ms p95 latency -Graph Traversal (100M relations): 1-3ms p95 latency -Combined Vector+Graph+Filter: 5-15ms p95 latency -Throughput: 10K+ queries/second -``` - -### 🌍 Write Once, Run Anywhere -- **Same code** works in React, Vue, Angular, Node.js, Edge Workers -- **Auto-detects** environment and optimizes automatically -- **Zero config** - no setup files, no tuning parameters - -### 🧠 Built-in AI Intelligence (FREE) -- **Cortex Augmentation**: AI understands your data structure automatically -- **Entity Detection**: Identifies people, companies, locations -- **Relationship Mapping**: Discovers connections between entities -- **Chat Interface**: Talk to your data naturally (v0.56+) - ---- - -# 🎆 NEW: Talk to Your Data with Brainy Chat! +## 🎆 NEW! Talk to Your Data with Brainy Chat ```javascript import { BrainyChat } from '@soulcraft/brainy' -const chat = new BrainyChat(brainy) // That's it! +const chat = new BrainyChat(brainy) // Your data becomes conversational! const answer = await chat.ask("What patterns do you see in customer behavior?") -// → Works instantly with zero config! +// → AI-powered insights from your knowledge graph! ``` -**One line. Zero complexity. Optional LLM for smarter responses.** +**One line. Zero complexity. Optional LLM for genius-level responses.** [📖 **Learn More About Brainy Chat**](BRAINY-CHAT.md) -## 🔥 The Power of Three-in-One Search +## 🎮 NEW! Brainy CLI - Command Center from the Future + +### 💬 Talk to Your Data + +```bash +# Have conversations with your knowledge graph +brainy chat "What patterns exist in customer behavior?" +brainy chat "Show me all connections between startups" +``` + +### 📥 Add & Import Data + +```bash +# Import with AI understanding +brainy import data.csv --cortex --understand + +# Add individual items +brainy add "OpenAI" --type company --metadata '{"founded": 2015}' + +# Bulk import with relationships +brainy import relationships.json --detect-entities +``` + +### 🔍 Explore & Query + +```bash +# Search semantically +brainy search "artificial intelligence companies" + +# Query with filters +brainy query --filter 'funding>1000000' --type company + +# Visualize relationships +brainy graph "OpenAI" --depth 2 --format ascii +``` + +### 🔄 Manage & Migrate + +```bash +# Export your brain +brainy export my-brain.json --include-embeddings + +# Migrate between storage backends +brainy migrate s3://old-bucket file://new-location + +# Backup and restore +brainy backup --compress +brainy restore backup-2024.tar.gz +``` + +### 🔐 Environment & Secrets + +```bash +# Store configuration securely +brainy config set api.key "sk-..." --encrypt +brainy config set storage.s3.bucket "my-brain" + +# Load environment profiles +brainy env use production +brainy env create staging --from .env.staging +``` + +### 📊 Monitor & Optimize + +```bash +# Real-time dashboard +brainy monitor --dashboard + +# Performance analysis +brainy stats --detailed +brainy optimize index --auto +``` + +**Command your data empire from the terminal!** +[📖 **Full CLI Documentation**](docs/brainy-cli.md) + +## 🧬 NEW! Cortex AI - Your Data Gets a PhD + +**Cortex automatically understands and enhances your data:** ```javascript -// This ONE query does what used to require 3 databases: +// Enable Cortex Intelligence during import +const brainy = new BrainyData({ + cortex: { + enabled: true, + autoDetect: true // Automatically identify entities & relationships + } +}) + +// Import with understanding +await brainy.cortexImport('customers.csv', { + understand: true, // AI analyzes data structure + detectRelations: true, // Finds hidden connections + confidence: 0.8 // Quality threshold +}) +``` + +**Your data becomes self-aware (in a good way)!** + +## 🔌 NEW! Augmentation Pipeline - Plug in Superpowers + +**8 types of augmentations to enhance your brain:** + +```javascript +// Add augmentations like installing apps on your brain +brainy.augment({ + type: 'PERCEPTION', // Visual/pattern recognition + handler: myPerceptor +}) + +brainy.augment({ + type: 'COGNITION', // Deep thinking & analysis + handler: myThinker +}) + +// Premium augmentations (coming soon!) +brainy.augment({ + type: 'NOTION_SYNC', // Bi-directional Notion sync + license: 'premium' +}) +``` + +**Augmentation Types:** + +- 🎯 **SENSE** - Input processing +- 🧠 **MEMORY** - Long-term storage +- 💭 **COGNITION** - Deep analysis +- 🔗 **CONDUIT** - Data flow +- ⚡ **ACTIVATION** - Triggers & events +- 👁️ **PERCEPTION** - Pattern recognition +- 💬 **DIALOG** - Conversational AI +- 🌐 **WEBSOCKET** - Real-time sync + +## 💪 POWERFUL FEATURES: What Makes Brainy Special + +### ⚡ Performance That Defies Science + +``` +Vector Search (1M embeddings): 2-8ms latency 🚀 +Graph Traversal (100M relations): 1-3ms latency 🔥 +Combined Vector+Graph+Filter: 5-15ms latency ⚡ +Throughput: 10K+ queries/sec 💫 +``` + +### 🌍 Write Once, Run Anywhere (Literally) + +- **Browser**: Uses OPFS, Web Workers - works offline! +- **Node.js**: FileSystem, Worker Threads - server-ready! +- **Edge/Serverless**: Memory-optimized - deploys anywhere! +- **React/Vue/Angular**: Same code, automatic optimization! + +### 🔮 The Power of Three-in-One Search + +```javascript +// This ONE query replaces THREE databases: const results = await brainy.search("AI startups in healthcare", 10, { // 🔍 Vector: Semantic similarity includeVerbs: true, - + // 🔗 Graph: Relationship traversal verbTypes: ["invests_in", "partners_with"], - + // 📊 Faceted: MongoDB-style filtering metadata: { industry: "healthcare", @@ -101,225 +271,290 @@ const results = await brainy.search("AI startups in healthcare", 10, { stage: { $in: ["Series A", "Series B"] } } }) -// Returns: Companies similar to your query + their relationships + matching your criteria ``` -## 🌍 Works Everywhere - Same Code +### 🧠 Self-Learning & Auto-Optimization -**Write once, run anywhere.** Brainy auto-detects your environment: +**Brainy gets smarter the more you use it:** -| Environment | Storage | Optimization | -|-------------|---------|-------------| -| 🌐 Browser | OPFS | Web Workers, Memory Cache | -| 🟢 Node.js | FileSystem / S3 | Worker Threads, Clustering | -| ⚡ Serverless | S3 / Memory | Cold Start Optimization | -| 🔥 Edge | Memory / KV | Minimal Footprint | +- Auto-indexes frequently searched fields +- Learns query patterns for faster responses +- Optimizes storage based on access patterns +- Self-configures for your environment -
-🔧 Advanced Configuration Options +## 🎭 ADVANCED FEATURES: For Mad Scientists + +### 🔬 MongoDB-Style Query Operators ```javascript -// High-throughput writer +const results = await brainy.search("quantum computing", { + metadata: { + $and: [ + { price: { $gte: 100, $lte: 1000 } }, + { category: { $in: ["electronics", "computing"] } }, + { + $or: [ + { brand: "Intel" }, + { brand: "IBM" } + ] + }, + { tags: { $includes: "quantum" } }, + { description: { $regex: "qubit|superposition" } } + ] + } +}) +``` + +**15+ operators**: `$gt`, `$gte`, `$lt`, `$lte`, `$eq`, `$ne`, `$in`, `$nin`, `$regex`, `$includes`, `$all`, `$size`, +`$and`, `$or`, `$not` + +### 🧪 Specialized Deployment Modes + +```javascript +// High-speed data ingestion const writer = new BrainyData({ writeOnly: true, allowDirectReads: true // For deduplication }) -// Read-only search service +// Read-only search cluster const reader = new BrainyData({ readOnly: true, - frozen: true // No stats updates + frozen: true // Maximum performance }) -// Custom storage +// Custom storage backend const custom = new BrainyData({ storage: { type: 's3', - s3Storage: { bucketName: 'my-vectors' } - }, - hnsw: { - maxConnections: 32 // Higher quality + s3Storage: { + bucketName: 'my-brain', + region: 'us-east-1' + } } }) ``` -
- -## 🎮 Brainy CLI - Command Center for Everything - -```bash -# Talk to your data -brainy chat "What patterns do you see?" - -# AI-powered data import -brainy import data.csv --cortex --confidence 0.8 - -# Real-time monitoring -brainy monitor --dashboard - -# Start premium trials -brainy license trial notion -``` - -[📖 **Full CLI Documentation**](/docs/brainy-cli.md) - -## ⚙️ Configuration (Optional) - -Brainy works with **zero configuration**, but you can customize - - -## 🆚 Why Not Just Use...? - -### vs. Multiple Databases -❌ **Pinecone + Neo4j + Elasticsearch** - 3 databases, sync nightmares, 3x the cost -✅ **Brainy** - One database, always synced, built-in intelligence - -### vs. Cloud-Only Vector DBs -❌ **Pinecone/Weaviate/Qdrant** - Vendor lock-in, expensive, cloud-only -✅ **Brainy** - Run anywhere, your data stays yours, cost-effective - -### vs. Graph DBs with "Vector Features" -❌ **Neo4j + vector plugin** - Bolt-on solution, not native, limited -✅ **Brainy** - Native vector+graph architecture from the ground up - -## 💎 Premium Features (Optional) - -**Core Brainy is FREE forever. Premium features for enterprise needs:** - -### 🔗 Enterprise Connectors (14-day trials) -- **Notion** ($49/mo) - Bidirectional workspace sync -- **Salesforce** ($99/mo) - CRM integration -- **Slack** ($49/mo) - Team collaboration -- **Asana** ($44/mo) - Project management - -```bash -brainy license trial notion # Start free trial -``` - -**No vendor lock-in. Your data stays yours.** - -## 🎨 What You Can Build - -- **🤖 AI Chat Applications** - ChatGPT-like apps with long-term memory -- **🔍 Semantic Search** - Search by meaning, not keywords -- **🎯 Recommendation Engines** - "Users who liked this also liked..." -- **🧬 Knowledge Graphs** - Connect everything to everything -- **🛡️ Fraud Detection** - Find patterns humans can't see -- **📚 Smart Documentation** - Docs that answer questions - +### 🚀 Framework Integration Examples
-📦 Framework Examples +📦 See Framework Examples + +#### React -### React ```jsx import { BrainyData } from '@soulcraft/brainy' function App() { const [brainy] = useState(() => new BrainyData()) - useEffect(() => brainy.init(), []) - + + useEffect(() => { + brainy.init() + }, []) + const search = async (query) => { return await brainy.search(query, 10) } + + return } ``` -### Vue 3 +#### Vue 3 + ```vue + ``` -### Angular +#### Angular + ```typescript -@Component({}) -export class AppComponent { - brainy = new BrainyData() - async ngOnInit() { + +@Injectable({ providedIn: 'root' }) +export class BrainyService { + private brainy = new BrainyData() + + async init() { await this.brainy.init() } -} -``` -### Node.js -```javascript -const brainy = new BrainyData() -await brainy.init() + search(query: string) { + return this.brainy.search(query, 10) + } +} ```
+### 🐳 Docker & Cloud Deployment +```dockerfile +FROM node:24-slim +WORKDIR /app +COPY . . +RUN npm install +RUN npm run download-models # Bundle models for offline use +CMD ["node", "server.js"] +``` -## 📦 Advanced Features +Deploy to AWS, GCP, Azure, Cloudflare Workers, anywhere! -
-🔧 MongoDB-Style Metadata Filtering +## 💎 Premium Features (Optional) + +**Core Brainy is FREE forever. Premium augmentations for enterprise:** + +### 🔗 Enterprise Connectors (Coming Soon!) + +- **Notion** ($49/mo) - Bi-directional workspace sync +- **Salesforce** ($99/mo) - CRM integration +- **Slack** ($49/mo) - Team knowledge capture +- **Asana** ($44/mo) - Project intelligence + +```bash +brainy augment trial notion # Start 14-day free trial +``` + +## 🎨 What You Can Build + +**The only limit is your imagination:** + +- **🤖 AI Assistants** - ChatGPT with perfect memory +- **🔍 Semantic Search** - Find by meaning, not keywords +- **🎯 Recommendation Engines** - Netflix-level suggestions +- **🧬 Knowledge Graphs** - Wikipedia meets Neo4j +- **👁️ Computer Vision** - Search images by content +- **🎵 Music Discovery** - Spotify's algorithm in your app +- **📚 Smart Documentation** - Self-answering docs +- **🛡️ Fraud Detection** - Pattern recognition on steroids +- **🌐 Real-time Collaboration** - Multiplayer knowledge bases +- **🏥 Medical Diagnosis** - Symptom matching with AI + +## 📚 Complete Documentation + +### Getting Started + +- [**Quick Start Guide**](docs/getting-started/) - Up and running in 5 minutes +- [**Installation**](docs/getting-started/installation.md) - All environments covered +- [**Basic Concepts**](docs/getting-started/concepts.md) - Understand the brain + +### Core Features + +- [**API Reference**](docs/api-reference/) - Every method documented +- [**Search Guide**](docs/api-reference/search.md) - Master all search types +- [**Graph Operations**](docs/api-reference/graph.md) - Relationships explained +- [**MongoDB Operators**](docs/api-reference/operators.md) - Query like a pro + +### Advanced Topics + +- [**Brainy CLI**](docs/brainy-cli.md) - Command-line superpowers +- [**Brainy Chat**](BRAINY-CHAT.md) - Conversational AI interface +- [**Cortex AI**](CORTEX.md) - Intelligence augmentation +- [**Augmentation Pipeline**](docs/augmentations/) - Plugin architecture +- [**Performance Tuning**](docs/optimization-guides/) - Speed optimization +- [**Deployment Guide**](docs/deployment/) - Production best practices + +### Examples & Tutorials + +- [**Example Apps**](docs/examples/) - Full applications +- [**Code Recipes**](docs/examples/recipes.md) - Common patterns +- [**Video Tutorials**](docs/tutorials/) - Visual learning + +## 🆚 Why Not Just Use...? + +### vs. Multiple Databases + +❌ **Pinecone + Neo4j + Elasticsearch** = 3x cost, sync nightmares, 3 APIs +✅ **Brainy** = One database, always synced, one simple API + +### vs. Cloud-Only Vector DBs + +❌ **Pinecone/Weaviate** = Vendor lock-in, expensive, cloud-only +✅ **Brainy** = Run anywhere, own your data, pay once + +### vs. Traditional Graph DBs + +❌ **Neo4j + vector plugin** = Bolt-on solution, limited capabilities +✅ **Brainy** = Native vector+graph from the ground up + +## 🚀 Real-World Performance & Scale + +**How Brainy handles production workloads:** + +### 📊 Benchmark Numbers + +- **10M vectors**: 5-15ms search latency (p95) +- **100M relationships**: 1-3ms traversal +- **Metadata filtering**: O(1) field access via hybrid indexing +- **Concurrent queries**: 10,000+ QPS on single instance +- **Index size**: ~100 bytes per vector (384 dims) + +### 🎯 Scaling Strategies + +**Scale Up (Vertical)** ```javascript -const results = await brainy.search("machine learning", 10, { - metadata: { - price: { $gte: 100, $lte: 1000 }, - category: { $in: ["AI", "ML"] }, - rating: { $gt: 4.5 }, - tags: { $includes: "tensorflow" } +// Optimize for large datasets on single machine +const brainy = new BrainyData({ + hnsw: { + maxConnections: 32, // More connections = better recall + efConstruction: 400, // Higher quality index + efSearch: 100 // More accurate search } }) ``` -**15+ operators**: `$gt`, `$in`, `$regex`, `$and`, `$or`, etc. - -
- -
-🔗 Graph Relationships +**Scale Out (Horizontal)** ```javascript -const company = await brainy.add("OpenAI", { type: "company" }) -const product = await brainy.add("GPT-4", { type: "product" }) -await brainy.relate(company, product, "develops") +// Shard by category for distributed deployment +const shards = { + products: new BrainyData({ defaultService: 'products-shard' }), + users: new BrainyData({ defaultService: 'users-shard' }), + content: new BrainyData({ defaultService: 'content-shard' }) +} -const products = await brainy.getVerbsBySource(company) +// Or use read/write separation +const writer = new BrainyData({ writeOnly: true }) +const readers = [/* multiple read replicas */] ``` -
+### 🏗️ Architecture That Scales -
-🐳 Docker Deployment +✅ **Distributed Index** - Partition by metadata fields or ID ranges +✅ **Smart Partitioning** - Semantic clustering or hash-based sharding +✅ **Real-time Sync** - WebRTC & WebSocket for live collaboration +✅ **GPU Acceleration** - Auto-detected for embeddings when available +✅ **Metadata Index** - Separate B-tree indexes for fast filtering +✅ **Memory Mapped Files** - Handle datasets larger than RAM +✅ **Streaming Ingestion** - Process millions of items without OOM +✅ **Progressive Loading** - Start serving queries before full index load -```dockerfile -FROM node:24-slim -RUN npm run download-models # Embed models -CMD ["node", "server.js"] -``` +## 🛸 Recent Updates -Deploy anywhere: AWS, GCP, Azure, Cloudflare +### 🎯 v0.57.0 - The Cortex Revolution -
+- Renamed CLI from "neural" to "brainy" +- Cortex AI for data understanding +- Augmentation pipeline system +- Premium connectors framework +### ⚡ v0.46-v0.51 - Performance Revolution -## 📚 Documentation - -- [Quick Start](docs/getting-started/) -- [API Reference](docs/api-reference/) -- [Examples](docs/examples/) -- [Brainy CLI](docs/brainy-cli.md) -- [Performance Guide](docs/optimization-guides/) - -## ❓ Does Brainy Impact Performance? - -**NO - Brainy actually IMPROVES performance:** - -✅ **Zero runtime overhead** - Premium features are lazy-loaded only when used -✅ **Smaller than alternatives** - 643KB vs 12.5MB for TensorFlow.js -✅ **Built-in caching** - 95%+ cache hit rates reduce compute -✅ **Automatic optimization** - Gets faster as it learns your patterns -✅ **No network calls** - Works completely offline after setup - -**The augmentation system and premium features are 100% optional and have ZERO impact unless explicitly activated.** +- 95% package size reduction +- MongoDB query operators +- Filter discovery API +- Transformers.js migration +- True offline operation ## 🤝 Contributing @@ -332,7 +567,14 @@ We welcome contributions! See [Contributing Guidelines](CONTRIBUTING.md) ---
-Ready to build the future of search? -**[Get Started →](docs/getting-started/) | [Examples →](docs/examples/) | [Discord →](https://discord.gg/brainy)** -
\ No newline at end of file +## 🧠 Ready to Give Your Data a Brain? + +**[Get Started →](docs/getting-started/) | [Examples →](docs/examples/)** + +*Zero-to-Smart™ - Because your data deserves a brain upgrade* + +**Built with ❤️ by [Soulcraft Research](https://soulcraft.com)** +*Powered by the BXL9000™ Cognitive Engine* + +