From cfe1fd9e225f6f99f07207800c4dbcc6333cafcc Mon Sep 17 00:00:00 2001 From: David Snelling Date: Thu, 19 Jun 2025 15:00:08 -0700 Subject: [PATCH] **feat: refine README and enhance embedding workflows** MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ### Changes: - Updated README.md: - Refined the project description to highlight its lightweight and powerful nature. - Enhanced feature list, replacing outdated entries with modern updates (e.g., TensorFlow integration replacing LLM creation). - Included detailed installation and usage instructions for TensorFlow.js support and embedding workflows. - Added "Usage Options" section to define different import methods and supported environments. - Expanded "Backup and Restore" details, showcasing CLI and scripting examples. - Introduced optimized HNSW indexing for large datasets. - Replaced examples with a unified and simplified demonstration approach. - Embedded advanced build and environment-specific configurations for better clarity. - Removed outdated content related to language model generation, including LLM-related commands and examples. - Added README.demo.md with full instructions for running the interactive demo. ### Purpose: This update modernizes the documentation and focuses on Brainy’s current capabilities, simplifying onboarding for new users while removing deprecated features. It also streamlines project understanding, enabling developers to better utilize its features in diverse environments. --- README.demo.md | 56 + README.md | 367 +++-- examples/demo.html | 3836 ++++++++++++++++++++++++++++++++++++++------ 3 files changed, 3630 insertions(+), 629 deletions(-) create mode 100644 README.demo.md diff --git a/README.demo.md b/README.demo.md new file mode 100644 index 00000000..68f3dcc5 --- /dev/null +++ b/README.demo.md @@ -0,0 +1,56 @@ +# Running the Brainy Demo + +The Brainy interactive demo showcases the library's features in a web browser. Follow these steps to run it: + +## Prerequisites + +- Make sure you have Node.js installed (version 23.11.0 or higher) +- Ensure the project is built (`npm run build:all`) + +## Running the Demo + +### Option 1: Using the npm script (recommended) + +Run the following command from the project root: + +```bash +npm run demo +``` + +This will start an HTTP server and automatically open the demo in your default browser. + +### Option 2: Manual setup + +1. Start an HTTP server in the project root: + +```bash +npx http-server +``` + +2. Open your browser and navigate to: + http://localhost:8080/examples/demo.html + +## Troubleshooting + +If you see the error "Could not load Brainy library. Please ensure the project is built and served over HTTP", check the following: + +1. Make sure you've built the project with `npm run build:all` +2. Ensure you're accessing the demo through HTTP (not by opening the file directly) +3. Check your browser's console for additional error messages + +If issues persist, try clearing your browser cache or using a private/incognito window. + +## Build Process + +The Brainy library uses a two-step build process: + +1. `npm run build` - Compiles TypeScript files to JavaScript (used for Node.js environments) +2. `npm run build:browser` - Creates a browser-compatible bundle using Rollup + +You can run both steps together with: + +```bash +npm run build:all +``` + +The browser bundle is created from `examples/browser_compatible_exports.ts`, which exports only browser-compatible parts of the library. This ensures that the demo works correctly in browser environments while the full library still works in Node.js environments. diff --git a/README.md b/README.md index d530b533..de3cbc37 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md) [![Cartographer](https://img.shields.io/badge/Cartographer-Official%20Standard-brightgreen)](https://github.com/sodal-project/cartographer) -**A fun lightweight graph & vector data platform for AI applications across any environment** +**A lightweight and powerful graph & vector data platform for AI applications across any environment** @@ -33,8 +33,7 @@ and connections. - **Extensible Augmentations** - Customize and extend functionality with pluggable components (LEGO blocks for your data!) - **Built-in Conduits** - Sync and scale across instances with WebSocket and WebRTC (your data's teleportation system!) -- **LLM Creation & Training** - Build, train, and deploy language models from your graph data (your own personal AI - factory!) +- **TensorFlow Integration** - Use TensorFlow.js for high-quality embeddings (included as a required dependency) - **Adaptive Intelligence** - Automatically optimizes for your environment and usage patterns - **Cross-Platform** - Works everywhere you do: browsers, Node.js, and server environments - **Persistent Storage** - Data persists across sessions and scales to any size (no memory loss here, even for @@ -49,14 +48,22 @@ and connections. taste) - **Knowledge Graphs** - Build connected data structures with relationships (your data's family tree) - **AI Applications** - Store and retrieve embeddings for machine learning models (brain food for your AI) -- **Custom Language Models** - Create, train, and deploy LLMs from your graph data (your personal GPT factory!) +- **AI-Enhanced Applications** - Build applications that leverage vector embeddings for intelligent data processing - **Data Organization Tools** - Automatically categorize and connect related information (like having a librarian in your code) - **Adaptive Experiences** - Create applications that learn and evolve with your users (digital chameleons!) ## πŸ”§ Installation -Due to a dependency conflict between TensorFlow.js packages, use the `--legacy-peer-deps` flag when installing: +```bash +npm install @soulcraft/brainy +``` + +### TensorFlow.js Support + +TensorFlow-based embeddings are now included as required dependencies. All necessary TensorFlow.js packages are automatically installed when you install Brainy. + +Note: If you encounter dependency conflicts with TensorFlow.js packages, you may need to use the `--legacy-peer-deps` flag: ```bash npm install @soulcraft/brainy --legacy-peer-deps @@ -64,8 +71,13 @@ npm install @soulcraft/brainy --legacy-peer-deps ## 🏁 Quick Start +Brainy now uses a unified build that automatically adapts to your environment (Node.js, browser, or serverless), so you can use the same code everywhere! + ```typescript -import {BrainyData, NounType, VerbType} from '@soulcraft/brainy' +import {BrainyData, NounType, VerbType, environment} from '@soulcraft/brainy' + +// Check which environment we're running in (optional) +console.log(`Running in ${environment.isBrowser ? 'browser' : environment.isNode ? 'Node.js' : 'serverless'} environment`) // Create and initialize the database const db = new BrainyData() @@ -94,6 +106,42 @@ await db.addVerb(catId, dogId, { }) ``` +### Usage Options + +Brainy's unified build works in all environments, but you have several import options: + +```typescript +// Standard import - automatically adapts to any environment +import {BrainyData, NounType, VerbType, environment} from '@soulcraft/brainy' + +// Minified version for production +import {BrainyData, NounType, VerbType} from '@soulcraft/brainy/min' + +// Use the same API in any environment +const db = new BrainyData() +await db.init() +// ... +``` + +#### Using a script tag in HTML + +```html + +``` + +Modern bundlers like Webpack, Rollup, and Vite will automatically use the unified build which adapts to any environment. + ## 🧩 How It Works (The Magic Behind the Curtain) Brainy combines four key technologies to create its adaptive intelligence: @@ -134,7 +182,7 @@ making future operations even faster and more relevant! 2. **Embedding Generation** 🧠 - Text is transformed into numerical vectors using embedding models (language β†’ math magic) - - Choose between TensorFlow Universal Sentence Encoder (high quality) or Simple Embedding (faster) + - Uses TensorFlow Universal Sentence Encoder for high-quality text embeddings - Custom embedding functions can be plugged in for specialized domains (bring your own secret sauce) 3. **Vector Indexing** πŸ” @@ -178,10 +226,9 @@ Brainy uses a powerful augmentation system to extend functionality. Augmentation 3. **COGNITION** 🧠 - Enables advanced reasoning, inference, and logical operations - Analyzes relationships between entities - - Creates and trains language models from graph data - Examples: - Inferring new connections between existing data - - Building custom LLMs from your nouns and verbs + - Deriving insights from graph relationships 4. **CONDUIT** πŸ”Œ - Establishes high-bandwidth channels for structured data exchange @@ -240,6 +287,36 @@ Brainy's pipeline is designed to handle streaming data efficiently: - Configurable execution modes (SEQUENTIAL, PARALLEL, THREADED) - Example: `executeTypedPipeline(augmentations, method, args, { mode: ExecutionMode.THREADED })` +### πŸ—οΈ Build System + +Brainy uses a modern build system that optimizes for both Node.js and browser environments: + +1. **ES Modules** πŸ“¦ + - Built as ES modules for maximum compatibility + - Works in modern browsers and Node.js environments + - Separate optimized builds for browser and Node.js + +2. **Environment-Specific Builds** πŸ”§ + - **Node.js Build**: Optimized for server environments with full functionality + - **Browser Build**: Optimized for browser environments with reduced bundle size + - Conditional exports in package.json for automatic environment detection + +3. **Environment Detection** πŸ” + - Automatically detects whether it's running in a browser or Node.js + - Loads appropriate dependencies and functionality based on the environment + - Provides consistent API across all environments + +4. **TypeScript** πŸ“ + - Written in TypeScript for type safety and better developer experience + - Generates type definitions for TypeScript users + - Compiled to ES2020 for modern JavaScript environments + +5. **Build Scripts** πŸ› οΈ + - `npm run build`: Builds the Node.js version + - `npm run build:browser`: Builds the browser-optimized version + - `npm run build:all`: Builds both versions + - `npm run demo`: Builds all versions and starts a demo server + ### πŸƒβ€β™€οΈ Running the Pipeline The pipeline runs automatically when you: @@ -414,41 +491,6 @@ npm run cli generate-random-graph --noun-count 20 --verb-count 40 - `-t, --data-type ` - Type of data to process (default: 'text') - `-v, --verbose` - Show detailed output -#### LLM Commands: - -- `llm create` - Create a new LLM model from Brainy data - - `-n, --name ` - Name of the model - - `-d, --description ` - Description of the model - - `-t, --type ` - Type of model (simple, transformer, custom) - - `-v, --vocab-size ` - Vocabulary size - - `-e, --embedding-dim ` - Embedding dimension - - `-h, --hidden-dim ` - Hidden dimension - - `-l, --layers ` - Number of layers - - `--heads ` - Number of attention heads (for transformer models) -- `llm train ` - Train an LLM model on Brainy data - - `-s, --max-samples ` - Maximum number of training samples - - `-v, --validation-split ` - Validation split ratio - - `-e, --epochs ` - Number of training epochs - - `-b, --batch-size ` - Batch size - - `-p, --patience ` - Early stopping patience -- `llm test ` - Test an LLM model on Brainy data - - `-s, --test-size ` - Number of test samples - - `-g, --generate-samples` - Generate sample predictions - - `-c, --sample-count ` - Number of samples to generate -- `llm export ` - Export an LLM model for deployment - - `-f, --format ` - Export format (tfjs, json) - - `-o, --output ` - Output path - - `-m, --include-metadata` - Include metadata - - `-v, --include-vocab` - Include vocabulary -- `llm deploy ` - Deploy an LLM model to the specified target - - `-t, --target ` - Deployment target (browser, node, cloud) - - `-p, --provider ` - Cloud provider (aws, gcp, azure) - - `-e, --endpoint ` - Endpoint URL for cloud deployment - - `-r, --region ` - Region for cloud deployment -- `llm generate ` - Generate text using an LLM model - - `-t, --temperature ` - Temperature for sampling - - `-k, --top-k ` - Number of top tokens to consider - - `-l, --max-length ` - Maximum length of generated text ## πŸ”Œ API Reference @@ -463,6 +505,12 @@ await db.clear() // Get database status const status = await db.status() + +// Backup all data from the database +const backupData = await db.backup() + +// Restore data into the database +const restoreResult = await db.restore(backupData, { clearExisting: true }) ``` ### Working with Nouns (Entities) @@ -496,6 +544,7 @@ const thingNouns = await db.searchByNounTypes([NounType.Thing], numResults) ### Working with Verbs (Relationships) + ```typescript // Add a relationship between nouns await db.addVerb(sourceId, targetId, { @@ -522,81 +571,17 @@ const verb = await db.getVerb(verbId) await db.deleteVerb(verbId) ``` -### Working with LLM Models - -```typescript -import {createLLMAugmentations} from '@soulcraft/brainy' - -// Create LLM augmentations -const {cognition, activation} = await createLLMAugmentations() - -// Create a new LLM model -const createResult = await cognition.createModel({ - name: 'my-model', - description: 'A simple LLM model trained on Brainy data', - modelType: 'simple', - vocabSize: 5000, - embeddingDim: 64, - hiddenDim: 128, - numLayers: 1 -}) - -const modelId = createResult.data.modelId - -// Train the model -const trainResult = await cognition.trainModel(modelId, { - maxSamples: 100, - validationSplit: 0.2, - earlyStoppingPatience: 2 -}) - -// Test the model -const testResult = await cognition.testModel(modelId, { - testSize: 20, - generateSamples: true, - sampleCount: 3 -}) - -// Generate text with the model -const generateResult = await cognition.generateText(modelId, 'What is a', { - temperature: 0.7, - topK: 5 -}) - -// Export the model -const exportResult = await cognition.exportModel(modelId, { - format: 'json', - includeMetadata: true, - includeVocab: true -}) - -// Deploy the model -const deployResult = await cognition.deployModel(modelId, { - target: 'browser' -}) - -// Using the augmentation pipeline -const pipelineResult = await augmentationPipeline.executeCognitionPipeline( - 'createModel', - [{ - name: 'pipeline-model', - modelType: 'transformer', - numHeads: 2, - numLayers: 1 - }] -) -``` ## βš™οΈ Advanced Configuration -### Custom Embedding +### Embedding ```typescript -import {BrainyData, createSimpleEmbeddingFunction} from '@soulcraft/brainy' +import {BrainyData, createTensorFlowEmbeddingFunction} from '@soulcraft/brainy' -// Use a custom embedding function (faster but less accurate) +// Use the TensorFlow Universal Sentence Encoder embedding function const db = new BrainyData({ - embeddingFunction: createSimpleEmbeddingFunction() + embeddingFunction: createTensorFlowEmbeddingFunction() }) await db.init() @@ -641,6 +626,65 @@ const db = new BrainyData({ }) ``` +### Optimized HNSW for Large Datasets + +Brainy includes an optimized HNSW index implementation designed specifically for large datasets that may not fit entirely in memory. This implementation uses a hybrid approach combining: + +1. **Product Quantization** - Reduces vector dimensionality while preserving similarity relationships +2. **Disk-Based Storage** - Offloads vectors to disk when memory usage exceeds a threshold +3. **Memory-Efficient Indexing** - Optimizes memory usage for large-scale vector collections + +```typescript +import {BrainyData} from '@soulcraft/brainy' + +// Configure with optimized HNSW index for large datasets +const db = new BrainyData({ + // Use the optimized HNSW index instead of the standard one + hnswOptimized: { + // Standard HNSW parameters + M: 16, // Max connections per noun + efConstruction: 200, // Construction candidate list size + efSearch: 50, // Search candidate list size + + // Memory threshold in bytes - when exceeded, will use disk-based approach + memoryThreshold: 1024 * 1024 * 1024, // 1GB default threshold + + // Product quantization settings for dimensionality reduction + productQuantization: { + enabled: true, // Enable product quantization + numSubvectors: 16, // Number of subvectors to split the vector into + numCentroids: 256 // Number of centroids per subvector + }, + + // Whether to use disk-based storage for the index + useDiskBasedIndex: true // Enable disk-based storage + }, + + // Storage configuration (required for disk-based index) + storage: { + // Choose appropriate storage for your environment + requestPersistentStorage: true + } +}) + +// The optimized index automatically adapts based on dataset size: +// 1. For small datasets: Uses standard in-memory approach +// 2. For medium datasets: Applies product quantization to reduce memory usage +// 3. For large datasets: Combines product quantization with disk-based storage + +// Check status to see memory usage and optimization details +const status = await db.status() +console.log(status.details.index) +// Example output: +// { +// indexSize: 10000, +// optimized: true, +// memoryUsage: 536870912, // Memory usage in bytes +// productQuantization: true, +// diskBasedIndex: true +// } +``` + ## πŸ§ͺ Distance Functions - `cosineDistance` (default) @@ -648,10 +692,84 @@ const db = new BrainyData({ - `manhattanDistance` - `dotProductDistance` -## πŸ”‹ Embedding Options +## πŸ“€πŸ“₯ Backup and Restore -- Default: TensorFlow Universal Sentence Encoder (high quality) -- Alternative: Simple character-based embedding (faster) +Brainy provides powerful backup and restore capabilities that allow you to: +- Back up your data +- Transfer data between Brainy instances +- Restore existing data into Brainy for vectorization and indexing +- Backup data for analysis or visualization in other tools + +### Backing Up Data + +```typescript +// Backup all data from the database +const backupData = await db.backup() + +// The backup data includes: +// - All nouns (entities) with their vectors and metadata +// - All verbs (relationships) between nouns +// - Noun types and verb types +// - HNSW index data for fast similarity search +// - Version information + +// Save the backup data to a file (Node.js environment) +import fs from 'fs' +fs.writeFileSync('brainy-backup.json', JSON.stringify(backupData, null, 2)) +``` + +### Restoring Data + +Brainy's restore functionality is flexible and can handle: +1. Complete backups with vectors and index data +2. Sparse data without vectors (vectors will be created during restore) +3. Data without HNSW index (index will be reconstructed if needed) + +```typescript +// Restore data with all options +const restoreResult = await db.restore(backupData, { + clearExisting: true // Whether to clear existing data before restore +}) + +// Restore sparse data (without vectors) +// Vectors will be automatically created using the embedding function +const sparseData = { + nouns: [ + { + id: '123', + // No vector field - will be created during restore + metadata: { + noun: 'Thing', + text: 'This text will be used to generate a vector' + } + } + ], + verbs: [], + version: '1.0.0' +} + +const sparseRestoreResult = await db.restore(sparseData) +``` + +### CLI Backup/Restore + +```bash +# Backup data to a file +brainy backup --output brainy-backup.json + +# Restore data from a file +brainy restore --input brainy-backup.json --clear-existing + +# Restore sparse data (without vectors) +brainy restore --input sparse-data.json +``` + +## πŸ”‹ Embedding + +Brainy uses the following embedding approach: + +- TensorFlow Universal Sentence Encoder (high-quality text embeddings) +- Custom embedding functions can be plugged in for specialized domains ## 🧰 Extensions @@ -660,7 +778,7 @@ Brainy includes an augmentation system for extending functionality: - **Memory Augmentations**: Different storage backends - **Sense Augmentations**: Process raw data - **Cognition Augmentations**: Reasoning and inference -- **Dialog Augmentations**: Natural language processing +- **Dialog Augmentations**: Text processing and interaction - **Perception Augmentations**: Data interpretation and visualization - **Activation Augmentations**: Trigger actions @@ -701,16 +819,15 @@ To get started with cloud deployment, see the [Cloud Wrapper README](cloud-wrapp - **[Cartographer](https://github.com/sodal-project/cartographer)** - A companion project that provides standardized interfaces for interacting with Brainy -## πŸ“š Examples +## πŸ“š Demo -The repository includes several examples: +The repository includes a comprehensive demo that showcases Brainy's main features: -- Web demo: `examples/demo.html` -- Basic usage: `examples/basicUsage.js` -- Custom storage: `examples/customStorage.js` -- Memory augmentations: `examples/memoryAugmentationExample.js` -- Conduit augmentations: `examples/conduitAugmentationExample.js` -- Browser-server search: `examples/browser-server-search/` - Search a server-hosted Brainy instance from a browser +- `examples/demo.html` - A single demo page with animations demonstrating Brainy's features. Run it with + `npm run demo` (see [demo instructions](README.demo.md) for details): + - How Brainy runs in different environments (browser, Node.js, server, cloud) + - How the noun-verb data model works + - How HNSW search works ### Syncing Brainy Instances @@ -889,7 +1006,7 @@ const id = await db.addToBoth('Deep learning is a subset of machine learning', { await db.shutDown() ``` -For a complete example with HTML interface, see the [browser-server-search example](examples/browser-server-search/). +For a complete demonstration of Brainy's features, see the [demo page](examples/demo.html). ## πŸ“‹ Requirements diff --git a/examples/demo.html b/examples/demo.html index ed259ce3..c8142c9d 100644 --- a/examples/demo.html +++ b/examples/demo.html @@ -1,592 +1,3420 @@ - - - Brainy - Vector and Graph Database Demo - + .demo-section { + background-color: var(--card-bg); + border-radius: var(--border-radius); + box-shadow: var(--box-shadow); + padding: 30px; + margin-bottom: 30px; + } + + .demo-controls { + display: flex; + flex-wrap: wrap; + gap: 10px; + margin: 20px 0; + } + + button { + background-color: var(--primary-color); + color: white; + border: none; + padding: 10px 20px; + border-radius: var(--border-radius); + cursor: pointer; + transition: var(--transition); + } + + button:hover { + background-color: #388E3C; + transform: translateY(-2px); + } + + button.secondary { + background-color: var(--secondary-color); + } + + button.secondary:hover { + background-color: #1976D2; + } + + button.accent { + background-color: var(--accent-color); + } + + button.accent:hover { + background-color: #F57C00; + } + + button.danger { + background-color: var(--danger-color); + } + + button.danger:hover { + background-color: #D32F2F; + } + + input, select, textarea { + padding: 10px; + border: 1px solid #ddd; + border-radius: var(--border-radius); + flex-grow: 1; + } + + textarea { + min-height: 100px; + font-family: monospace; + } + + .visualization { + height: 500px; + background-color: var(--light-bg); + border-radius: var(--border-radius); + margin: 20px 0; + position: relative; + overflow: hidden; + } + + /* Cartographer styles */ + .zoom-controls { + position: absolute; + bottom: 1.5rem; + right: 1rem; + z-index: 10; + display: flex; + flex-direction: column; + gap: 0.5rem; + } + + .zoom-fit-button { + width: 2.5rem; + height: 2.5rem; + border-radius: 50%; + background-color: var(--md3-primary); + color: var(--md3-on-primary); + border: none; + cursor: pointer; + display: flex; + align-items: center; + justify-content: center; + box-shadow: 0 0.125rem 0.25rem var(--md3-shadow); + transition: background-color 0.2s, transform 0.2s; + } + + .zoom-fit-button:hover { + background-color: #7B68B5; + transform: scale(1.05); + } + + .zoom-fit-button:active { + background-color: #553B93; + transform: scale(0.95); + } + + .graph-container { + width: 100%; + height: 100%; + background-color: #FDF8FF; + display: flex; + align-items: center; + justify-content: center; + flex: 1; + overflow: hidden; + } + + .graph-container svg { + display: block; + width: 100%; + height: 100%; + font-family: 'Roboto', 'Inter', sans-serif; + } + + .graph-container .edges line { + transition: stroke-opacity 0.2s ease-in-out; + } + + .graph-container .node-group { + transition: transform 0.15s cubic-bezier(0.4, 0, 0.2, 1), filter 0.3s ease; + transform-origin: center; + } + + .graph-container .node-group:hover { + filter: drop-shadow(0 0.5rem 1rem rgba(0, 0, 0, 0.25)); + animation: jiggle 0.5s ease; + } + + .graph-container .node-group path { + transition: fill-opacity 0.15s ease-in-out; + } + + .graph-container .overlay-container { + pointer-events: none; + } + + .graph-container .overlay-container .node-info-popup { + pointer-events: auto; + } + + @keyframes jiggle { + 0% { + transform: rotate(0deg); + } + 25% { + transform: rotate(0.5deg) scale(1.01); + } + 50% { + transform: rotate(-0.5deg) scale(1.01); + } + 75% { + transform: rotate(0.5deg) scale(1.01); + } + 100% { + transform: rotate(0deg); + } + } + + .results { + background-color: var(--light-bg); + border-radius: var(--border-radius); + padding: 15px; + max-height: 300px; + overflow-y: auto; + font-family: monospace; + white-space: pre-wrap; + } + + .tabs { + display: flex; + margin-bottom: 20px; + flex-wrap: wrap; + } + + .tab { + padding: 10px 20px; + background-color: var(--light-bg); + cursor: pointer; + border-radius: var(--border-radius) var(--border-radius) 0 0; + margin-right: 5px; + margin-bottom: 5px; + } + + .tab.active { + background-color: var(--primary-color); + color: white; + } + + .tab-content { + display: none; + } + + .tab-content.active { + display: block; + } + + .section-content, .subsection-content { + display: block; + margin-bottom: 30px; + } + + .quickstart-section { + background: linear-gradient(135deg, #193c41, #3f6e65); + color: white; + padding: 30px; + border-radius: var(--border-radius); + margin-bottom: 30px; + box-shadow: var(--box-shadow); + } + + .quickstart-section h2 { + color: white; + margin-top: 0; + } + + .code-tabs { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(400px, 1fr)); + gap: 20px; + margin-top: 20px; + } + + .code-tab { + background-color: rgba(255, 255, 255, 0.1); + border-radius: var(--border-radius); + padding: 20px; + } + + .code-tab h3 { + margin-top: 0; + color: white; + } + + .code-tab p { + color: rgba(255, 255, 255, 0.9); + } + + .section-divider { + height: 1px; + background-color: var(--light-bg); + margin: 20px 0; + } + + .code-block { + background-color: var(--dark-bg); + color: white; + padding: 15px; + border-radius: var(--border-radius); + overflow-x: auto; + margin: 20px 0; + font-family: monospace; + } + + .feature-grid { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(300px, 1fr)); + gap: 20px; + margin: 30px 0; + } + + .feature-card { + background-color: var(--card-bg); + border-radius: var(--border-radius); + box-shadow: var(--box-shadow); + padding: 20px; + transition: var(--transition); + } + + .feature-card:hover { + transform: translateY(-5px); + box-shadow: 0 10px 20px rgba(0, 0, 0, 0.1); + } + + .feature-card h3 { + margin-top: 0; + color: var(--primary-color); + } + + .tooltip { + position: absolute; + padding: 8px; + background: rgba(0, 0, 0, 0.8); + color: #fff; + border-radius: 4px; + pointer-events: none; + z-index: 10; + font-size: 12px; + } + + .cli-terminal { + background-color: var(--dark-bg); + color: #fff; + padding: 15px; + border-radius: var(--border-radius); + font-family: monospace; + margin: 20px 0; + position: relative; + } + + .cli-prompt { + color: var(--accent-color); + } + + .cli-input { + background: transparent; + border: none; + color: #fff; + font-family: monospace; + width: 80%; + outline: none; + } + + .cli-output { + margin-top: 10px; + white-space: pre-wrap; + max-height: 300px; + overflow-y: auto; + } + + .cli-history { + margin-bottom: 10px; + } + + .cli-command { + margin-bottom: 5px; + } + + .form-group { + margin-bottom: 15px; + } + + .form-group label { + display: block; + margin-bottom: 5px; + font-weight: bold; + } + + .features-overview { + background: linear-gradient(135deg, #193c41, #3f6e65); + color: white; + padding: 30px; + border-radius: var(--border-radius); + margin-bottom: 30px; + box-shadow: var(--box-shadow); + } + + .features-overview h2 { + color: white; + margin-top: 0; + } + + .features-list { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); + gap: 15px; + margin-top: 20px; + } + + .feature-item { + background-color: var(--card-bg); /* Changed from transparent white to card background */ + padding: 15px; + border-radius: var(--border-radius); + /* backdrop-filter: blur(10px); Removed as it's less relevant with solid background */ + box-shadow: 0 2px 4px rgba(0, 0, 0, 0.05); /* Added a light shadow to feature items */ + } + + .feature-item h4 { + margin: 0 0 10px 0; + color: var(--secondary-color); /* Changed from white to secondary color for better contrast */ + } + + .feature-item p { + margin: 0; + font-size: 0.9em; + opacity: 1; /* Changed from 0.9 to 1 for better readability */ + color: var(--text-color); /* Ensure paragraph text is also dark */ + } + + footer { + text-align: center; + padding: 20px; + margin-top: 40px; + background-color: var(--card-bg); + border-radius: var(--border-radius); + box-shadow: var(--box-shadow); + } + + @media (max-width: 768px) { + .container { + padding: 10px; + } + + .demo-controls { + flex-direction: column; + } + + .feature-grid { + grid-template-columns: 1fr; + } + + .features-list { + grid-template-columns: 1fr; + } + } + -
- Brainy Logo -

Brainy - Vector and Graph Database Demo

-
-

This demo shows how to use Brainy as both a vector database (with embeddings and similarity search) and a graph - database (with GraphNoun nodes and GraphVerb relationships).

-
-
-

1. Initialize Database

- -
-
+ -
-

2. Configure Pipeline

- -
-
+
+ Brainy Logo +

Brainy Interactive Demo

+

A lightweight but powerful graph & vector data platform for AI applications across any environment

+
-
-

3. Add Sample Data

- -
-
+ +
+

πŸš€ Key Features

+

Explore all of Brainy's powerful capabilities in this interactive demo:

-
-

4. Vector Search

-
- - - +
+
+

πŸƒβ€β™‚οΈ Run Everywhere

+

Run Brainy in a browser, container, serverless cloud service or the terminal!

+
+
+

πŸ” Vector Search

+

Find semantically similar content using embeddings (like having ESP for your data!)

+
+
+

🌐 Graph Database

+

Connect data with meaningful relationships (your data's social network)

+
+
+

🌊 Streaming Pipeline

+

Process data in real-time as it flows through the system (like a data waterslide!)

+
+
+

🧩 Extensible Augmentations

+

Customize and extend functionality with pluggable components (LEGO blocks for your data!)

+
+
+

πŸ”Œ Built-in Conduits

+

Sync and scale across instances with WebSocket and WebRTC (your data's teleportation system!)

+
+
+

πŸ€– TensorFlow Integration

+

Use TensorFlow.js for high-quality embeddings (included as a required dependency)

+
+
+

🧠 Adaptive Intelligence

+

Automatically optimizes for your environment and usage patterns

+
+
+

🌍 Cross-Platform

+

Works everywhere you do: browsers, Node.js, and server environments

+
+
+

πŸ’Ύ Persistent Storage

+

Data persists across sessions and scales to any size (no memory loss here!)

+
+
+

πŸ”§ TypeScript Support

+

Fully typed API with generics (for those who like their code tidy)

+
+
+

πŸ“± CLI Tools

+

Powerful command-line interface for data management (command line wizardry)

+
+
+

πŸ“Š Data Visualization

+

Interactive graph and vector space visualizations

+
+
+

☁️ Cloud Ready

+

Easy deployment to Google, AWS, Azure, and other cloud platforms

+
+
+
+ + +
+

🏁 Quick Start

+ +
+
+

Node.js Environment

+ 
+import {BrainyData, NounType, VerbType} from '@soulcraft/brainy'
+
+// Create and initialize the database
+const db = new BrainyData()
+await db.init()
+
+// Add data (automatically converted to vectors)
+const catId = await db.add("Cats are independent pets", {
+    noun: NounType.Thing,
+    category: 'animal'
+})
+
+const dogId = await db.add("Dogs are loyal companions", {
+    noun: NounType.Thing,
+    category: 'animal'
+})
+
+// Search for similar items
+const results = await db.searchText("feline pets", 2)
+console.log(results)
+// Returns items similar to "feline pets" with similarity scores
+
+// Add a relationship between items
+await db.addVerb(catId, dogId, {
+    verb: VerbType.RelatedTo,
+    description: 'Both are common household pets'
+})
+
+
+ +
+

Browser Environment

+

You can import the browser-optimized version in two ways:

+ +

Option 1: Using the browser-specific import

+ 
+// Import the browser-optimized version
+import {BrainyData, NounType, VerbType} from '@soulcraft/brainy/browser'
+
+// Use the same API as in Node.js
+const db = new BrainyData()
+await db.init()
+// ...
+
+ +

Option 2: Using a script tag in HTML

+ 
+<script type="module">
+  // Import Brainy library directly as a module
+  import {BrainyData, NounType, VerbType} from '../node_modules/@soulcraft/brainy/dist/brainy.js'
+
+  // Use the same API as in Node.js
+  const db = new BrainyData()
+  await db.init()
+  // ...
+</script>
+
+

Modern bundlers like Webpack, Rollup, and Vite will automatically select the browser-optimized version when + targeting browser environments.

+
+
+
+ +
+

Interactive Brainy Explorer

+

This interactive demo allows you to experiment with all of Brainy's features. Initialize the database, add + data, create relationships, and perform searches - all with real-time visualizations.

+ + + + +
+

Database Operations

+

Initialize the database, add sample data, or clear all data.

+ +
+ + + + + + +
+ +
+ Database not initialized. Click "Initialize Database" to start. +
+ +
+
+
- 
// Search results will appear here
-
- -
-

5. Text Search

-
- - - +
+
+ Database visualization will appear here after initialization +
- 
// Text search results will appear here
+
-
-

6. Graph Operations

-
- - + +
+

Data Model Management

+

Create entities (nouns), define relationships (verbs), and manage your data schema.

+ +
+ + +
+
+ +
- 
// Graph operation results will appear here
+ +
+ + +
+ +
+ + +
+ +
+ + +
+ +
+ + +
+ +
+ + +
+
+ + +
+
+ + +
+ +
+ + + +
+ +
+ + +
+ +
+ + +
+ +
+ + +
+ +
+ + +
+
+ +
+ Entity operations will appear here... +
-
-

Console Output

- 
// Output will appear here
+ +
+

Vector Search & Semantic Queries

+

Perform semantic searches using vector embeddings and similarity matching.

+ +
+ + +
+ +
+ + +
+ +
+ + + +
+ +
+ Search results will appear here... +
+ +
+
+ Search results visualization +
+
+ + + +
+

Data Visualization

+

Interactive visualizations of your graph data and vector spaces.

+ +
+ + + + +
+ +
+
+ Select a visualization type above +
+
+ +
+ + + +
+
+ + +
+

CLI Command Interface

+

Simulate CLI commands and explore Brainy's command-line interface.

+ +
+
+
+ brainy$ + +
+
+
+ +
+ + + + + +
+ +
+ Available Commands:
+ brainy init - Initialize a new database
+ brainy add <data> - Add data to the database
+ brainy search <query> - Search for entities
+ brainy relate <from> <to> - Create relationships
+ brainy export - Export database
+ brainy status - Show database status
+ brainy help - Show all commands +
+
+ + +
+

Feature Demonstrations

+

Explore specific features and capabilities of Brainy with guided demonstrations.

+ +
+
+

πŸ” HNSW Vector Search

+

Demonstrate fast approximate nearest neighbor search with hierarchical navigable small world + graphs.

+ +
+ +
+

🌐 Graph Traversal

+

Show graph traversal algorithms and relationship queries.

+ +
+ +
+

🧠 Embedding Pipeline

+

Demonstrate automatic text-to-vector conversion and similarity computation.

+ +
+ +
+

πŸ“Š Real-time Updates

+

Show live data synchronization and event streaming capabilities.

+ +
+ +
+

πŸ’Ύ Storage Backends

+

Compare different storage options: memory, filesystem, and cloud.

+ +
+ +
+

πŸ”§ TypeScript Integration

+

Show type-safe operations and schema validation.

+ +
+
+ +
+ Select a feature above to see a detailed demonstration... +
+
+
+ +
- -