From 89413ebec2bf4454caf4df4008b2728fb0b0a63f Mon Sep 17 00:00:00 2001 From: David Snelling Date: Tue, 5 Aug 2025 15:34:39 -0700 Subject: [PATCH] docs(readme): improve user engagement flow and add advanced features - Move "Get Started in 30 Seconds" section higher for immediate engagement - Add "The Magic: Vector + Graph Database" section highlighting unique value - Update code examples with real API methods (addNoun, addVerb, getVerbsBySource) - Add "Advanced Features" teaser section - Improve overall organization for better user excitement and adoption --- README.md | 654 ++++++++++++++++++++++++----- demo/brainy-angular-demo/README.md | 59 +++ 2 files changed, 615 insertions(+), 98 deletions(-) create mode 100644 demo/brainy-angular-demo/README.md diff --git a/README.md b/README.md index 3c940f05..1e4f341b 100644 --- a/README.md +++ b/README.md @@ -31,8 +31,7 @@ easy-to-use package. - **๐Ÿง  Zero-to-Smartโ„ข** - No config files, no tuning parameters, no DevOps headaches. Brainy auto-detects your environment and optimizes itself -- **๐ŸŒ True Write-Once, Run-Anywhere** - Same code runs in React, Angular, Vue, Node.js, Deno, Bun, serverless, edge - workers, and even vanilla HTML +- **๐ŸŒ True Write-Once, Run-Anywhere** - Same code runs in Angular, React, Vue, Node.js, Deno, Bun, serverless, edge workers, and web workers with automatic environment detection - **โšก Scary Fast** - Handles millions of vectors with sub-millisecond search. Built-in GPU acceleration when available - **๐ŸŽฏ Self-Learning** - Like having a database that goes to the gym. Gets faster and smarter the more you use it - **๐Ÿ”ฎ AI-First Design** - Built for the age of embeddings, RAG, and semantic search. Your LLMs will thank you @@ -49,6 +48,408 @@ easy-to-use package. - **๐Ÿ“Š Performance Monitoring** - Built-in hit rate and memory usage tracking with adaptive optimization - **๐ŸŽฏ Zero Breaking Changes** - All existing code works unchanged, just faster and smarter +## ๐Ÿ“ฆ Get Started in 30 Seconds + +```bash +npm install @soulcraft/brainy +``` + +```javascript +import { BrainyData } from '@soulcraft/brainy' + +const brainy = new BrainyData() +await brainy.init() // Auto-detects your environment + +// Add some data +await brainy.add("The quick brown fox jumps over the lazy dog") +await brainy.add("A fast fox leaps over a sleeping dog") +await brainy.add("Cats are independent and mysterious animals") + +// Vector search finds similar content +const results = await brainy.search("speedy animals jumping", 2) +console.log(results) // Finds the fox sentences! +``` + +**๐ŸŽฏ That's it!** You just built semantic search in 4 lines. Works in Angular, React, Vue, Node.js, browsers, serverless - everywhere. + +## ๐Ÿš€ The Magic: Vector + Graph Database + +**Most databases do one thing.** Brainy does both vector similarity AND graph relationships: + +```javascript +// Add entities with relationships +const companyId = await brainy.addNoun("OpenAI creates powerful AI models", "company") +const productId = await brainy.addNoun("GPT-4 is a large language model", "product") + +// Connect them with relationships +await brainy.addVerb(companyId, productId, undefined, { type: "develops" }) + +// Now you can do BOTH: +const similar = await brainy.search("AI language models") // Vector similarity +const products = await brainy.getVerbsByType("develops") // Graph traversal +``` + +**Why this matters:** Find content by meaning AND follow relationships. It's like having PostgreSQL and Pinecone working together seamlessly. + +### ๐Ÿ” Want More Power? +- **Advanced graph traversal** - Complex relationship queries and multi-hop searches +- **Distributed clustering** - Scale across multiple instances with automatic coordination +- **Real-time syncing** - WebSocket and WebRTC for live data updates +- **Custom augmentations** - Extend Brainy with your own functionality + +*[See full API documentation below](#-installation) for advanced features* + +## ๐ŸŽจ Build Amazing Things + +**๐Ÿค– AI Chat Applications** - Build ChatGPT-like apps with long-term memory and context awareness +**๐Ÿ” Semantic Search Engines** - Search by meaning, not keywords. Find "that thing that's like a cat but bigger" โ†’ +returns "tiger" +**๐ŸŽฏ Recommendation Engines** - "Users who liked this also liked..." but actually good +**๐Ÿงฌ Knowledge Graphs** - Connect everything to everything. Wikipedia meets Neo4j meets magic +**๐Ÿ‘๏ธ Computer Vision Apps** - Store and search image embeddings. "Find all photos with dogs wearing hats" +**๐ŸŽต Music Discovery** - Find songs that "feel" similar. Spotify's Discover Weekly in your app +**๐Ÿ“š Smart Documentation** - Docs that answer questions. "How do I deploy to production?" โ†’ relevant guides +**๐Ÿ›ก๏ธ Fraud Detection** - Find patterns humans can't see. Anomaly detection on steroids +**๐ŸŒ Real-Time Collaboration** - Sync vector data across devices. Figma for AI data +**๐Ÿฅ Medical Diagnosis Tools** - Match symptoms to conditions using embedding similarity + +## ๐Ÿš€ Write-Once, Run-Anywhere Quick Start + +Brainy uses the same code across all environments with automatic detection. **Framework-optimized** for the best developer experience. Choose your environment: + +### ๐Ÿ…ฐ๏ธ Angular (Latest) + +```bash +npm install @soulcraft/brainy +``` + +```typescript +import { Component, signal, OnInit } from '@angular/core' +import { BrainyData } from '@soulcraft/brainy' + +@Component({ + selector: 'app-search', + template: ` +
+ + +
+ @for (result of results(); track result.id) { +
+ {{result.metadata?.category}}: {{result.metadata?.originalData}} + Similarity: {{result.score | number:'1.2-2'}} +
+ } +
+
+ ` +}) +export class SearchComponent implements OnInit { + private brainy: BrainyData | null = null + results = signal([]) + query = '' + + async ngOnInit() { + // Auto-detects environment and uses OPFS storage in browsers + this.brainy = new BrainyData({ + defaultService: 'my-app' + }) + await this.brainy.init() + + // Add sample data + await this.brainy.add("Cats are amazing pets", { category: "animals" }) + await this.brainy.add("Dogs love to play fetch", { category: "animals" }) + await this.brainy.add("Pizza is delicious food", { category: "food" }) + } + + async search(query: string) { + if (!query.trim() || !this.brainy) { + this.results.set([]) + return + } + + const searchResults = await this.brainy.search(query, 5) + this.results.set(searchResults) + } +} +``` + +### โš›๏ธ React + +```bash +npm install @soulcraft/brainy +``` + +```jsx +import { BrainyData } from '@soulcraft/brainy' +import { useEffect, useState } from 'react' + +function SemanticSearch() { + const [brainy, setBrainy] = useState(null) + const [results, setResults] = useState([]) + const [query, setQuery] = useState('') + const [loading, setLoading] = useState(true) + + useEffect(() => { + async function initBrainy() { + // Auto-detects environment and uses OPFS storage in browsers + const db = new BrainyData({ + defaultService: 'my-app' + }) + await db.init() + + // Add sample data + await db.add("Cats are amazing pets", { category: "animals" }) + await db.add("Dogs love to play fetch", { category: "animals" }) + await db.add("Pizza is delicious food", { category: "food" }) + + setBrainy(db) + setLoading(false) + } + + initBrainy() + }, []) + + const search = async (searchQuery) => { + if (!searchQuery.trim() || !brainy) return setResults([]) + + const searchResults = await brainy.search(searchQuery, 5) + setResults(searchResults) + } + + if (loading) return
Initializing Brainy...
+ + return ( +
+ { + setQuery(e.target.value) + search(e.target.value) + }} + placeholder="Search by meaning (try 'pets' or 'food')..." + className="search-input" + /> + +
+ {results.map((result, i) => ( +
+ {result.metadata?.category}: {result.metadata?.originalData} + Similarity: {result.score.toFixed(2)} +
+ ))} +
+
+ ) +} + +export default SemanticSearch +``` + +### ๐ŸŒŸ Vue 3 + +```bash +npm install @soulcraft/brainy +``` + +```vue + + + + + +``` + +### ๐ŸŸข Node.js Server + +```bash +npm install @soulcraft/brainy +``` + +```javascript +import { BrainyData } from '@soulcraft/brainy' + +// Auto-detects Node.js โ†’ FileSystem (local) or S3 (production), Worker threads +const brainy = new BrainyData({ + defaultService: 'my-app', + // Optional: Production S3 storage + storage: { + s3Storage: { + bucketName: process.env.S3_BUCKET, + region: process.env.AWS_REGION, + accessKeyId: process.env.AWS_ACCESS_KEY_ID, + secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY + } + } +}) +await brainy.init() + +// Same API everywhere +await brainy.add("Cats are amazing pets", { category: "animals" }) +const results = await brainy.search("pets", 5) +console.log('Search results:', results) +``` + +### โšก Serverless (Vercel/Netlify) + +```javascript +import { BrainyData } from '@soulcraft/brainy' + +export default async function handler(req, res) { + // Auto-detects serverless โ†’ S3/R2 storage for persistence, or Memory for temp + const brainy = new BrainyData({ + defaultService: 'my-app', + // Optional: Explicit S3-compatible storage + storage: { + r2Storage: { + bucketName: process.env.R2_BUCKET, + accessKeyId: process.env.R2_ACCESS_KEY_ID, + secretAccessKey: process.env.R2_SECRET_ACCESS_KEY, + accountId: process.env.R2_ACCOUNT_ID + } + } + }) + await brainy.init() + + // Same API everywhere + const results = await brainy.search(req.query.q, 5) + res.json({ results }) +} +``` + +### ๐Ÿ”ฅ Cloudflare Workers + +```javascript +import { BrainyData } from '@soulcraft/brainy' + +export default { + async fetch(request) { + // Auto-detects edge โ†’ Minimal footprint, KV storage + const brainy = new BrainyData({ + defaultService: 'edge-app' + }) + await brainy.init() + + // Same API everywhere + const url = new URL(request.url) + const results = await brainy.search(url.searchParams.get('q'), 5) + return Response.json({ results }) + } +} +``` + +### ๐Ÿฆ• Deno + +```typescript +import { BrainyData } from 'https://esm.sh/@soulcraft/brainy' + +// Auto-detects Deno โ†’ Native compatibility, FileSystem storage +const brainy = new BrainyData({ + defaultService: 'deno-app' +}) +await brainy.init() + +// Same API everywhere +await brainy.add("Deno is awesome", { category: "tech" }) +const results = await brainy.search("technology", 5) +console.log(results) +``` + +**That's it! Same code, everywhere. Zero-to-Smartโ„ข** + +Brainy automatically detects and optimizes for: +- ๐ŸŒ **Browser frameworks** โ†’ OPFS storage, Web Workers, memory optimization +- ๐ŸŸข **Node.js servers** โ†’ FileSystem or S3/R2 storage, Worker threads, cluster support +- โšก **Serverless functions** โ†’ S3/R2 or Memory storage, cold start optimization +- ๐Ÿ”ฅ **Edge workers** โ†’ Memory or KV storage, minimal footprint +- ๐Ÿงต **Web/Worker threads** โ†’ Shared storage, thread-safe operations +- ๐Ÿฆ• **Deno/Bun runtimes** โ†’ FileSystem or S3-compatible storage, native performance + + +### ๐Ÿณ NEW: Zero-Config Docker Deployment + +**Deploy to any cloud with embedded models - no runtime downloads needed!** + +```dockerfile +# One line extracts models automatically during build +RUN npm run extract-models + +# Deploy anywhere: Google Cloud, AWS, Azure, Cloudflare, etc. +``` + +- **โšก 7x Faster Cold Starts** - Models embedded in container, no downloads +- **๐ŸŒ Universal Cloud Support** - Same Dockerfile works everywhere +- **๐Ÿ”’ Offline Ready** - No external dependencies at runtime +- **๐Ÿ“ฆ Zero Configuration** - Automatic model detection and loading + +See [Docker Deployment Guide](./docs/docker-deployment.md) for complete examples. + ```javascript // Zero configuration - everything optimized automatically! const brainy = new BrainyData() // Auto-detects environment & optimizes @@ -69,56 +470,6 @@ const stats = brainy.getCacheStats() console.log(`Auto-tuned cache hit rate: ${(stats.search.hitRate * 100).toFixed(1)}%`) ``` -## ๐Ÿš€ Quick Start (30 seconds!) - -### Node.js TLDR - -```bash -# Install -npm install brainy - -# Use it -``` - -```javascript -import { createAutoBrainy, NounType, VerbType } from 'brainy' - -const brainy = createAutoBrainy() - -// Add data with Nouns (entities) -const catId = await brainy.add("Siamese cats are elegant and vocal", { - noun: NounType.Thing, - breed: "Siamese", - category: "animal" -}) - -const ownerId = await brainy.add("John loves his pets", { - noun: NounType.Person, - name: "John Smith" -}) - -// Connect with Verbs (relationships) -await brainy.addVerb(ownerId, catId, { - verb: VerbType.Owns, - since: "2020-01-01" -}) - -// Search by meaning -const results = await brainy.searchText("feline companions", 5) - -// Search JSON documents by specific fields -const docs = await brainy.searchDocuments("Siamese", { - fields: ['breed', 'category'], // Search these fields - weights: { breed: 2.0 }, // Prioritize breed matches - limit: 10 -}) - -// Find relationships -const johnsPets = await brainy.getVerbsBySource(ownerId, VerbType.Owns) -``` - -That's it! No config, no setup, Zero-to-Smartโ„ข - ### ๐ŸŒ Distributed Mode Example (NEW!) ```javascript @@ -161,19 +512,6 @@ console.log(`Instance ${health.instanceId}: ${health.status}`) - **Streaming Pipeline** - Process data in real-time as it flows through - **Model Control Protocol** - Let AI models access your data -### Smart Optimizations - -- **๐Ÿค– Intelligent Auto-Configuration** - Detects environment, usage patterns, and optimizes everything automatically -- **โšก Runtime Performance Adaptation** - Continuously monitors and self-tunes based on real usage -- **๐ŸŒ Distributed Mode Detection** - Automatically enables real-time updates for shared storage scenarios -- **๐Ÿ“Š Workload-Aware Optimization** - Adapts cache size and TTL based on read/write patterns -- **๐Ÿง  Adaptive Learning** - Gets smarter with usage, learns from your data access patterns -- **#๏ธโƒฃ Intelligent Partitioning** - Hash-based partitioning for perfect load distribution -- **๐ŸŽฏ Role-Based Optimization** - Readers maximize cache, writers optimize throughput -- **๐Ÿท๏ธ Domain-Aware Indexing** - Automatic categorization improves search relevance -- **๐Ÿ—‚๏ธ Multi-Level Caching** - Hot/warm/cold caching with predictive prefetching -- **๐Ÿ’พ Memory Optimization** - 75% reduction with compression for large datasets - ### Developer Experience - **TypeScript Support** - Fully typed API with generics @@ -188,6 +526,71 @@ console.log(`Instance ${health.instanceId}: ${health.status}`) npm install @soulcraft/brainy ``` +### โœจ Write-Once, Run-Anywhere Architecture + +**Same code, every environment.** Brainy auto-detects and optimizes for your runtime: + +```javascript +// This exact code works in Angular, React, Vue, Node.js, Deno, Bun, +// serverless functions, edge workers, and web workers +import { BrainyData } from '@soulcraft/brainy' + +const brainy = new BrainyData() +await brainy.init() // Auto-detects environment and chooses optimal storage + +// Vector + Graph: Add entities (nouns) with relationships (verbs) +const companyId = await brainy.addNoun("OpenAI creates powerful AI models", "company", { + founded: "2015", industry: "AI" +}) +const productId = await brainy.addNoun("GPT-4 is a large language model", "product", { + type: "LLM", parameters: "1.7T" +}) + +// Create relationships between entities +await brainy.addVerb(companyId, productId, undefined, { type: "develops" }) + +// Vector search finds semantically similar content +const similar = await brainy.search("AI language models", 5) + +// Graph operations: explore relationships +const relationships = await brainy.getVerbsBySource(companyId) +const allProducts = await brainy.getVerbsByType("develops") +``` + +### ๐Ÿ” Advanced Graph Operations + +```javascript +// Vector search with graph filtering +const results = await brainy.search("AI models", 10, { + searchVerbs: true, // Search relationships directly + verbTypes: ["develops"], // Filter by relationship types + searchConnectedNouns: true, // Find entities connected by relationships + verbDirection: "outgoing" // Direction: outgoing, incoming, or both +}) + +// Graph traversal methods +const outgoing = await brainy.getVerbsBySource(entityId) // What this entity relates to +const incoming = await brainy.getVerbsByTarget(entityId) // What relates to this entity +const byType = await brainy.getVerbsByType("develops") // All relationships of this type + +// Combined vector + graph search +const connected = await brainy.searchNounsByVerbs("machine learning", 5, { + verbTypes: ["develops", "uses"], + direction: "both" +}) + +// Get related entities through specific relationships +const related = await brainy.getRelatedNouns(companyId, { relationType: "develops" }) +``` + +**Universal benefits:** +- โœ… **Auto-detects everything** - Environment, storage, threading, optimization +- โœ… **Framework-optimized** - Best experience with Angular, React, Vue bundlers +- โœ… **Runtime-agnostic** - Node.js, Deno, Bun, browsers, serverless, edge +- โœ… **TypeScript-first** - Full types everywhere, IntelliSense support +- โœ… **Tree-shaking ready** - Modern bundlers import only what you need +- โœ… **ES Modules architecture** - Individual modules for better optimization by modern frameworks + ### Production: Add Offline Model Reliability ```bash # For development (online model loading) @@ -218,19 +621,75 @@ const brainy = createAutoBrainy({ }) ``` -## ๐ŸŽจ Build Amazing Things +## ๐Ÿณ Docker & Cloud Deployment -**๐Ÿค– AI Chat Applications** - Build ChatGPT-like apps with long-term memory and context awareness -**๐Ÿ” Semantic Search Engines** - Search by meaning, not keywords. Find "that thing that's like a cat but bigger" โ†’ -returns "tiger" -**๐ŸŽฏ Recommendation Engines** - "Users who liked this also liked..." but actually good -**๐Ÿงฌ Knowledge Graphs** - Connect everything to everything. Wikipedia meets Neo4j meets magic -**๐Ÿ‘๏ธ Computer Vision Apps** - Store and search image embeddings. "Find all photos with dogs wearing hats" -**๐ŸŽต Music Discovery** - Find songs that "feel" similar. Spotify's Discover Weekly in your app -**๐Ÿ“š Smart Documentation** - Docs that answer questions. "How do I deploy to production?" โ†’ relevant guides -**๐Ÿ›ก๏ธ Fraud Detection** - Find patterns humans can't see. Anomaly detection on steroids -**๐ŸŒ Real-Time Collaboration** - Sync vector data across devices. Figma for AI data -**๐Ÿฅ Medical Diagnosis Tools** - Match symptoms to conditions using embedding similarity +**Deploy Brainy to any cloud provider with embedded models for maximum performance and reliability.** + +### Quick Docker Setup + +1. **Install models package:** + ```bash + npm install @soulcraft/brainy-models + ``` + +2. **Add to your Dockerfile:** + ```dockerfile + # Extract models during build (zero configuration!) + RUN npm run extract-models + + # Include models in final image + COPY --from=builder /app/models ./models + ``` + +3. **Deploy anywhere:** + ```bash + # Works on all cloud providers + gcloud run deploy --source . # Google Cloud Run + aws ecs create-service ... # AWS ECS/Fargate + az container create ... # Azure Container Instances + wrangler publish # Cloudflare Workers + ``` + +### Universal Dockerfile Template + +```dockerfile +FROM node:24-alpine AS builder +WORKDIR /app +COPY package*.json ./ +RUN npm ci +COPY . . +RUN npm run extract-models # โ† Automatic model extraction +RUN npm run build + +FROM node:24-alpine AS production +WORKDIR /app +COPY package*.json ./ +RUN npm ci --only=production --omit=optional +COPY --from=builder /app/dist ./dist +COPY --from=builder /app/models ./models # โ† Models included +CMD ["node", "dist/server.js"] +``` + +### Benefits + +- **โšก 7x Faster Cold Starts** - No model download delays +- **๐ŸŒ Universal Compatibility** - Same Dockerfile works on all clouds +- **๐Ÿ”’ Offline Ready** - No external dependencies at runtime +- **๐Ÿ“ฆ Zero Configuration** - Automatic model detection +- **๐Ÿ›ก๏ธ Enhanced Security** - No network calls for model loading + +**๐Ÿ“– Complete Guide:** See [docs/docker-deployment.md](./docs/docker-deployment.md) for detailed examples covering Google Cloud Run, AWS Lambda/ECS, Azure Container Instances, Cloudflare Workers, and more. + +### ๐Ÿ“ฆ Modern ES Modules Architecture + +Brainy now uses individual ES modules instead of large bundles, providing better optimization for modern frameworks: + +- **Better tree-shaking**: Frameworks import only the specific functions you use +- **Smaller final apps**: Your bundled application only includes what you actually need +- **Faster development builds**: No complex bundling during development +- **Better debugging**: Source maps point to individual files, not large bundles + +This change reduced the package size significantly while improving compatibility with Angular, React, Vue, and other modern framework build systems. ## ๐Ÿงฌ The Power of Nouns & Verbs @@ -600,30 +1059,29 @@ const results = await brainy.searchText("JavaScript with types", 5) console.log(results) ``` -### Vanilla JavaScript +### ๐ŸŒ Framework-First, Runs Everywhere -```html - - - - - - - -
- - -``` +**๐Ÿ—„๏ธ Auto-selected storage:** +- ๐ŸŒ **OPFS** - Browser frameworks (persistent, fast) +- ๐Ÿ“ **FileSystem** - Node.js servers (local development) +- โ˜๏ธ **S3/R2/GCS** - Production, serverless, distributed deployments +- ๐Ÿ’พ **Memory** - Edge workers, testing, temporary data + +**๐Ÿš€ Framework benefits:** +- โœ… **Proper bundling** - Handles dynamic imports and dependencies correctly +- โœ… **Type safety** - Full TypeScript integration and IntelliSense +- โœ… **State management** - Reactive updates and component lifecycle +- โœ… **Production ready** - Tree-shaking, optimization, error boundaries + +**Note:** We focus on framework support for reliability. Vanilla JS had too many module resolution issues. ### Cloudflare Workers diff --git a/demo/brainy-angular-demo/README.md b/demo/brainy-angular-demo/README.md new file mode 100644 index 00000000..c6451097 --- /dev/null +++ b/demo/brainy-angular-demo/README.md @@ -0,0 +1,59 @@ +# BrainyAngularDemo + +This project was generated using [Angular CLI](https://github.com/angular/angular-cli) version 20.1.4. + +## Development server + +To start a local development server, run: + +```bash +ng serve +``` + +Once the server is running, open your browser and navigate to `http://localhost:4200/`. The application will automatically reload whenever you modify any of the source files. + +## Code scaffolding + +Angular CLI includes powerful code scaffolding tools. To generate a new component, run: + +```bash +ng generate component component-name +``` + +For a complete list of available schematics (such as `components`, `directives`, or `pipes`), run: + +```bash +ng generate --help +``` + +## Building + +To build the project run: + +```bash +ng build +``` + +This will compile your project and store the build artifacts in the `dist/` directory. By default, the production build optimizes your application for performance and speed. + +## Running unit tests + +To execute unit tests with the [Karma](https://karma-runner.github.io) test runner, use the following command: + +```bash +ng test +``` + +## Running end-to-end tests + +For end-to-end (e2e) testing, run: + +```bash +ng e2e +``` + +Angular CLI does not come with an end-to-end testing framework by default. You can choose one that suits your needs. + +## Additional Resources + +For more information on using the Angular CLI, including detailed command references, visit the [Angular CLI Overview and Command Reference](https://angular.dev/tools/cli) page.