diff --git a/README.md b/README.md index 9b793e66..0be620e9 100644 --- a/README.md +++ b/README.md @@ -58,9 +58,20 @@ RUN npm run download-models # Download during build for offline production ## ✨ What is Brainy? -Imagine a database that thinks like you do - connecting ideas, finding patterns, and getting smarter over time. Brainy -is the **AI-native database** that brings vector search and knowledge graphs together in one powerful, ridiculously -easy-to-use package. +**One API. Every environment. Zero configuration.** + +Brainy is the **AI-native database** that combines vector search and knowledge graphs in one unified API. Write your code once, and it runs everywhere - browsers, Node.js, serverless, edge workers - with automatic optimization for each environment. + +```javascript +// This same code works EVERYWHERE +const brainy = new BrainyData() +await brainy.init() + +// Vector search (like Pinecone) + Graph database (like Neo4j) +await brainy.add("OpenAI", { type: "company" }) // Nouns +await brainy.relate(openai, gpt4, "develops") // Verbs +const results = await brainy.search("AI", 10) // Semantic search +``` ### πŸ†• NEW: Distributed Mode (v0.38+) @@ -103,41 +114,40 @@ npm install @soulcraft/brainy ```javascript import { BrainyData } from '@soulcraft/brainy' +// Same code works EVERYWHERE - browser, Node.js, cloud, edge const brainy = new BrainyData() await brainy.init() // Auto-detects your environment -// Add some data -await brainy.add("The quick brown fox jumps over the lazy dog") -await brainy.add("A fast fox leaps over a sleeping dog") -await brainy.add("Cats are independent and mysterious animals") +// 1️⃣ Simple vector search (like Pinecone) +await brainy.add("The quick brown fox jumps over the lazy dog", { type: "sentence" }) +await brainy.add("Cats are independent and mysterious animals", { type: "sentence" }) -// Vector search finds similar content -const results = await brainy.search("speedy animals jumping", 2) -console.log(results) // Finds the fox sentences! +const results = await brainy.search("fast animals", 5) +// Finds similar content by meaning, not keywords! + +// 2️⃣ Graph relationships (like Neo4j) +const openai = await brainy.add("OpenAI", { type: "company", founded: 2015 }) +const gpt4 = await brainy.add("GPT-4", { type: "product", released: 2023 }) +const sam = await brainy.add("Sam Altman", { type: "person", role: "CEO" }) + +// Create relationships between entities +await brainy.relate(openai, gpt4, "develops") +await brainy.relate(sam, openai, "leads") +await brainy.relate(gpt4, sam, "created_by") + +// 3️⃣ Combined power: Vector search + Graph traversal +const similar = await brainy.search("AI language models", 10) // Find by meaning +const products = await brainy.getVerbsBySource(openai) // Get relationships +const graph = await brainy.findSimilar(gpt4, { relationType: "develops" }) + +// 4️⃣ Advanced: Search with context +const contextual = await brainy.search("Who leads AI companies?", 5, { + includeVerbs: true, // Include relationships in results + nounTypes: ["person"], // Filter to specific entity types +}) ``` -**🎯 That's it!** You just built semantic search in 4 lines. Works in Angular, React, Vue, Node.js, browsers, -serverless - everywhere. - -## πŸš€ The Magic: Vector + Graph Database - -**Most databases do one thing.** Brainy does both vector similarity AND graph relationships: - -```javascript -// Add entities with relationships -const companyId = await brainy.addNoun("OpenAI creates powerful AI models", "company") -const productId = await brainy.addNoun("GPT-4 is a large language model", "product") - -// Connect them with relationships -await brainy.addVerb(companyId, productId, undefined, { type: "develops" }) - -// Now you can do BOTH: -const similar = await brainy.search("AI language models") // Vector similarity -const products = await brainy.getVerbsByType("develops") // Graph traversal -``` - -**Why this matters:** Find content by meaning AND follow relationships. It's like having PostgreSQL and Pinecone working -together seamlessly. +**🎯 That's it!** Vector search + graph database + works everywhere. No config needed. ### πŸ” Want More Power? @@ -162,336 +172,146 @@ returns "tiger" **🌐 Real-Time Collaboration** - Sync vector data across devices. Figma for AI data **πŸ₯ Medical Diagnosis Tools** - Match symptoms to conditions using embedding similarity -## πŸš€ Write-Once, Run-Anywhere Quick Start +## πŸš€ Works Everywhere - Same Code -Brainy uses the same code across all environments with automatic detection. **Framework-optimized** for the best -developer experience. Choose your environment: +**Write once, run anywhere.** Brainy auto-detects your environment and optimizes automatically: -### πŸ…°οΈ Angular (Latest) +### 🌐 Browser Frameworks (React, Angular, Vue) -```bash -npm install @soulcraft/brainy +```javascript +import { BrainyData } from '@soulcraft/brainy' + +// SAME CODE in React, Angular, Vue, Svelte, etc. +const brainy = new BrainyData() +await brainy.init() // Auto-uses OPFS in browsers + +// Add entities and relationships +const john = await brainy.add("John is a software engineer", { type: "person" }) +const jane = await brainy.add("Jane is a data scientist", { type: "person" }) +const ai = await brainy.add("AI Project", { type: "project" }) + +await brainy.relate(john, ai, "works_on") +await brainy.relate(jane, ai, "leads") + +// Search by meaning +const engineers = await brainy.search("software developers", 5) + +// Traverse relationships +const team = await brainy.getVerbsByTarget(ai) // Who works on AI Project? ``` +
+πŸ“¦ Full Angular Component Example + ```typescript import { Component, signal, OnInit } from '@angular/core' import { BrainyData } from '@soulcraft/brainy' @Component({ selector: 'app-search', - template: ` -
- - -
- @for (result of results(); track result.id) { -
- {{result.metadata?.category}}: {{result.metadata?.originalData}} - Similarity: {{result.score | number:'1.2-2'}} -
- } -
-
- ` + template: `` }) export class SearchComponent implements OnInit { - private brainy: BrainyData | null = null - results = signal([]) - query = '' - + brainy = new BrainyData() + async ngOnInit() { - // Auto-detects environment and uses OPFS storage in browsers - this.brainy = new BrainyData({ - defaultService: 'my-app' - }) await this.brainy.init() - - // Add sample data - await this.brainy.add("Cats are amazing pets", { category: "animals" }) - await this.brainy.add("Dogs love to play fetch", { category: "animals" }) - await this.brainy.add("Pizza is delicious food", { category: "food" }) + // Add your data... } - + async search(query: string) { - if (!query.trim() || !this.brainy) { - this.results.set([]) - return - } - - const searchResults = await this.brainy.search(query, 5) - this.results.set(searchResults) + const results = await this.brainy.search(query, 5) + // Display results... } } ``` +
-### βš›οΈ React - -```bash -npm install @soulcraft/brainy -``` +
+πŸ“¦ Full React Example ```jsx import { BrainyData } from '@soulcraft/brainy' import { useEffect, useState } from 'react' -function SemanticSearch() { +function Search() { const [brainy, setBrainy] = useState(null) const [results, setResults] = useState([]) - const [query, setQuery] = useState('') - const [loading, setLoading] = useState(true) useEffect(() => { - async function initBrainy() { - // Auto-detects environment and uses OPFS storage in browsers - const db = new BrainyData({ - defaultService: 'my-app' - }) + const init = async () => { + const db = new BrainyData() await db.init() - - // Add sample data - await db.add("Cats are amazing pets", { category: "animals" }) - await db.add("Dogs love to play fetch", { category: "animals" }) - await db.add("Pizza is delicious food", { category: "food" }) - + // Add your data... setBrainy(db) - setLoading(false) } - - initBrainy() + init() }, []) - const search = async (searchQuery) => { - if (!searchQuery.trim() || !brainy) return setResults([]) - - const searchResults = await brainy.search(searchQuery, 5) - setResults(searchResults) + const search = async (query) => { + const results = await brainy?.search(query, 5) || [] + setResults(results) } - if (loading) return
Initializing Brainy...
- - return ( -
- { - setQuery(e.target.value) - search(e.target.value) - }} - placeholder="Search by meaning (try 'pets' or 'food')..." - className="search-input" - /> - -
- {results.map((result, i) => ( -
- {result.metadata?.category}: {result.metadata?.originalData} - Similarity: {result.score.toFixed(2)} -
- ))} -
-
- ) + return search(e.target.value)} placeholder="Search..." /> } - -export default SemanticSearch ``` +
-### 🌟 Vue 3 - -```bash -npm install @soulcraft/brainy -``` +
+πŸ“¦ Full Vue Example ```vue - - - - + ``` +
-### 🟒 Node.js Server - -```bash -npm install @soulcraft/brainy -``` +### 🟒 Node.js / Serverless / Edge ```javascript import { BrainyData } from '@soulcraft/brainy' -// Auto-detects Node.js β†’ FileSystem (local) or S3 (production), Worker threads -const brainy = new BrainyData({ - defaultService: 'my-app', - // Optional: Production S3 storage - storage: { - s3Storage: { - bucketName: process.env.S3_BUCKET, - region: process.env.AWS_REGION, - accessKeyId: process.env.AWS_ACCESS_KEY_ID, - secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY - } +// SAME CODE works in Node.js, Vercel, Netlify, Cloudflare Workers, Deno, Bun +const brainy = new BrainyData() +await brainy.init() // Auto-detects environment and optimizes + +// Add entities and relationships +await brainy.add("Python is great for data science", { type: "fact" }) +await brainy.add("JavaScript rules the web", { type: "fact" }) + +// Search by meaning +const results = await brainy.search("programming languages", 5) + +// Optional: Production with S3/R2 storage (auto-detected in cloud environments) +const productionBrainy = new BrainyData({ + storage: { + s3Storage: { bucketName: process.env.BUCKET_NAME } } }) -await brainy.init() - -// Same API everywhere -await brainy.add("Cats are amazing pets", { category: "animals" }) -const results = await brainy.search("pets", 5) -console.log('Search results:', results) ``` -### ⚑ Serverless (Vercel/Netlify) - -```javascript -import { BrainyData } from '@soulcraft/brainy' - -export default async function handler(req, res) { - // Auto-detects serverless β†’ S3/R2 storage for persistence, or Memory for temp - const brainy = new BrainyData({ - defaultService: 'my-app', - // Optional: Explicit S3-compatible storage - storage: { - r2Storage: { - bucketName: process.env.R2_BUCKET, - accessKeyId: process.env.R2_ACCESS_KEY_ID, - secretAccessKey: process.env.R2_SECRET_ACCESS_KEY, - accountId: process.env.R2_ACCOUNT_ID - } - } - }) - await brainy.init() - - // Same API everywhere - const results = await brainy.search(req.query.q, 5) - res.json({ results }) -} -``` - -### πŸ”₯ Cloudflare Workers - -```javascript -import { BrainyData } from '@soulcraft/brainy' - -export default { - async fetch(request) { - // Auto-detects edge β†’ Minimal footprint, KV storage - const brainy = new BrainyData({ - defaultService: 'edge-app' - }) - await brainy.init() - - // Same API everywhere - const url = new URL(request.url) - const results = await brainy.search(url.searchParams.get('q'), 5) - return Response.json({ results }) - } -} -``` - -### πŸ¦• Deno - -```typescript -import { BrainyData } from 'https://esm.sh/@soulcraft/brainy' - -// Auto-detects Deno β†’ Native compatibility, FileSystem storage -const brainy = new BrainyData({ - defaultService: 'deno-app' -}) -await brainy.init() - -// Same API everywhere -await brainy.add("Deno is awesome", { category: "tech" }) -const results = await brainy.search("technology", 5) -console.log(results) -``` **That's it! Same code, everywhere. Zero-to-Smartβ„’** @@ -684,6 +504,7 @@ npm install @soulcraft/brainy @soulcraft/brainy-models - **🎯 Zero Configuration** - Automatic detection with graceful fallback - **πŸ” Enhanced Security** - Complete air-gapping support for sensitive environments - **🏒 Enterprise Ready** - Works behind corporate firewalls and restricted networks +- **βš–οΈ Compliance & Forensics** - Frozen mode for audit trails and legal discovery The offline models provide the **same functionality** with maximum reliability. Your existing code works unchanged - Brainy automatically detects and uses bundled models when available. @@ -876,7 +697,8 @@ const writer = createAutoBrainy({ const reader = createAutoBrainy({ storage: { s3Storage: { bucketName: 'my-bucket' } }, - readOnly: true, // Automatically becomes 'reader' role + readOnly: true, // Automatically becomes 'reader' role (allows optimizations) + // frozen: true, // Optional: Complete immutability for compliance/forensics distributed: true }) ``` @@ -938,6 +760,25 @@ const health = brainy.getHealthStatus() - **Writers**: Optimized write batching, minimal cache - **Hybrid**: Adaptive based on workload +## βš–οΈ Compliance & Forensics Mode + +For legal discovery, audit trails, and compliance requirements: + +```javascript +// Create a completely immutable snapshot +const auditDb = new BrainyData({ + storage: { s3Storage: { bucketName: 'audit-snapshots' } }, + readOnly: true, + frozen: true // Complete immutability - no changes allowed +}) + +// Perfect for: +// - Legal discovery (data cannot be modified) +// - Compliance audits (guaranteed state) +// - Forensic analysis (preserved evidence) +// - Regulatory snapshots (unchanging records) +``` + ### Deployment Examples **Docker Compose** @@ -1005,334 +846,148 @@ spec: ## πŸš€ Getting Started in 30 Seconds -### React +**The same Brainy code works everywhere - React, Vue, Angular, Node.js, Serverless, Edge Workers.** -```jsx -import { createAutoBrainy } from 'brainy' -import { useEffect, useState } from 'react' +```javascript +// This EXACT code works in ALL environments +import { BrainyData } from '@soulcraft/brainy' -function SemanticSearch() { - const [brainy] = useState(() => createAutoBrainy()) - const [results, setResults] = useState([]) +const brainy = new BrainyData() +await brainy.init() - const search = async (query) => { - const items = await brainy.searchText(query, 10) - setResults(items) - } +// Add nouns (entities) +const openai = await brainy.add("OpenAI", { type: "company" }) +const gpt4 = await brainy.add("GPT-4", { type: "product" }) - return ( - search(e.target.value)} - placeholder="Search by meaning..." /> - ) -} +// Add verbs (relationships) +await brainy.relate(openai, gpt4, "develops") + +// Vector search + Graph traversal +const similar = await brainy.search("AI companies", 5) +const products = await brainy.getVerbsBySource(openai) ``` -### Angular +
+πŸ” See Framework Examples -```typescript -import { Component, OnInit } from '@angular/core' -import { createAutoBrainy } from 'brainy' - -@Component({ - selector: 'app-search', - template: ` - -
- {{ result.text }} -
- ` -}) -export class SearchComponent implements OnInit { - brainy = createAutoBrainy() - results = [] - - async search(query: string) { - this.results = await this.brainy.searchText(query, 10) +### React +```jsx +function App() { + const [brainy] = useState(() => new BrainyData()) + useEffect(() => brainy.init(), []) + + const search = async (query) => { + return await brainy.search(query, 10) } + // Same API as above } ``` ### Vue 3 - ```vue - - - ``` -### Svelte - -```svelte - - - -{#each results as result} -
{result.text}
-{/each} -``` - -### Next.js (App Router) - -```jsx -// app/search/page.js -import { createAutoBrainy } from 'brainy' - -export default function SearchPage() { - async function search(formData) { - 'use server' - const brainy = createAutoBrainy({ bucketName: 'vectors' }) - const query = formData.get('query') - return await brainy.searchText(query, 10) - } - - return ( -
- - -
- ) } ``` -### Node.js / Bun / Deno - +### Node.js / Deno / Bun ```javascript -import { createAutoBrainy } from 'brainy' - -const brainy = createAutoBrainy() - -// Add some data -await brainy.add("TypeScript is a typed superset of JavaScript", { - category: 'programming' -}) - -// Search for similar content -const results = await brainy.searchText("JavaScript with types", 5) -console.log(results) +const brainy = new BrainyData() +await brainy.init() +// Same API as above ``` +
### 🌍 Framework-First, Runs Everywhere -**Brainy is designed for modern frameworks** with automatic environment detection and storage selection: +**Brainy automatically detects your environment and optimizes everything:** -**✨ Supported environments:** +| Environment | Storage | Optimization | +|------------|---------|-------------| +| 🌐 Browser | OPFS | Web Workers, Memory Cache | +| 🟒 Node.js | FileSystem / S3 | Worker Threads, Clustering | +| ⚑ Serverless | S3 / Memory | Cold Start Optimization | +| πŸ”₯ Edge Workers | Memory / KV | Minimal Footprint | +| πŸ¦• Deno/Bun | FileSystem / S3 | Native Performance | -- βš›οΈ **React/Vue/Angular** - Framework-optimized builds with proper bundling -- 🟒 **Node.js/Deno/Bun** - Full server-side capabilities -- ⚑ **Serverless/Edge** - Optimized for cold starts and minimal footprint -- 🧡 **Web/Worker threads** - Thread-safe, shared storage +## 🌐 Deploy to Any Cloud -**πŸ—„οΈ Auto-selected storage:** - -- 🌐 **OPFS** - Browser frameworks (persistent, fast) -- πŸ“ **FileSystem** - Node.js servers (local development) -- ☁️ **S3/R2/GCS** - Production, serverless, distributed deployments -- πŸ’Ύ **Memory** - Edge workers, testing, temporary data - -**πŸš€ Framework benefits:** - -- βœ… **Proper bundling** - Handles dynamic imports and dependencies correctly -- βœ… **Type safety** - Full TypeScript integration and IntelliSense -- βœ… **State management** - Reactive updates and component lifecycle -- βœ… **Production ready** - Tree-shaking, optimization, error boundaries - -**Note:** We focus on framework support for reliability. Vanilla JS had too many module resolution issues. +
+☁️ See Cloud Platform Examples ### Cloudflare Workers - ```javascript -import { createAutoBrainy } from 'brainy' +import { BrainyData } from '@soulcraft/brainy' export default { - async fetch(request, env) { - const brainy = createAutoBrainy({ - bucketName: env.R2_BUCKET - }) - + async fetch(request) { + const brainy = new BrainyData() + await brainy.init() + const url = new URL(request.url) - const query = url.searchParams.get('q') - - const results = await brainy.searchText(query, 10) + const results = await brainy.search(url.searchParams.get('q'), 10) return Response.json(results) } } ``` ### AWS Lambda - ```javascript -import { createAutoBrainy } from 'brainy' +import { BrainyData } from '@soulcraft/brainy' export const handler = async (event) => { - const brainy = createAutoBrainy({ - bucketName: process.env.S3_BUCKET - }) - - const results = await brainy.searchText(event.query, 10) - - return { - statusCode: 200, - body: JSON.stringify(results) - } -} -``` - -### Azure Functions - -```javascript -import { createAutoBrainy } from 'brainy' - -module.exports = async function(context, req) { - const brainy = createAutoBrainy({ - bucketName: process.env.AZURE_STORAGE_CONTAINER - }) - - const results = await brainy.searchText(req.query.q, 10) - - context.res = { - body: results - } + const brainy = new BrainyData() + await brainy.init() + + const results = await brainy.search(event.query, 10) + return { statusCode: 200, body: JSON.stringify(results) } } ``` ### Google Cloud Functions - ```javascript -import { createAutoBrainy } from 'brainy' +import { BrainyData } from '@soulcraft/brainy' export const searchHandler = async (req, res) => { - const brainy = createAutoBrainy({ - bucketName: process.env.GCS_BUCKET - }) - - const results = await brainy.searchText(req.query.q, 10) + const brainy = new BrainyData() + await brainy.init() + + const results = await brainy.search(req.query.q, 10) res.json(results) } ``` -### Google Cloud Run - -```dockerfile -# Dockerfile -FROM node:24-slim -USER node -WORKDIR /app -COPY package*.json ./ -RUN npm install brainy -COPY . . -CMD ["node", "server.js"] -``` - -```javascript -// server.js -import { createAutoBrainy } from 'brainy' -import express from 'express' - -const app = express() -const brainy = createAutoBrainy({ - bucketName: process.env.GCS_BUCKET -}) - -app.get('/search', async (req, res) => { - const results = await brainy.searchText(req.query.q, 10) - res.json(results) -}) - -const port = process.env.PORT || 8080 -app.listen(port, () => console.log(`Brainy on Cloud Run: ${port}`)) -``` - -```bash -# Deploy to Cloud Run -gcloud run deploy brainy-api \ - --source . \ - --platform managed \ - --region us-central1 \ - --allow-unauthenticated -``` - ### Vercel Edge Functions - ```javascript -import { createAutoBrainy } from 'brainy' +import { BrainyData } from '@soulcraft/brainy' -export const config = { - runtime: 'edge' -} +export const config = { runtime: 'edge' } export default async function handler(request) { - const brainy = createAutoBrainy() + const brainy = new BrainyData() + await brainy.init() + const { searchParams } = new URL(request.url) - const query = searchParams.get('q') - - const results = await brainy.searchText(query, 10) + const results = await brainy.search(searchParams.get('q'), 10) return Response.json(results) } ``` +
-### Netlify Functions - -```javascript -import { createAutoBrainy } from 'brainy' - -export async function handler(event, context) { - const brainy = createAutoBrainy() - const query = event.queryStringParameters.q - - const results = await brainy.searchText(query, 10) - - return { - statusCode: 200, - body: JSON.stringify(results) - } -} -``` - -### Supabase Edge Functions - -```typescript -import { createAutoBrainy } from 'brainy' -import { serve } from 'https://deno.land/std@0.168.0/http/server.ts' - -serve(async (req) => { - const brainy = createAutoBrainy() - const url = new URL(req.url) - const query = url.searchParams.get('q') - - const results = await brainy.searchText(query, 10) - - return new Response(JSON.stringify(results), { - headers: { 'Content-Type': 'application/json' } - }) -}) -``` ### Docker Container @@ -1420,6 +1075,7 @@ services: - [**Search and Metadata**](docs/user-guides/) - Advanced search techniques - [**JSON Document Search**](docs/guides/json-document-search.md) - Field-based searching +- [**Read-Only & Frozen Modes**](docs/guides/readonly-frozen-modes.md) - Immutability options for production - [**Production Migration**](docs/guides/production-migration-guide.md) - Deployment best practices ### API Reference diff --git a/docs/README.md b/docs/README.md index 83d76766..58796b3b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -17,6 +17,7 @@ Comprehensive guides for using Brainy features. - **[Search and Metadata Guide](user-guides/SEARCH_AND_METADATA_GUIDE.md)** - Advanced search techniques - **[Write-Only Mode](user-guides/WRITEONLY_MODE_IMPLEMENTATION.md)** - Optimized data ingestion +- **[Read-Only & Frozen Modes](guides/readonly-frozen-modes.md)** - Immutability control for production - **[Cache Configuration](guides/cache-configuration.md)** - Memory and caching optimization - **[JSON Document Search](guides/json-document-search.md)** - Searching within JSON documents - **[HNSW Field Search](guides/hnsw-field-search.md)** - Field-specific vector search diff --git a/docs/guides/readonly-frozen-modes.md b/docs/guides/readonly-frozen-modes.md new file mode 100644 index 00000000..99722ccc --- /dev/null +++ b/docs/guides/readonly-frozen-modes.md @@ -0,0 +1,257 @@ +# Read-Only and Frozen Modes Guide + +## Overview + +Brainy provides two levels of immutability control for different use cases: + +1. **`readOnly`** - Prevents data mutations while allowing performance optimizations +2. **`frozen`** - Completely freezes the database, preventing all changes + +## Quick Start + +```javascript +// Read-only mode (default behavior - allows optimizations) +const db = new BrainyData({ + readOnly: true // frozen defaults to false +}) + +// Completely frozen database (no changes at all) +const db = new BrainyData({ + readOnly: true, + frozen: true +}) +``` + +## Understanding the Modes + +### Read-Only Mode (`readOnly: true, frozen: false`) + +This is the **default and recommended** configuration for read-only databases. It provides: + +- βœ… **Data Protection**: No adds, updates, or deletes allowed +- βœ… **Performance Optimization**: Index rebalancing and cache updates continue +- βœ… **Live Monitoring**: Statistics and metrics stay current +- βœ… **Real-time Updates**: Can detect external changes if configured + +**Use cases:** +- Production read replicas +- Public-facing search APIs +- Analytics dashboards +- Most read-only scenarios + +```javascript +const db = new BrainyData({ + readOnly: true, + // frozen: false (default) + realtimeUpdates: { + enabled: true, // Still works! + interval: 30000 + } +}) + +// Data mutations are blocked +await db.add(data) // ❌ Throws error + +// But optimizations continue +await db.getStatistics({ forceRefresh: true }) // βœ… Works +await db.flushStatistics() // βœ… Works +// Index optimizations happen automatically // βœ… Works +``` + +### Frozen Mode (`frozen: true`) + +Complete immutability for special scenarios requiring absolutely no changes: + +- ❌ **No Data Changes**: Same as readOnly +- ❌ **No Optimizations**: Index remains exactly as-is +- ❌ **No Statistics Updates**: Metrics are frozen +- ❌ **No Real-time Updates**: All monitoring stopped + +**Use cases:** +- Forensic analysis +- Compliance/audit snapshots +- Deterministic testing +- Cryptographic verification + +```javascript +const db = new BrainyData({ + readOnly: true, // Usually set together + frozen: true // Complete immutability +}) + +// Everything is blocked or becomes a no-op +await db.add(data) // ❌ Throws error +await db.flushStatistics() // ⚠️ No-op (does nothing) +// No index changes // ❌ Disabled +// No cache updates // ❌ Disabled +``` + +## Dynamic Mode Changes + +You can change modes at runtime: + +```javascript +const db = new BrainyData() +await db.init() + +// Switch to read-only (optimizations continue) +db.setReadOnly(true) +console.log(db.isReadOnly()) // true +console.log(db.isFrozen()) // false + +// Freeze completely +db.setFrozen(true) +console.log(db.isFrozen()) // true + +// Unfreeze (real-time updates restart if configured) +db.setFrozen(false) + +// Allow writes again +db.setReadOnly(false) +``` + +## Configuration Examples + +### Example 1: High-Performance Read Replica + +```javascript +const readReplica = new BrainyData({ + readOnly: true, // Prevent writes + // frozen: false (default - allows optimizations) + + // Enable real-time sync with primary + realtimeUpdates: { + enabled: true, + interval: 10000, + updateStatistics: true, + updateIndex: true + }, + + // Aggressive caching for performance + cache: { + autoTune: true, + hotCacheMaxSize: 50000 + } +}) +``` + +### Example 2: Compliance Snapshot + +```javascript +const auditSnapshot = new BrainyData({ + readOnly: true, + frozen: true, // Complete immutability for compliance + + storage: { + s3Storage: { + bucketName: 'audit-snapshots', + // ... S3 credentials + } + } +}) + +// This database will never change, perfect for: +// - Legal discovery +// - Compliance audits +// - Historical analysis +``` + +### Example 3: Testing Environment + +```javascript +describe('Search Tests', () => { + let db + + beforeAll(async () => { + db = new BrainyData({ + readOnly: true, + frozen: true // Deterministic state for tests + }) + await db.init() + // Load test data... + }) + + it('should return consistent results', async () => { + // Tests run against unchanging data + const results = await db.search('test query') + expect(results).toHaveLength(3) + }) +}) +``` + +## Migration Guide + +If you're upgrading and using `readOnly`: + +### Previous Behavior (< v0.49.0) +```javascript +// Old: readOnly prevented ALL changes +const db = new BrainyData({ readOnly: true }) +// No optimizations, no statistics updates +``` + +### New Behavior (>= v0.49.0) +```javascript +// New default: readOnly allows optimizations +const db = new BrainyData({ readOnly: true }) +// Optimizations and statistics continue! + +// To get old behavior, add frozen: +const db = new BrainyData({ + readOnly: true, + frozen: true // Matches old behavior +}) +``` + +## Best Practices + +1. **Default to `readOnly` without `frozen`** for most read-only use cases +2. **Only use `frozen: true`** when you specifically need complete immutability +3. **Consider performance impact** - frozen mode disables beneficial optimizations +4. **Use dynamic switching** for temporary freezing during sensitive operations + +## API Reference + +### Configuration Options + +```typescript +interface BrainyDataConfig { + // Prevent data mutations (adds, updates, deletes) + readOnly?: boolean + + // Completely freeze database (no changes at all) + // Default: false (allows optimizations in readOnly mode) + frozen?: boolean + + // Other options... +} +``` + +### Methods + +```typescript +// Check current state +db.isReadOnly(): boolean +db.isFrozen(): boolean + +// Change state dynamically +db.setReadOnly(readOnly: boolean): void +db.setFrozen(frozen: boolean): void +``` + +### Behavior Matrix + +| Operation | Normal | ReadOnly | Frozen | +|-----------|--------|----------|---------| +| Add/Update/Delete | βœ… | ❌ | ❌ | +| Search/Get | βœ… | βœ… | βœ… | +| Statistics Refresh | βœ… | βœ… | ❌ | +| Index Optimization | βœ… | βœ… | ❌ | +| Real-time Updates | βœ… | βœ… | ❌ | +| Cache Updates | βœ… | βœ… | ❌ | + +## See Also + +- [Cache Configuration Guide](./cache-configuration.md) +- [Production Migration Guide](./production-migration-guide.md) +- [Real-time Updates Documentation](../technical/REALTIME_UPDATES.md) \ No newline at end of file diff --git a/docs/user-guides/README.md b/docs/user-guides/README.md index e2d77cc7..8c07cd33 100644 --- a/docs/user-guides/README.md +++ b/docs/user-guides/README.md @@ -20,6 +20,14 @@ Optimize data ingestion with write-only mode. - Performance optimization for writes - Use cases and implementations +### πŸ”’ [Read-Only & Frozen Modes](../guides/readonly-frozen-modes.md) +Control immutability for different production scenarios. + +- Read-only with optimizations (default) +- Completely frozen for compliance +- Dynamic mode switching +- Migration from older versions + ### πŸ’Ύ [Cache Configuration](../guides/cache-configuration.md) Configure caching for optimal performance. diff --git a/src/brainyData.ts b/src/brainyData.ts index bf4c460e..378c4bae 100644 --- a/src/brainyData.ts +++ b/src/brainyData.ts @@ -137,9 +137,18 @@ export interface BrainyDataConfig { /** * Set the database to read-only mode * When true, all write operations will throw an error + * Note: Statistics and index optimizations are still allowed unless frozen is also true */ readOnly?: boolean + /** + * Completely freeze the database, preventing all changes including statistics and index optimizations + * When true, the database is completely immutable (no data changes, no index rebalancing, no statistics updates) + * This is useful for forensic analysis, testing with deterministic state, or compliance scenarios + * Default: false (allows optimizations even in readOnly mode) + */ + frozen?: boolean + /** * Enable lazy loading in read-only mode * When true and in read-only mode, the index is not fully loaded during initialization @@ -370,6 +379,7 @@ export class BrainyData implements BrainyDataInterface { private distanceFunction: DistanceFunction private requestPersistentStorage: boolean private readOnly: boolean + private frozen: boolean private lazyLoadInReadOnlyMode: boolean private writeOnly: boolean private storageConfig: BrainyDataConfig['storage'] = {} @@ -488,6 +498,9 @@ export class BrainyData implements BrainyDataInterface { // Set read-only flag this.readOnly = config.readOnly || false + // Set frozen flag (defaults to false to allow optimizations in readOnly mode) + this.frozen = config.frozen || false + // Set lazy loading in read-only mode flag this.lazyLoadInReadOnlyMode = config.lazyLoadInReadOnlyMode || false @@ -602,6 +615,18 @@ export class BrainyData implements BrainyDataInterface { } } + /** + * Check if the database is frozen and throw an error if it is + * @throws Error if the database is frozen + */ + private checkFrozen(): void { + if (this.frozen) { + throw new Error( + 'Cannot perform operation: database is frozen (no changes allowed)' + ) + } + } + /** * Check if the database is in write-only mode and throw an error if it is * @param allowExistenceChecks If true, allows existence checks (get operations) in write-only mode @@ -625,6 +650,14 @@ export class BrainyData implements BrainyDataInterface { return } + // If the database is frozen, do not start real-time updates + if (this.frozen) { + if (this.loggingConfig?.verbose) { + console.log('Real-time updates disabled: database is frozen') + } + return + } + // If the update timer is already running, do nothing if (this.updateTimerId !== null) { return @@ -739,6 +772,11 @@ export class BrainyData implements BrainyDataInterface { return } + // If the database is frozen, do not perform updates + if (this.frozen) { + return + } + try { // Record the current time const startTime = Date.now() @@ -3873,6 +3911,11 @@ export class BrainyData implements BrainyDataInterface { throw new Error('Storage not initialized') } + // If the database is frozen, do not flush statistics + if (this.frozen) { + return + } + // Call the flushStatisticsToStorage method on the storage adapter await this.storage.flushStatisticsToStorage() } @@ -3881,6 +3924,11 @@ export class BrainyData implements BrainyDataInterface { * Update storage sizes if needed (called periodically for performance) */ private async updateStorageSizesIfNeeded(): Promise { + // If the database is frozen, do not update storage sizes + if (this.frozen) { + return + } + // Only update every minute to avoid performance impact const now = Date.now() const lastUpdate = (this as any).lastStorageSizeUpdate || 0 @@ -3963,8 +4011,8 @@ export class BrainyData implements BrainyDataInterface { await this.ensureInitialized() try { - // If forceRefresh is true, flush statistics to storage first - if (options.forceRefresh && this.storage) { + // If forceRefresh is true and not frozen, flush statistics to storage first + if (options.forceRefresh && this.storage && !this.frozen) { await this.storage.flushStatisticsToStorage() } @@ -4150,6 +4198,32 @@ export class BrainyData implements BrainyDataInterface { } } + /** + * Check if the database is frozen (completely immutable) + * @returns True if the database is frozen, false otherwise + */ + public isFrozen(): boolean { + return this.frozen + } + + /** + * Set the database to frozen mode (completely immutable) + * When frozen, no changes are allowed including statistics updates and index optimizations + * @param frozen True to freeze the database, false to allow optimizations + */ + public setFrozen(frozen: boolean): void { + this.frozen = frozen + + // If unfreezing and real-time updates are configured, restart them + if (!frozen && this.realtimeUpdateConfig.enabled && this.isInitialized) { + this.startRealtimeUpdates() + } + // If freezing, stop real-time updates + else if (frozen && this.updateTimerId !== null) { + this.stopRealtimeUpdates() + } + } + /** * Check if the database is in write-only mode * @returns True if the database is in write-only mode, false otherwise diff --git a/tests/frozen-flag.test.ts b/tests/frozen-flag.test.ts new file mode 100644 index 00000000..408fef50 --- /dev/null +++ b/tests/frozen-flag.test.ts @@ -0,0 +1,160 @@ +import { describe, it, expect, beforeAll, afterAll } from 'vitest' +import { BrainyData } from '../src/brainyData.js' +import { MemoryStorage } from '../src/storage/adapters/memoryStorage.js' + +describe('Frozen Flag Behavior', () => { + let db: BrainyData<{ content: string }> + + beforeAll(async () => { + // Create a test database with memory storage + db = new BrainyData({ + storageAdapter: new MemoryStorage() + }) + await db.init() + + // Add some test data + await db.add('test item 1', { content: 'First item' }) + await db.add('test item 2', { content: 'Second item' }) + }) + + afterAll(async () => { + await db.shutDown() + }) + + describe('readOnly mode without frozen', () => { + it('should prevent data mutations but allow statistics updates', async () => { + // Set to readOnly mode (frozen defaults to false) + db.setReadOnly(true) + expect(db.isReadOnly()).toBe(true) + expect(db.isFrozen()).toBe(false) + + // Data mutations should fail + await expect(db.add('test item 3', { content: 'Third item' })).rejects.toThrow('read-only mode') + await expect(db.delete('test-id')).rejects.toThrow('read-only mode') + + // Statistics should still be refreshable + const stats1 = await db.getStatistics() + await db.flushStatistics() // Should not throw + const stats2 = await db.getStatistics({ forceRefresh: true }) + + // Statistics operations should succeed + expect(stats1).toBeDefined() + expect(stats2).toBeDefined() + + // Reset + db.setReadOnly(false) + }) + + it('should allow real-time updates to continue', async () => { + // Enable real-time updates + db.enableRealtimeUpdates({ interval: 100 }) + + // Set to readOnly mode + db.setReadOnly(true) + + // Real-time updates should still be enabled + const config = db.getRealtimeUpdateConfig() + expect(config.enabled).toBe(true) + + // Disable and reset + db.disableRealtimeUpdates() + db.setReadOnly(false) + }) + }) + + describe('frozen mode', () => { + it('should prevent all changes including statistics and updates', async () => { + // Set to frozen mode + db.setFrozen(true) + expect(db.isFrozen()).toBe(true) + + // Enable real-time updates before freezing + db.enableRealtimeUpdates({ interval: 100 }) + + // Freeze the database + db.setFrozen(true) + + // Real-time updates should be stopped + const config = db.getRealtimeUpdateConfig() + // The config might still say enabled, but updates won't run + + // Statistics flush should be a no-op (not throw, just do nothing) + await db.flushStatistics() // Should not throw but does nothing + + // Reset + db.setFrozen(false) + db.disableRealtimeUpdates() + }) + + it('should restart real-time updates when unfrozen', async () => { + // Enable real-time updates + db.enableRealtimeUpdates({ interval: 100 }) + const configBefore = db.getRealtimeUpdateConfig() + expect(configBefore.enabled).toBe(true) + + // Freeze the database + db.setFrozen(true) + + // Unfreeze the database + db.setFrozen(false) + + // Real-time updates should restart + const configAfter = db.getRealtimeUpdateConfig() + expect(configAfter.enabled).toBe(true) + + // Cleanup + db.disableRealtimeUpdates() + }) + }) + + describe('readOnly with frozen', () => { + it('should enforce complete immutability', async () => { + // Set both readOnly and frozen + db.setReadOnly(true) + db.setFrozen(true) + + expect(db.isReadOnly()).toBe(true) + expect(db.isFrozen()).toBe(true) + + // Data mutations should fail + await expect(db.add('test item 3', { content: 'Third item' })).rejects.toThrow('read-only mode') + + // Statistics flush should be a no-op + await db.flushStatistics() // Should not throw but does nothing + + // Reset + db.setReadOnly(false) + db.setFrozen(false) + }) + }) + + describe('configuration via constructor', () => { + it('should respect frozen flag from constructor', async () => { + const frozenDb = new BrainyData({ + storageAdapter: new MemoryStorage(), + readOnly: true, + frozen: true + }) + await frozenDb.init() + + expect(frozenDb.isReadOnly()).toBe(true) + expect(frozenDb.isFrozen()).toBe(true) + + await frozenDb.shutDown() + }) + + it('should default frozen to false when readOnly is true', async () => { + const readOnlyDb = new BrainyData({ + storageAdapter: new MemoryStorage(), + readOnly: true + // frozen not specified, should default to false + }) + await readOnlyDb.init() + + expect(readOnlyDb.isReadOnly()).toBe(true) + expect(readOnlyDb.isFrozen()).toBe(false) + + await readOnlyDb.shutDown() + }) + }) +}) \ No newline at end of file