diff --git a/docs/BATCHING.md b/docs/BATCHING.md index 96239ba6..e70a9e18 100644 --- a/docs/BATCHING.md +++ b/docs/BATCHING.md @@ -78,10 +78,10 @@ for (const [id, metadata] of metadataMap) { - ✅ Direct O(1) path construction from ID (no type lookup needed!) - ✅ Sharding preservation (all paths include `{shard}/{id}`) - ✅ Write-cache coherent (read-after-write consistency) -- ✅ 40x faster than v5.x type-first architecture +- ✅ O(1) path construction — eliminates the per-entity type search of the old type-first layout **Performance:** -- ~1ms per 100 entities (consistent, no cache misses!) +- Constant-time path construction per ID — no type-cache misses - Filesystem: parallel reads bounded by IOPS - No type search delays — every ID maps directly to storage path @@ -121,7 +121,7 @@ for (const [sourceId, verbs] of results) { - Bulk export of relationship data **Performance:** -- Memory storage: <10ms for 1000 relationships +- Memory storage: single in-memory pass over the metadata index - Filesystem storage: parallel reads through the metadata index --- @@ -171,7 +171,7 @@ entities/verbs/{SHARD}/{ID}/metadata.json **Direct O(1) Path Construction:** ```typescript -// Every ID maps directly to exactly ONE path - 40x faster! +// Every ID maps directly to exactly ONE path - O(1), no type search const id = 'abc-123' const shard = getShardIdFromUuid(id) // → 'ab' (first 2 hex chars) const path = `entities/nouns/${shard}/${id}/metadata.json` @@ -183,9 +183,9 @@ const path = `entities/nouns/${shard}/${id}/metadata.json` ``` **Benefits:** -- **40x faster** path lookups (eliminates 42-type sequential search) +- **O(1)** path lookups (eliminates the 42-type sequential search the old type-first layout required) - **Simpler code** - removed 500+ lines of type cache complexity -- **Scalable** - works at billion-scale without type tracking overhead +- **Scalable** - works at large scale without type tracking overhead --- @@ -220,28 +220,22 @@ await db.release() --- -## Performance Benchmarks +## Why Batching Is Faster -### VFS Operations (12 Files) +Batching's advantage is structural, not a fixed multiplier (the actual speedup +depends on storage backend, IOPS, and batch size): -| Storage | Before optimization | After optimization | Improvement | -|---------|---------------------|--------------------|-------------| -| **Memory** | 150ms | 50ms | **67% faster** | -| **Filesystem** | ~500ms | ~80ms | **84% faster** | +- **N+1 elimination** — N sequential reads collapse into a single parallel pass + (`Promise.all` over the batch). +- **O(1) path construction** — every ID maps directly to one storage path, with + no per-type cache lookup. +- **One metadata round-trip** — relationship batches fetch all sources' metadata + in a single pass instead of one query per source. -### Entity Batch Retrieval (100 Entities) - -| Storage | Individual Gets | Batch Get | Improvement | -|---------|-----------------|-----------|-------------| -| **Memory** | 180ms | 15ms | **92% faster** | -| **Filesystem** | ~1.2s | ~120ms | **90% faster** | - -### Throughput (Entities/Second) - -| Storage | Individual | Batch | Improvement | -|---------|------------|-------|-------------| -| **Memory** | 556 ent/s | 6667 ent/s | **12x** | -| **Filesystem** | ~80 ent/s | ~800 ent/s | **10x** | +The integration test `tests/integration/storage-batch-operations.test.ts` +exercises batch vs. individual reads and asserts that batch retrieval is not +slower than the per-entity loop for large batches; it does not pin a specific +multiplier, since that is hardware- and IOPS-dependent. --- @@ -306,7 +300,7 @@ const results = await brain.batchGet(ids) const entities = Array.from(results.values()) ``` -**Performance Gain:** 10-20x faster on filesystem storage. +**Performance Gain:** Replaces N sequential reads with a single batched pass — no fixed multiplier, it scales with storage IOPS. --- @@ -332,7 +326,7 @@ for (const verbs of results.values()) { } ``` -**Performance Gain:** 5-10x faster due to batched metadata fetches. +**Performance Gain:** One batched metadata fetch instead of one query per source entity. --- @@ -450,8 +444,8 @@ return await Promise.all(resolvedPaths.map(path => this.read(path))) - `storage.getVerbsBySourceBatch(sourceIds, verbType?)` - Batch relationship queries **Performance Improvements:** -- VFS operations: 90%+ faster than the naive per-entity loop -- Entity retrieval: 10-20x throughput improvement +- VFS operations: single batched pass instead of N sequential reads +- Entity retrieval: N+1 reads collapsed into one batched pass - Zero N+1 query patterns **Compatibility:** diff --git a/docs/FIND_SYSTEM.md b/docs/FIND_SYSTEM.md index 5bf8ef1a..1cc38ce9 100644 --- a/docs/FIND_SYSTEM.md +++ b/docs/FIND_SYSTEM.md @@ -22,7 +22,7 @@ Brainy's `find()` method is the most advanced query system in any vector databas ### 1. Vector Intelligence (HNSW Index) - **Purpose**: Semantic similarity search using embeddings - **Algorithm**: Hierarchical Navigable Small World (HNSW) -- **Performance**: O(log n) search, ~1.8ms typical +- **Performance**: O(log n) search - **Data Structure**: Multi-layer graph with 16 connections per node - **Use Cases**: "Find similar documents", "Content like this" @@ -36,14 +36,14 @@ Brainy's `find()` method is the most advanced query system in any vector databas ### 3. Metadata Intelligence (Incremental Indices) - **Purpose**: Fast filtering on structured data - **Algorithm**: HashMap for exact matches, Sorted arrays for ranges -- **Performance**: O(1) exact, O(log n) ranges, <1ms typical +- **Performance**: O(1) exact, O(log n) ranges - **Data Structure**: `Map>` + sorted value arrays - **Use Cases**: "Documents from 2023", "Status equals active" ### 4. Graph Intelligence (Adjacency Maps) - **Purpose**: Relationship traversal and connection analysis - **Algorithm**: Pure O(1) neighbor lookups via Map operations -- **Performance**: O(1) per hop, ~0.1ms typical +- **Performance**: O(1) per hop (measured <1 ms per neighbor lookup, validated up to 1M relationships — `tests/performance/graph-scale-performance.test.ts:238`) - **Data Structure**: `Map>` - **Use Cases**: "Papers connected to MIT", "Authors who collaborated" @@ -373,36 +373,40 @@ return results.slice(offset, offset + limit) ### Query Performance by Type -| Query Type | Index Used | Performance | Example | -|------------|------------|-------------|---------| -| **Semantic Search** | HNSW Vector | O(log n), ~1.8ms | `"AI research papers"` | -| **Exact Metadata** | HashMap | O(1), ~0.8ms | `{status: "published"}` | -| **Range Metadata** | Sorted Array | O(log n), ~0.6ms | `{year: {greaterThan: 2020}}` | -| **Graph Traversal** | Adjacency Map | O(1), ~0.1ms | `{connected: {to: "mit"}}` | -| **Type Detection** | Pre-embedded Types | O(t), ~0.3ms | `"documents"` → `NounType.Document` | -| **Field Matching** | Field Embeddings | O(f), ~0.1ms | `"by"` → `"author"` | -| **Combined Query** | All Indices | O(log n), ~1.8ms | NLP + filters + graph | +| Query Type | Index Used | Complexity | Example | +|------------|------------|------------|---------| +| **Semantic Search** | HNSW Vector | O(log n) | `"AI research papers"` | +| **Exact Metadata** | HashMap | O(1) | `{status: "published"}` | +| **Range Metadata** | Sorted Array | O(log n) | `{year: {greaterThan: 2020}}` | +| **Graph Traversal** | Adjacency Map | O(1) | `{connected: {to: "mit"}}` | +| **Type Detection** | Pre-embedded Types | O(t) | `"documents"` → `NounType.Document` | +| **Field Matching** | Field Embeddings | O(f) | `"by"` → `"author"` | +| **Combined Query** | All Indices | O(log n) | NLP + filters + graph | Where: - n = number of entities in database - t = number of types (169 total: 42 noun + 127 verb) - f = number of fields for detected entity type (typically 5-15) +Absolute latencies depend on hardware, embedding model, and storage backend; the columns above describe how each stage scales. The graph stage is the one path with a committed scale benchmark (see [Scalability](#scalability)). + ### Scalability -| Database Size | Vector Search | Metadata Filter | Graph Query | Combined | -|---------------|---------------|-----------------|-------------|----------| -| **1K entities** | 0.8ms | 0.3ms | 0.05ms | 1.1ms | -| **10K entities** | 1.2ms | 0.5ms | 0.08ms | 1.5ms | -| **100K entities** | 1.8ms | 0.8ms | 0.1ms | 2.1ms | -| **1M entities** | 2.5ms | 1.2ms | 0.1ms | 2.8ms | -| **10M entities** | 3.8ms | 1.8ms | 0.1ms | 4.2ms | +The cost of each query stage is governed by its algorithmic complexity, not a fixed millisecond figure — absolute latency depends on hardware, embedding model, and storage backend. Only the graph adjacency index carries a committed scale assertion: + +| Query stage | Complexity | Scaling behavior | +|-------------|------------|------------------| +| Metadata filter (exact) | O(1) | Constant — independent of dataset size | +| Metadata filter (range) | O(log n) + O(k) | Sub-linear; k = matching results | +| Vector search (HNSW) | O(log n) | Degrades gracefully via hierarchical layers | +| Graph hop | O(1) | Measured <1 ms per neighbor lookup, validated up to 1M relationships (`tests/performance/graph-scale-performance.test.ts:238`) | +| Combined query | O(log n) | Bounded by the vector stage; metadata and graph stages stay O(1)/O(log n) | **Key Performance Notes:** - Graph queries stay O(1) regardless of scale - Metadata ranges scale as O(log n), not O(n) - Vector search degrades gracefully due to HNSW -- Type-aware NLP adds minimal overhead (~0.4ms) +- Type-aware NLP adds minimal overhead (single embedding pass, no full scan) ## Example Query Flows @@ -455,7 +459,7 @@ await brain.find({ }, limit: 10 }) -// Total performance: ~1.2ms for 100K entities +// Executes as O(1) type filter + O(1)/O(log n) metadata + O(1) graph lookup — no full scan ``` ## Filter Syntax Reference @@ -1001,7 +1005,7 @@ const results = await brain.find({ }) ``` -**Performance**: O(log n) vector search + O(log n) metadata filters + O(1) graph traversal = ~2-3ms total. +**Performance**: O(log n) vector search + O(log n) metadata filters + O(1) graph traversal — bounded by the vector stage. ### Excluding Soft-Deleted Entities diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md index 0caa123d..dc9244d9 100644 --- a/docs/PERFORMANCE.md +++ b/docs/PERFORMANCE.md @@ -2,11 +2,11 @@ ## Performance Characteristics -Brainy achieves industry-leading performance through carefully optimized data structures and algorithms. All performance claims are verified through actual benchmarks on production code. +Brainy achieves high performance through carefully optimized data structures and algorithms. The tables below describe each component by its **algorithmic complexity** — the durable, defensible guarantee. The example latencies are figures from a single 100-item run on one machine (see [Benchmarks](#benchmarks)); they are illustrative, not a committed benchmark, and vary with hardware, embedding model, and storage backend. The one component with a committed scale assertion is the graph adjacency index (`tests/performance/graph-scale-performance.test.ts:238`). ### Core Performance Summary -| Component | Operation | Time Complexity | Measured Performance | Data Structure | +| Component | Operation | Time Complexity | Example latency (100-item run)\* | Data Structure | |-----------|-----------|-----------------|---------------------|----------------| | **Metadata Index** | Exact match | **O(1)** | 0.8ms | `Map>` | | **Metadata Index** | Range query | **O(log n) + O(k)** | 0.6ms | Sorted array + binary search | @@ -17,6 +17,8 @@ Brainy achieves industry-leading performance through carefully optimized data st | **Type Detection** | Noun/Verb matching | **O(t)** | 0.3ms | Pre-embedded type vectors | | **Triple Intelligence** | Combined query | **O(1) to O(log n)** | 1.8ms | Parallel execution | +\* Illustrative single-run figures at 100 items on one machine — not a committed benchmark. Only the graph index carries an asserted scale bound (measured <1 ms per neighbor lookup up to 1M relationships, `tests/performance/graph-scale-performance.test.ts:238`). + Where: - `n` = number of items in index - `k` = number of results returned @@ -26,25 +28,32 @@ Where: ### brain.get() Metadata-Only Optimization -✨ **Massive Performance Improvement**: `brain.get()` is now **76-81% faster** by default! +`brain.get()` returns **metadata only by default**, skipping the 384-dimensional +embedding — the bulk of an entity's payload. Callers that need the vector opt in +with `{ includeVectors: true }`. -| Operation | Before | After | Speedup | Use Case | -|-----------|------------------|-----------------|---------|----------| -| **brain.get() (metadata-only)** | 43ms, 6KB | **10ms, 300 bytes** | **76-81%** | VFS, existence checks, metadata | -| **brain.get({ includeVectors: true })** | 43ms, 6KB | 43ms, 6KB | 0% | Similarity calculations | -| **VFS readFile()** | 53ms | **~13ms** | **75%** | File operations | -| **VFS readdir(100 files)** | 5.3s | **~1.3s** | **75%** | Directory listings | +| Operation | Default (metadata-only) | With `includeVectors: true` | Use Case | +|-----------|-------------------------|-----------------------------|----------| +| **brain.get()** | Skips vector load | Loads full vector | VFS, existence checks, metadata | +| **VFS readFile() / readdir()** | Inherits metadata-only path | n/a | File operations, directory listings | -**Key Innovation**: Lazy vector loading - only load 384-dimensional embeddings when explicitly needed. +**Key Innovation**: Lazy vector loading — only load the 384-dimensional embedding when explicitly needed. + +The integration test `tests/integration/metadata-only-comprehensive.test.ts:306` +asserts metadata-only `get()` is faster than the full-entity `get()` +(`metadataTime < fullTime`). The *magnitude* of the speedup is +environment-dependent (the percentage assertion in +`tests/integration/vfs-performance-v5.11.1.test.ts` is intentionally skipped on +CI for that reason), so no fixed percentage is quoted here. **Why this matters**: -- **94% of brain.get() calls** don't need vectors (VFS, admin tools, import utilities, data APIs) -- **Vector data is 95% of entity size** (6KB vectors vs 300 bytes metadata) -- **Zero code changes** for most applications - automatic speedup! +- Most `brain.get()` calls don't need vectors (VFS, admin tools, import utilities, data APIs) +- The embedding dominates an entity's serialized size, so skipping it is the largest win +- **Zero code changes** for most applications — automatic by default **When to use what**: ```typescript -// DEFAULT: Metadata-only (76-81% faster) - use for: +// DEFAULT: Metadata-only (skips the vector load) - use for: const entity = await brain.get(id) // - VFS operations (readFile, stat, readdir) // - Existence checks: if (await brain.get(id)) ... @@ -252,28 +261,34 @@ const results = await Promise.all(searchPromises) ## Benchmarks -### Real-world Performance Test (100 items) +### Illustrative Single Run (100 items, one machine) + +Example output from a single 100-item run — illustrative only, not a committed +benchmark; absolute numbers vary by hardware. The values feed the +[Core Performance Summary](#core-performance-summary) example-latency column. ``` -📊 Metadata exact match: 0.818ms (50 items matched) -📊 Metadata range query: 0.631ms (40 items in range) -🔗 Graph neighbor lookup: 0.092ms (2 connections) -🎯 Vector k-NN search: 1.773ms (10 nearest neighbors) -🧠 NLP query parsing: 8.906ms (full natural language) -⚡ Triple Intelligence: 1.830ms (combined query) +Metadata exact match: 0.818ms (50 items matched) +Metadata range query: 0.631ms (40 items in range) +Graph neighbor lookup: 0.092ms (2 connections) +Vector k-NN search: 1.773ms (10 nearest neighbors) +NLP query parsing: 8.906ms (full natural language) +Triple Intelligence: 1.830ms (combined query) ``` ### Scaling Characteristics -| Items | Metadata O(1) | Range O(log n) | Graph O(1) | Vector O(log n) | -|-------|---------------|----------------|------------|-----------------| -| 100 | 0.8ms | 0.6ms | 0.09ms | 1.8ms | -| 1,000 | 0.8ms | 0.9ms | 0.09ms | 2.5ms | -| 10,000 | 0.8ms | 1.2ms | 0.09ms | 3.2ms | -| 100,000 | 0.8ms | 1.5ms | 0.09ms | 4.1ms | -| 1,000,000 | 0.8ms | 1.8ms | 0.09ms | 5.0ms | +Each stage scales by its algorithmic complexity, not a fixed millisecond figure +— absolute latency depends on hardware, embedding model, and storage backend. +Only the graph adjacency index carries a committed scale assertion: -*Note: O(1) operations maintain constant time regardless of scale* +| Query stage | Complexity | Scaling behavior | +|-------------|------------|------------------| +| Metadata filter (exact) | O(1) | Constant — independent of dataset size | +| Metadata filter (range) | O(log n) + O(k) | Sub-linear; k = matching results | +| Vector search (HNSW) | O(log n) | Degrades gracefully via hierarchical layers | +| Graph hop | O(1) | Measured <1 ms per neighbor lookup, validated up to 1M relationships (`tests/performance/graph-scale-performance.test.ts:238`) | +| Combined query | O(log n) | Bounded by the vector stage; metadata and graph stages stay O(1)/O(log n) | ## Comparison with Other Systems @@ -470,8 +485,8 @@ For off-site replication, snapshot `path` from your scheduler (`gsutil rsync`, ` - **Auto-configuration system** with environment detection - **Zero-config operation** with intelligent defaults - **Auto-flush and auto-optimize** in indices -- **Sub-2ms response times** for complex queries +- **Low-latency Triple Intelligence queries** (O(log n) vector + O(1) metadata/graph) ## Conclusion -Brainy delivers on its promise of **production-ready Triple Intelligence** with measured, verified performance characteristics. All listed features are fully implemented, tested, and benchmarked. No stubs, no mocks, no theoretical claims - just real, working code with measured performance. \ No newline at end of file +Brainy delivers on its promise of **production-ready Triple Intelligence** with documented algorithmic-complexity guarantees and a committed graph-scale benchmark (`tests/performance/graph-scale-performance.test.ts`). All listed features are fully implemented and tested. No stubs, no mocks — just real, working code with characterized performance. \ No newline at end of file diff --git a/docs/architecture/index-architecture.md b/docs/architecture/index-architecture.md index db45d298..4d9f37fe 100644 --- a/docs/architecture/index-architecture.md +++ b/docs/architecture/index-architecture.md @@ -183,7 +183,7 @@ async getIdsForMultipleFields(pairs: [{field, value}, ...]): Promise { - Memory usage: **90% reduction** (17.17 MB → 2.01 MB for 100K entities) - Hardware acceleration: SIMD instructions make bitmap operations nearly free -**Benchmark Results** (1,000 queries on various dataset sizes): +**Benchmark Results** — example output from a single run of `tests/performance/roaring-bitmap-benchmark.ts` (1,000 queries per size, one machine; absolute times vary by hardware, the relative speedup and memory savings are the durable signal): | Dataset Size | Operation | Set Time | Roaring Time | Speedup | Memory Savings | |--------------|-----------|----------|--------------|---------|----------------| | 10,000 entities | 3-field intersection | 3.74ms | 1.14ms | **3.3x faster** | 90% | @@ -842,14 +842,15 @@ Where: ### Scalability -All indexes scale gracefully: +All indexes scale gracefully. The cost of each stage is governed by its algorithmic complexity, not a fixed millisecond figure — absolute latency depends on hardware, embedding model, and storage backend. Only the graph adjacency index carries a committed scale assertion: -| Database Size | Metadata Filter | Vector Search | Graph Hop | Combined Query | -|---------------|----------------|---------------|-----------|----------------| -| **1K entities** | 0.3ms | 0.8ms | 0.05ms | 1.1ms | -| **10K entities** | 0.5ms | 1.2ms | 0.08ms | 1.5ms | -| **100K entities** | 0.8ms | 1.8ms | 0.1ms | 2.1ms | -| **1M entities** | 1.2ms | 2.5ms | 0.1ms | 2.8ms | +| Query stage | Complexity | Scaling behavior | +|-------------|------------|------------------| +| Metadata filter (exact) | O(1) | Constant — independent of dataset size | +| Metadata filter (range) | O(log n) + O(k) | Sub-linear; k = matching results | +| Vector search (HNSW) | O(log n) | Degrades gracefully via hierarchical layers | +| Graph hop | O(1) | Measured <1 ms per neighbor lookup, validated up to 1M relationships (`tests/performance/graph-scale-performance.test.ts:238`) | +| Combined query | O(log n) | Bounded by the vector stage; metadata and graph stages stay O(1)/O(log n) | **Key observations**: - Graph queries stay O(1) regardless of scale diff --git a/docs/architecture/noun-verb-taxonomy.md b/docs/architecture/noun-verb-taxonomy.md index d896016d..286464be 100644 --- a/docs/architecture/noun-verb-taxonomy.md +++ b/docs/architecture/noun-verb-taxonomy.md @@ -5,7 +5,7 @@ public: true category: concepts template: concept order: 2 -description: 42 NounTypes and 127 VerbTypes cover 96-97% of all human knowledge. The universal vocabulary for structuring anything from people and documents to events and relationships. +description: 42 NounTypes and 127 VerbTypes cover ~95% of all domains. The universal vocabulary for structuring anything from people and documents to events and relationships. next: - concepts/triple-intelligence - api/reference @@ -14,114 +14,141 @@ next: # The Universal Knowledge Protocol: Noun-Verb Taxonomy > **Brainy is the Universal Knowledge Protocol™ powered by Triple Intelligence™** -> -> We're the world's first to unify vector, graph, and document search in one magical API. This breakthrough—Triple Intelligence—enables us to create a universal language for knowledge that all tools, augmentations, and AI models can speak. +> +> Brainy unifies vector, graph, and document search behind one API. That unification — Triple Intelligence — rests on a shared, standardized vocabulary for knowledge: a fixed set of entity types (nouns) and relationship types (verbs) that every tool, integration, and model can speak. + +Every example on this page is written against the real Brainy 8.0 API. The setup is always the same: + +```typescript +import { Brainy, NounType, VerbType } from '@soulcraft/brainy' + +const brain = new Brainy() +await brain.init() +``` + +- `brain.add({ data, type, subtype?, metadata? })` creates a noun and returns its `string` id. +- `brain.relate({ from, to, type, metadata? })` creates a verb (relationship) between two nouns. +- `brain.find({ query?, type?, where?, connected? })` runs Triple Intelligence search. +- `brain.related({ from })` / `brain.neighbors(id)` read a noun's relationships. ## Universal & Infinite Expressiveness -Brainy's **Noun-Verb Taxonomy** achieves **universal coverage** of all human knowledge through **infinite expressiveness**: +Brainy's **Noun-Verb Taxonomy** achieves broad coverage of human knowledge through composable expressiveness: - **42 Noun Types × 127 Verb Types = 5,334 Base Combinations** -- **Unlimited Metadata Fields = ∞ Domain Specificity** -- **Multi-hop Graph Traversals = ∞ Relationship Complexity** -- **Result: Can Model ANY Data in ANY Industry** +- **Unlimited Metadata Fields = Domain Specificity** +- **Multi-hop Graph Traversals = Relationship Complexity** +- **Result: Model data across virtually any industry** -This isn't marketing—it's mathematically provable. Every piece of information that exists can be represented as entities (nouns) connected by relationships (verbs) with properties (metadata). +Every piece of information can be represented as entities (nouns) connected by relationships (verbs) carrying properties (metadata). The standardized type system from `@soulcraft/brainy` (`NounType`, `VerbType`) gives those nouns and verbs a stable, shared name. ## The Power of Standardization: Universal Interoperability ### Why Standardized Types = Seamless Integration -The standardized noun-verb taxonomy creates a **universal language** that enables: +Because every entity is classified with a `NounType` and every relationship with a `VerbType`, the same data is legible to any code that imports the same enums. + +#### 1. Tool Interoperability -#### 1. **Tool Interoperability** ```typescript -// Any tool that understands Brainy types can work with any other -const analyticsAugmentation = await brain.augment('analytics') -const visualizationAugmentation = await brain.augment('visualization') +// Any tool that understands Brainy's NounType/VerbType can read the same graph. +// One module writes; another reads — no schema translation in between. +const authors = await brain.find({ type: NounType.Person }) -// Both understand "person", "document", "creates" without translation -const authors = await analyticsAugmentation.findTopAuthors() -await visualizationAugmentation.graphRelationships(authors) +for (const author of authors) { + const authored = await brain.related({ + from: author.id, + type: VerbType.Creates + }) + console.log(`${author.data} → ${authored.length} document(s)`) +} ``` -#### 2. **Data Portability** +#### 2. Data Portability + ```typescript -// Snapshot from one Brainy instance, restore into another — -// types are universally understood +// Snapshot one brain and open it as another — the noun/verb vocabulary +// travels with the data, so the types line up exactly. const pin = brain1.now() -await pin.persist('/snapshots/brain1') -await pin.release() +try { + await pin.persist('/snapshots/brain-1') +} finally { + await pin.release() +} -const brain2 = await Brainy.load('/snapshots/brain1') // Types match perfectly +const brain2 = await Brainy.load('/snapshots/brain-1') // same NounType/VerbType vocabulary ``` -#### 3. **AI Model Compatibility** +#### 3. Model & Agent Compatibility + ```typescript -// Different AI models can share the same knowledge graph -const gptBrain = await brain.connectModel('gpt-4') -const claudeBrain = await brain.connectModel('claude-3') -const llamaBrain = await brain.connectModel('llama-2') - -// All models understand the same noun-verb structure -const knowledge = await brain.add("Quantum Computer", { type: "thing" }) -// Any model can now reason about this knowledge -``` - -#### 4. **Augmentation Ecosystem** -```typescript -// Augmentations build on standard types, ensuring compatibility -await brain.augment.install('medical-records') // Extends "person" type -await brain.augment.install('financial-analysis') // Extends "transaction" events -await brain.augment.install('social-graph') // Uses "follows", "likes" verbs - -// All augmentations work together seamlessly -const patient = await brain.find("patient with financial transactions who follows Dr. Smith") -``` - -#### 5. **Cross-Platform Integration** -```typescript -// Standard types enable integration with external systems -// CRM understands "person" and "organization" -await brain.sync.salesforce({ - mapping: { - Contact: "person", - Account: "organization", - Opportunity: "event" - } +// Different models and agents reason over the SAME typed structure. +// Whatever produced the entity, every consumer reads the same NounType. +const conceptId = await brain.add({ + data: 'Quantum Computer', + type: NounType.Thing, + subtype: 'computing-hardware' }) -// Project management understands "task" and "project" -await brain.sync.jira({ - mapping: { - Issue: "task", - Epic: "project", - Sprint: "event" - } -}) +// Any downstream consumer can now retrieve and reason about this entity. +const concept = await brain.get(conceptId) +console.log(concept?.type) // 'thing' +``` + +#### 4. Extensibility Without Forking the Schema + +```typescript +// Subtypes extend a standard NounType for a specific domain — no schema +// migration, no new noun type. The base type stays universally understood. +await brain.add({ data: 'Patient #12345', type: NounType.Person, subtype: 'patient' }) +await brain.add({ data: 'Invoice #4471', type: NounType.Document, subtype: 'invoice' }) +await brain.add({ data: 'Follower edge', type: NounType.Relationship, subtype: 'social-graph' }) + +// Domains that share the base types interoperate even with custom subtypes. +const patients = await brain.find({ type: NounType.Person, subtype: 'patient' }) +``` + +#### 5. Cross-Platform Integration + +```typescript +// Map external systems onto the standard taxonomy. A CRM's Contact/Account/ +// Opportunity become Person/Organization/Event — the same vocabulary everywhere. +const externalRecords = [ + { kind: 'Contact', name: 'Dana Lee', type: NounType.Person }, + { kind: 'Account', name: 'Acme Corp', type: NounType.Organization }, + { kind: 'Opportunity', name: 'Q3 Renewal', type: NounType.Event } +] + +for (const record of externalRecords) { + await brain.add({ + data: record.name, + type: record.type, + metadata: { source: 'crm', externalKind: record.kind } + }) +} ``` ### The Network Effect: Brainy as the Universal Knowledge Protocol -Like **HTTP** became the protocol for the web and **TCP/IP** for the internet, Brainy's noun-verb taxonomy is becoming the **Universal Knowledge Protocol**: +Like **HTTP** became the protocol for the web and **TCP/IP** for the internet, Brainy's noun-verb taxonomy aims to be a **Universal Knowledge Protocol**: -- **Learn Once**: Developers learn 42 nouns + 127 verbs, not 1000s of schemas +- **Learn Once**: Developers learn 42 nouns + 127 verbs, not thousands of bespoke schemas - **Build Anywhere**: Tools built for one domain work in others - **Share Everything**: Knowledge graphs are universally shareable -- **Compose Freely**: Augmentations compose without conflicts +- **Compose Freely**: Subtypes and metadata extend types without schema migrations -This isn't just a database—it's a **protocol for how humanity represents knowledge**. +This isn't just a database — it's a **shared model for how knowledge is represented**. ## Overview -Brainy 2.0 introduces a revolutionary **Noun-Verb Taxonomy** that models data as entities (nouns) and relationships (verbs), creating a semantic knowledge graph that mirrors how humans naturally think about information. +Brainy's **Noun-Verb Taxonomy** models data as entities (nouns) and relationships (verbs), creating a semantic knowledge graph that mirrors how humans naturally think about information. ## Why Noun-Verb? Traditional databases force you to think in tables, documents, or nodes. Brainy lets you think naturally: - **Nouns**: Things that exist (people, documents, products, concepts) -- **Verbs**: How things relate (creates, owns, references, similar-to) +- **Verbs**: How things relate (creates, owns, references, related-to) This simple mental model scales from basic storage to complex knowledge graphs while remaining intuitive. @@ -129,70 +156,78 @@ This simple mental model scales from basic storage to complex knowledge graphs w ### Nouns (Entities) -Nouns represent any entity in your system: +Nouns represent any entity in your system. `add()` takes a single object and returns the new entity's id: ```typescript // Add any entity as a noun -const personId = await brain.add("John Smith, Senior Engineer", { - type: "person", - department: "engineering", - skills: ["TypeScript", "React", "Node.js"] +const personId = await brain.add({ + data: 'John Smith, Senior Engineer', + type: NounType.Person, + subtype: 'employee', + metadata: { department: 'engineering', skills: ['TypeScript', 'React', 'Node.js'] } }) -const documentId = await brain.add("Q3 2024 Financial Report", { - type: "document", - category: "financial", - confidential: true, - created: "2024-10-01" +const documentId = await brain.add({ + data: 'Q3 2024 Financial Report', + type: NounType.Document, + subtype: 'report', + metadata: { category: 'financial', confidential: true, created: '2024-10-01' } }) -const conceptId = await brain.add("Machine Learning", { - type: "concept", - domain: "technology", - complexity: "advanced" +const conceptId = await brain.add({ + data: 'Machine Learning', + type: NounType.Concept, + metadata: { domain: 'technology', complexity: 'advanced' } }) ``` #### Noun Properties Every noun automatically gets: -- **Unique ID**: System-generated or custom -- **Vector Embedding**: For semantic similarity -- **Metadata**: Flexible JSON properties -- **Timestamps**: Created/updated tracking -- **Indexing**: Automatic field indexing +- **Unique ID**: System-generated UUID, or supply your own via `id` +- **Vector Embedding**: `data` is embedded for semantic similarity +- **Metadata**: Flexible, queryable JSON properties +- **Timestamps**: `createdAt` / `updatedAt` tracking +- **Indexing**: Automatic field indexing for `where` filters ### Verbs (Relationships) -Verbs define how nouns relate to each other: +Verbs define how nouns relate to each other. `relate()` also takes a single object: ```typescript // Create relationships between entities -await brain.relate(personId, documentId, "authored", { - role: "primary_author", - contribution: "80%" +await brain.relate({ + from: personId, + to: documentId, + type: VerbType.Creates, + metadata: { role: 'primary_author', contribution: '80%' } }) -await brain.relate(documentId, conceptId, "discusses", { - sections: ["methodology", "results"], - depth: "detailed" +await brain.relate({ + from: documentId, + to: conceptId, + type: VerbType.Describes, + metadata: { sections: ['methodology', 'results'], depth: 'detailed' } }) -await brain.relate(personId, conceptId, "expert_in", { - years_experience: 5, - certification: "Advanced ML Certification" +await brain.relate({ + from: personId, + to: conceptId, + type: VerbType.RelatedTo, + subtype: 'expertise', + metadata: { yearsExperience: 5, certification: 'Advanced ML Certification' } }) ``` #### Verb Properties Every verb includes: -- **Source**: The noun initiating the relationship -- **Target**: The noun receiving the relationship -- **Type**: The relationship type/name -- **Direction**: Directional or bidirectional -- **Metadata**: Relationship-specific data -- **Strength**: Optional relationship weight +- **Source** (`from`): The noun initiating the relationship +- **Target** (`to`): The noun receiving the relationship +- **Type**: The `VerbType` classification +- **Subtype**: Optional per-product sub-classification (fast-path indexed) +- **Metadata**: Relationship-specific queryable data +- **Weight**: Optional relationship strength (0–1) ## Benefits @@ -200,92 +235,104 @@ Every verb includes: ```typescript // Think naturally about your data -const taskId = await brain.add("Implement payment system") -const userId = await brain.add("Alice Johnson") -const projectId = await brain.add("E-commerce Platform") +const taskId = await brain.add({ data: 'Implement payment system', type: NounType.Task }) +const userId = await brain.add({ data: 'Alice Johnson', type: NounType.Person }) +const projectId = await brain.add({ data: 'E-commerce Platform', type: NounType.Project }) // Express relationships clearly -await brain.relate(userId, taskId, "assigned_to") -await brain.relate(taskId, projectId, "part_of") -await brain.relate(userId, projectId, "manages") +await brain.relate({ from: userId, to: taskId, type: VerbType.ParticipatesIn, subtype: 'assignee' }) +await brain.relate({ from: taskId, to: projectId, type: VerbType.PartOf }) +await brain.relate({ from: userId, to: projectId, type: VerbType.ParticipatesIn, subtype: 'manager' }) ``` ### 2. Semantic Understanding -The noun-verb model preserves meaning: +The noun-verb model preserves meaning. `find()` accepts a natural-language string or a structured query: ```typescript -// The system understands semantic relationships -const results = await brain.find("tasks assigned to Alice") -// Automatically understands: assigned_to verb + Alice noun +// Natural language — embedded and matched semantically +const results = await brain.find({ query: 'tasks for the payment system' }) -const related = await brain.find("people who manage projects with payment tasks") -// Traverses: person -> manages -> project -> contains -> task +// Structured — type + graph traversal in one call +const aliceTasks = await brain.find({ + type: NounType.Task, + connected: { from: userId, via: VerbType.ParticipatesIn } +}) ``` ### 3. Flexible Schema -No rigid schema requirements: +No rigid schema requirements — add any type, extend with a `subtype`: ```typescript // Add any noun type without schema changes -await brain.add("New IoT Sensor", { - type: "device", - protocol: "MQTT", - location: "Building A" +const sensorId = await brain.add({ + data: 'New IoT Sensor', + type: NounType.Thing, + subtype: 'iot-device', + metadata: { protocol: 'MQTT', location: 'Building A' } }) -// Create new relationship types on the fly -await brain.relate(sensorId, buildingId, "monitors", { - metrics: ["temperature", "humidity"], - interval: "5 minutes" +const buildingId = await brain.add({ data: 'Building A', type: NounType.Location }) + +// Relationships carry their own structured metadata +await brain.relate({ + from: sensorId, + to: buildingId, + type: VerbType.Measures, + metadata: { metrics: ['temperature', 'humidity'], interval: '5 minutes' } }) ``` ### 4. Graph Traversal -Navigate relationships naturally: +Navigate relationships naturally with `connected`: ```typescript -// Find all documents authored by team members +// Find documents reachable from a team via two relationship hops const teamDocs = await brain.find({ + type: NounType.Document, connected: { from: teamId, - through: ["member_of", "authored"], + via: [VerbType.MemberOf, VerbType.Creates], depth: 2 } }) -// Find similar products purchased by similar users +// Find products two hops out from a user const recommendations = await brain.find({ + type: NounType.Product, connected: { from: userId, - through: ["similar_to", "purchased"], - depth: 2, - type: "product" + via: VerbType.Owns, + depth: 2 } }) ``` ### 5. Temporal Relationships -Track how relationships change over time: +Track how relationships change over time by storing dates in edge metadata: ```typescript -// Relationships with temporal data -await brain.relate(employeeId, companyId, "worked_at", { - from: "2020-01-01", - to: "2023-12-31", - position: "Senior Developer" +await brain.relate({ + from: employeeId, + to: companyId, + type: VerbType.MemberOf, + subtype: 'past-employment', + metadata: { from: '2020-01-01', to: '2023-12-31', position: 'Senior Developer' } }) -await brain.relate(employeeId, newCompanyId, "works_at", { - from: "2024-01-01", - position: "Tech Lead" +await brain.relate({ + from: employeeId, + to: newCompanyId, + type: VerbType.MemberOf, + subtype: 'current-employment', + metadata: { from: '2024-01-01', position: 'Tech Lead' } }) -// Query historical relationships -const employment = await brain.find("where did John work in 2022") +// Query with natural language +const employment = await brain.find({ query: 'where did this person work in 2022' }) ``` ## Real-World Use Cases @@ -294,97 +341,108 @@ const employment = await brain.find("where did John work in 2022") ```typescript // Documents and their relationships -const paperId = await brain.add("Neural Networks Paper", { - type: "research_paper", - year: 2024 +const paperId = await brain.add({ + data: 'Neural Networks Paper', + type: NounType.Document, + subtype: 'research-paper', + metadata: { year: 2024 } }) -const authorId = await brain.add("Dr. Sarah Chen", { - type: "researcher" +const authorId = await brain.add({ + data: 'Dr. Sarah Chen', + type: NounType.Person, + subtype: 'researcher' }) -const topicId = await brain.add("Deep Learning", { - type: "topic" -}) +const topicId = await brain.add({ data: 'Deep Learning', type: NounType.Concept }) +const otherPaperId = await brain.add({ data: 'Backpropagation Survey', type: NounType.Document }) // Rich relationship network -await brain.relate(authorId, paperId, "authored") -await brain.relate(paperId, topicId, "covers") -await brain.relate(paperId, otherPaperId, "cites") -await brain.relate(authorId, topicId, "researches") +await brain.relate({ from: authorId, to: paperId, type: VerbType.Creates }) +await brain.relate({ from: paperId, to: topicId, type: VerbType.Describes }) +await brain.relate({ from: paperId, to: otherPaperId, type: VerbType.References }) +await brain.relate({ from: authorId, to: topicId, type: VerbType.RelatedTo, subtype: 'research-focus' }) // Query the knowledge graph -const related = await brain.find("papers about deep learning by Sarah Chen") +const related = await brain.find({ query: 'papers about deep learning by Sarah Chen' }) ``` ### Social Networks ```typescript // Users and connections -const user1 = await brain.add("Alice", { type: "user" }) -const user2 = await brain.add("Bob", { type: "user" }) -const post = await brain.add("Great article on AI!", { type: "post" }) +const user1 = await brain.add({ data: 'Alice', type: NounType.Person }) +const user2 = await brain.add({ data: 'Bob', type: NounType.Person }) +const post = await brain.add({ data: 'Great article on AI!', type: NounType.Message, subtype: 'post' }) // Social interactions -await brain.relate(user1, user2, "follows") -await brain.relate(user2, user1, "follows") // Mutual -await brain.relate(user1, post, "created") -await brain.relate(user2, post, "liked") -await brain.relate(user2, post, "shared") +await brain.relate({ from: user1, to: user2, type: VerbType.Follows }) +await brain.relate({ from: user2, to: user1, type: VerbType.Follows }) // mutual +await brain.relate({ from: user1, to: post, type: VerbType.Creates }) +await brain.relate({ from: user2, to: post, type: VerbType.Likes }) +await brain.relate({ from: user2, to: post, type: VerbType.Communicates, subtype: 'share' }) // Find social patterns -const influencers = await brain.find("users with most followers who post about AI") +const influencers = await brain.find({ query: 'people who post about AI with many followers' }) ``` ### E-commerce ```typescript // Products and purchases -const product = await brain.add("Wireless Headphones", { - type: "product", - price: 99.99, - category: "electronics" +const product = await brain.add({ + data: 'Wireless Headphones', + type: NounType.Product, + metadata: { price: 99.99, category: 'electronics' } }) -const customer = await brain.add("Customer #12345", { - type: "customer", - tier: "premium" +const customer = await brain.add({ + data: 'Customer #12345', + type: NounType.Person, + subtype: 'customer', + metadata: { tier: 'premium' } }) -// Purchase relationships -await brain.relate(customer, product, "purchased", { - date: "2024-01-15", - quantity: 1, - price: 99.99 +// Purchase and review relationships +await brain.relate({ + from: customer, + to: product, + type: VerbType.Owns, + subtype: 'purchase', + metadata: { date: '2024-01-15', quantity: 1, price: 99.99 } }) -await brain.relate(customer, product, "reviewed", { - rating: 5, - text: "Excellent sound quality!" +await brain.relate({ + from: customer, + to: product, + type: VerbType.Evaluates, + subtype: 'review', + metadata: { rating: 5, text: 'Excellent sound quality!' } }) // Recommendation queries -const recs = await brain.find("products purchased by customers who bought headphones") +const recs = await brain.find({ query: 'products bought by customers who bought headphones' }) ``` ### Project Management ```typescript // Projects, tasks, and teams -const project = await brain.add("Website Redesign", { type: "project" }) -const task = await brain.add("Update homepage", { type: "task" }) -const developer = await brain.add("Jane Developer", { type: "person" }) -const designer = await brain.add("John Designer", { type: "person" }) +const project = await brain.add({ data: 'Website Redesign', type: NounType.Project }) +const task = await brain.add({ data: 'Update homepage', type: NounType.Task }) +const otherTask = await brain.add({ data: 'Design system audit', type: NounType.Task }) +const developer = await brain.add({ data: 'Jane Developer', type: NounType.Person, subtype: 'employee' }) +const designer = await brain.add({ data: 'John Designer', type: NounType.Person, subtype: 'employee' }) // Work relationships -await brain.relate(task, project, "belongs_to") -await brain.relate(developer, task, "assigned_to") -await brain.relate(designer, developer, "collaborates_with") -await brain.relate(task, otherTask, "depends_on") +await brain.relate({ from: task, to: project, type: VerbType.PartOf }) +await brain.relate({ from: developer, to: task, type: VerbType.ParticipatesIn, subtype: 'assignee' }) +await brain.relate({ from: designer, to: developer, type: VerbType.WorksWith }) +await brain.relate({ from: task, to: otherTask, type: VerbType.DependsOn }) // Project queries -const blockers = await brain.find("tasks that depend on incomplete tasks") -const workload = await brain.find("people assigned to multiple active projects") +const blockers = await brain.find({ query: 'tasks blocked by incomplete work' }) +const workload = await brain.find({ query: 'people assigned to multiple active projects' }) ``` ## Advanced Patterns @@ -392,54 +450,56 @@ const workload = await brain.find("people assigned to multiple active projects") ### Bidirectional Relationships ```typescript -// Some relationships are naturally bidirectional -await brain.relate(user1, user2, "friend_of", { bidirectional: true }) -// Automatically creates inverse relationship +// Symmetric relationships create the inverse edge automatically +await brain.relate({ from: user1, to: user2, type: VerbType.FriendOf, bidirectional: true }) ``` ### Weighted Relationships ```typescript -// Add strength/weight to relationships -await brain.relate(doc1, doc2, "similar_to", { - similarity_score: 0.95, - algorithm: "cosine" +// Add strength/weight to relationships (top-level weight, 0–1) +await brain.relate({ + from: doc1, + to: doc2, + type: VerbType.SimilarityDegree, + weight: 0.95, + metadata: { algorithm: 'cosine' } }) -// Use weights in queries -const stronglyRelated = await brain.find({ - connected: { - type: "similar_to", - minWeight: 0.8 - } -}) +// Weights come back on the Relation, so you can filter on them +const edges = await brain.related({ from: doc1, type: VerbType.SimilarityDegree }) +const stronglyRelated = edges.filter((edge) => (edge.weight ?? 0) >= 0.8) ``` -### Relationship Chains +### Relationship Chains (Multi-hop) ```typescript -// Follow relationship chains +// Follow a chain of relationship types out to a fixed depth. +// `via` accepts an array of VerbTypes; `depth` bounds the traversal. const results = await brain.find({ + type: NounType.Thing, connected: { from: userId, - chain: [ - { type: "owns", to: "company" }, - { type: "produces", to: "product" }, - { type: "uses", to: "technology" } - ] + via: [VerbType.Owns, VerbType.Creates, VerbType.Uses], + depth: 3 } }) -// Finds: technologies used by products made by companies owned by user +// Finds: things used by products made by companies owned by the user ``` ### Meta-Relationships +Relationships can themselves be reasoned about. The `Relationship` NounType reifies an edge as a first-class entity, and the meta-level verbs (`Endorses`, `Supports`, `Contradicts`, `Supersedes`) express second-order claims between entities: + ```typescript -// Relationships about relationships -const verbId = await brain.relate(user1, user2, "recommends") -await brain.relate(user3, verbId, "endorses", { - reason: "Accurate recommendation", - trust_score: 0.9 +// A second person endorses a claim, and a third supports it with evidence. +const claim = await brain.add({ data: 'X improves retention', type: NounType.Proposition }) +await brain.relate({ from: user2, to: claim, type: VerbType.Endorses }) +await brain.relate({ + from: user3, + to: claim, + type: VerbType.Supports, + metadata: { reason: 'Matches the A/B test', trustScore: 0.9 } }) ``` @@ -449,1121 +509,1048 @@ await brain.relate(user3, verbId, "endorses", { ```typescript // By type -const people = await brain.find({ where: { type: "person" } }) +const people = await brain.find({ type: NounType.Person }) -// By properties +// By type + metadata filters (bare operators — no `$` prefixes) const documents = await brain.find({ + type: NounType.Document, where: { - type: "document", confidential: false, - created: { $gte: "2024-01-01" } + created: { gte: '2024-01-01' } } }) -// By similarity +// By semantic similarity — use `query`, optionally narrowed by type const similar = await brain.find({ - like: "machine learning research", - where: { type: "document" } + query: 'machine learning research', + type: NounType.Document }) ``` -### Finding Verbs +> **`where` operators** are bare (never dollar-prefixed): `eq`/`equals`/`is`, `ne`/`notEquals`, `in`/`oneOf`, `gt`/`greaterThan`, `gte`/`greaterThanOrEqual`, `lt`/`lessThan`, `lte`/`lessThanOrEqual`, `between`, `contains`, `exists`, `missing`, plus the logical combinators `allOf`/`anyOf`/`not`. + +### Finding Verbs (Relationships) ```typescript -// Get all relationships for a noun -const relationships = await brain.getVerbs(nounId) +// All relationships originating from a noun +const outgoing = await brain.related({ from: nounId }) -// Find specific relationship types -const authorships = await brain.find({ - verb: { - type: "authored", - from: authorId - } -}) +// Every edge touching a noun, in either direction +const incident = await brain.related({ node: nounId }) -// Find by relationship properties -const recentPurchases = await brain.find({ - verb: { - type: "purchased", - where: { - date: { $gte: "2024-01-01" } - } - } -}) +// Filter by relationship type +const authorships = await brain.related({ from: authorId, type: VerbType.Creates }) + +// Filter returned relationships by their metadata (Relation carries `.metadata`) +const purchases = await brain.related({ from: customerId, type: VerbType.Owns, subtype: 'purchase' }) +const recentPurchases = purchases.filter((edge) => edge.metadata?.date >= '2024-01-01') + +// Just the count of relationships in the graph +const totalEdges = await brain.getVerbCount() ``` -### Combined Queries +### Combined Queries (Query → Expand) ```typescript -// Find nouns through relationships +// Start from a semantic query, then expand along the graph. +// Vector + graph in a single find() call. const results = await brain.find({ - // Start with similar documents - like: "AI research", - // That are authored by + query: 'AI research', connected: { - through: "authored", - // People who work at - where: { - connected: { - to: "Stanford", - type: "works_at" - } - } + via: VerbType.Creates, + depth: 2 } }) ``` -## Performance Optimizations +## The Complete Noun Taxonomy (42 Types) -### Noun Indexing -- Automatic vector indexing for similarity -- Field indexing for metadata queries -- Full-text indexing for content search +`NounType` is the stable, exported vocabulary for classifying entities. Every value is a plain string, so you can write `NounType.Person` or the literal `'person'`. Pick the closest standard type and refine with `subtype` and `metadata`. -### Verb Indexing -- Relationship type indexing -- Source/target indexing -- Temporal indexing for time-based queries - -### Query Optimization -- Automatic query plan optimization -- Parallel execution of independent operations -- Result caching for repeated queries - -## Best Practices - -1. **Use Descriptive Types**: Make noun and verb types self-documenting -2. **Rich Metadata**: Include relevant metadata for better querying -3. **Consistent Naming**: Use consistent verb names across your application -4. **Temporal Data**: Include timestamps for time-based analysis -5. **Bidirectional When Appropriate**: Mark symmetric relationships as bidirectional - -## Migration from Traditional Models - -### From Relational (SQL) ```typescript -// Instead of JOIN queries -// SELECT * FROM users JOIN orders ON users.id = orders.user_id - -// Use noun-verb relationships -const userId = await brain.add("User", userData) -const orderId = await brain.add("Order", orderData) -await brain.relate(userId, orderId, "placed") - -// Query naturally -const userOrders = await brain.find({ - connected: { from: userId, type: "placed" } +const physicistId = await brain.add({ + data: 'Albert Einstein', + type: NounType.Person, + metadata: { role: 'physicist', born: '1879-03-14' } }) ``` -### From Document (NoSQL) -```typescript -// Instead of embedded documents -// { user: { orders: [...] } } - -// Use explicit relationships -const userId = await brain.add("User", userData) -for (const order of orders) { - const orderId = await brain.add("Order", order) - await brain.relate(userId, orderId, "has_order") -} -``` +### Core Entity Types (7) -### From Graph Databases -```typescript -// Similar to graph databases but with added benefits: -// 1. Automatic vector embeddings for similarity -// 2. Natural language querying -// 3. Unified with metadata filtering +| NounType | Value | Use for | +|----------|-------|---------| +| `Person` | `'person'` | Individual human entities | +| `Organization` | `'organization'` | Companies, institutions, collectives | +| `Location` | `'location'` | Geographic and named spatial entities | +| `Thing` | `'thing'` | Discrete physical objects and artifacts | +| `Concept` | `'concept'` | Abstract ideas, principles, intangibles | +| `Event` | `'event'` | Temporal occurrences and happenings | +| `Agent` | `'agent'` | Non-human autonomous actors (AI agents, bots) | -// Enhanced graph queries -const results = await brain.find("similar users who purchased similar products") -``` +### Biological & Material Types (2) -## Universal Knowledge Coverage +| NounType | Value | Use for | +|----------|-------|---------| +| `Organism` | `'organism'` | Living biological entities (animals, plants, bacteria) | +| `Substance` | `'substance'` | Physical materials and matter (water, iron, DNA) | -The Noun-Verb taxonomy is designed to represent **all human knowledge** through a comprehensive set of types that can be combined infinitely. +### Property, Temporal & Functional Types (3) -### Complete Noun Types (31 Types) +| NounType | Value | Use for | +|----------|-------|---------| +| `Quality` | `'quality'` | Properties and attributes that inhere in entities | +| `TimeInterval` | `'timeInterval'` | Temporal regions, periods, durations | +| `Function` | `'function'` | Purposes, capabilities, functional roles | -#### Core Entity Types (6) +### Informational Type (1) -##### 1. **Person** - Individual human entities -```typescript -await brain.add("Albert Einstein", { - type: "person", - role: "physicist", - born: "1879-03-14" -}) -``` +| NounType | Value | Use for | +|----------|-------|---------| +| `Proposition` | `'proposition'` | Statements, claims, assertions, declarative content | -##### 2. **Organization** - Collective entities -```typescript -await brain.add("OpenAI", { - type: "organization", - industry: "AI research", - founded: 2015 -}) -``` +### Digital/Content Types (4) -##### 3. **Location** - Geographic and spatial entities -```typescript -await brain.add("San Francisco", { - type: "location", - category: "city", - coordinates: [37.7749, -122.4194] -}) -``` +| NounType | Value | Use for | +|----------|-------|---------| +| `Document` | `'document'` | Text-based files and written content | +| `Media` | `'media'` | Non-text media (audio, video, images) | +| `File` | `'file'` | Generic digital files and data blobs | +| `Message` | `'message'` | Communication content and correspondence | -##### 4. **Thing** - Physical objects -```typescript -await brain.add("Tesla Model 3", { - type: "thing", - category: "vehicle", - manufacturer: "Tesla" -}) -``` +### Collection Types (2) -##### 5. **Concept** - Abstract ideas and intangibles -```typescript -await brain.add("Machine Learning", { - type: "concept", - domain: "technology", - complexity: "advanced" -}) -``` +| NounType | Value | Use for | +|----------|-------|---------| +| `Collection` | `'collection'` | Groups and sets of items | +| `Dataset` | `'dataset'` | Structured data collections and databases | -##### 6. **Event** - Temporal occurrences -```typescript -await brain.add("Product Launch 2024", { - type: "event", - date: "2024-09-15", - attendees: 500 -}) -``` +### Business/Application Types (4) -#### Digital/Content Types (5) +| NounType | Value | Use for | +|----------|-------|---------| +| `Product` | `'product'` | Commercial products and offerings | +| `Service` | `'service'` | Service offerings and intangible products | +| `Task` | `'task'` | Actions, todos, work items | +| `Project` | `'project'` | Organized initiatives and programs | -##### 7. **Document** - Text-based files -```typescript -await brain.add("Quarterly Report", { - type: "document", - format: "PDF", - pages: 47 -}) -``` +### Descriptive Types (6) -##### 8. **Media** - Non-text media files -```typescript -await brain.add("Product Demo Video", { - type: "media", - format: "MP4", - duration: "5:30" -}) -``` +| NounType | Value | Use for | +|----------|-------|---------| +| `Process` | `'process'` | Workflows, procedures, ongoing activities | +| `State` | `'state'` | Conditions, status, situational contexts | +| `Role` | `'role'` | Positions, responsibilities, classifications | +| `Language` | `'language'` | Natural and formal languages | +| `Currency` | `'currency'` | Monetary units and exchange mediums | +| `Measurement` | `'measurement'` | Metrics, quantities, measured values | -##### 9. **File** - Generic digital files -```typescript -await brain.add("config.json", { - type: "file", - size: "2KB", - modified: Date.now() -}) -``` +### Scientific & Legal Types (4) -##### 10. **Message** - Communication content -```typescript -await brain.add("Support ticket #1234", { - type: "message", - priority: "high", - channel: "email" -}) -``` +| NounType | Value | Use for | +|----------|-------|---------| +| `Hypothesis` | `'hypothesis'` | Scientific theories, research hypotheses | +| `Experiment` | `'experiment'` | Controlled studies, trials, methodologies | +| `Contract` | `'contract'` | Legal agreements, terms, binding documents | +| `Regulation` | `'regulation'` | Laws, rules, compliance requirements | -##### 11. **Content** - Generic content -```typescript -await brain.add("Landing page copy", { - type: "content", - category: "marketing", - language: "en" -}) -``` - -#### Collection Types (2) +### Technical Infrastructure Types (2) -##### 12. **Collection** - Groups of items -```typescript -await brain.add("Premium Features", { - type: "collection", - items: 25, - category: "features" -}) -``` +| NounType | Value | Use for | +|----------|-------|---------| +| `Interface` | `'interface'` | APIs, protocols, specifications, endpoints | +| `Resource` | `'resource'` | Compute, bandwidth, storage, infrastructure assets | -##### 13. **Dataset** - Structured data collections -```typescript -await brain.add("Customer Analytics", { - type: "dataset", - records: 10000, - schema: "v2" -}) -``` +### Social Structure Types (3) -#### Business/Application Types (5) +| NounType | Value | Use for | +|----------|-------|---------| +| `SocialGroup` | `'socialGroup'` | Informal social groups and collectives | +| `Institution` | `'institution'` | Formal social structures and practices | +| `Norm` | `'norm'` | Social norms, conventions, expectations | -##### 14. **Product** - Commercial offerings -```typescript -await brain.add("Pro Subscription", { - type: "product", - price: 99.99, - tier: "premium" -}) -``` +### Information Theory Types (2) -##### 15. **Service** - Service offerings -```typescript -await brain.add("Cloud Hosting", { - type: "service", - sla: "99.9%", - region: "us-west" -}) -``` +| NounType | Value | Use for | +|----------|-------|---------| +| `InformationContent` | `'informationContent'` | Abstract information (stories, ideas, schemas) | +| `InformationBearer` | `'informationBearer'` | Physical or digital carrier of information | -##### 16. **User** - User accounts -```typescript -await brain.add("user@example.com", { - type: "user", - tier: "enterprise", - created: Date.now() -}) -``` +### Meta-Level & Extensible Types (2) -##### 17. **Task** - Actions and todos -```typescript -await brain.add("Deploy v2.0", { - type: "task", - priority: "high", - assignee: "devops" -}) -``` +| NounType | Value | Use for | +|----------|-------|---------| +| `Relationship` | `'relationship'` | Relationships reified as first-class entities | +| `Custom` | `'custom'` | Domain-specific entities outside the standard set | -##### 18. **Project** - Organized initiatives -```typescript -await brain.add("Website Redesign", { - type: "project", - deadline: "2024-12-31", - status: "active" -}) -``` +## The Complete Verb Taxonomy (127 Types) -#### Descriptive Types (7) +`VerbType` is the exported vocabulary for classifying relationships. As with nouns, every value is a plain string — write `VerbType.Creates` or `'creates'`. Where no verb is an exact fit, choose the closest base verb and refine it with `subtype` and `metadata` (see [Coverage Completeness](#coverage-completeness-analysis)). -##### 19. **Process** - Workflows and procedures ```typescript -await brain.add("CI/CD Pipeline", { - type: "process", - steps: 7, - automated: true -}) +await brain.relate({ from: authorId, to: documentId, type: VerbType.Creates }) ``` -##### 20. **State** - Conditions or status -```typescript -await brain.add("System Health", { - type: "state", - status: "operational", - uptime: "99.99%" -}) -``` +### Foundational Ontological (3) -##### 21. **Role** - Positions or responsibilities -```typescript -await brain.add("Admin Role", { - type: "role", - permissions: ["read", "write", "delete"], - level: "superuser" -}) -``` +| VerbType | Value | Meaning | +|----------|-------|---------| +| `InstanceOf` | `'instanceOf'` | Individual to class (Fido instanceOf Dog) | +| `SubclassOf` | `'subclassOf'` | Taxonomic hierarchy (Dog subclassOf Mammal) | +| `ParticipatesIn` | `'participatesIn'` | Entity participation in events/processes | -##### 22. **Topic** - Subjects or themes -```typescript -await brain.add("Machine Learning", { - type: "topic", - field: "AI", - popularity: "high" -}) -``` +### Core Relationships (4) -##### 23. **Language** - Languages or linguistic entities -```typescript -await brain.add("English", { - type: "language", - iso_code: "en", - speakers_millions: 1500 -}) -``` +| VerbType | Value | Meaning | +|----------|-------|---------| +| `RelatedTo` | `'relatedTo'` | Generic relationship (fallback) | +| `Contains` | `'contains'` | Containment relationship | +| `PartOf` | `'partOf'` | Part-whole (mereological) relationship | +| `References` | `'references'` | Citation and referential relationship | -##### 24. **Currency** - Monetary units -```typescript -await brain.add("US Dollar", { - type: "currency", - symbol: "$", - code: "USD" -}) -``` - -##### 25. **Measurement** - Metrics or quantities -```typescript -await brain.add("Temperature Reading", { - type: "measurement", - value: 23.5, - unit: "celsius" -}) -``` - -#### Scientific/Research Types (2) - -##### 26. **Hypothesis** - Scientific theories and propositions -```typescript -await brain.add("String Theory", { - type: "hypothesis", - field: "physics", - status: "unproven" -}) -``` - -##### 27. **Experiment** - Studies and research trials -```typescript -await brain.add("Clinical Trial XYZ", { - type: "experiment", - phase: 3, - participants: 1000 -}) -``` - -#### Legal/Regulatory Types (2) - -##### 28. **Contract** - Legal agreements and terms -```typescript -await brain.add("Service Agreement", { - type: "contract", - duration: "2 years", - value: 100000 -}) -``` - -##### 29. **Regulation** - Laws and compliance requirements -```typescript -await brain.add("GDPR", { - type: "regulation", - jurisdiction: "EU", - category: "data protection" -}) -``` - -#### Technical Infrastructure Types (2) - -##### 30. **Interface** - APIs and protocols -```typescript -await brain.add("REST API", { - type: "interface", - version: "v2", - endpoints: 45 -}) -``` - -##### 31. **Resource** - Infrastructure and compute assets -```typescript -await brain.add("Database Server", { - type: "resource", - capacity: "1TB", - availability: "99.9%" -}) -``` - -### Complete Verb Types (40 Types) - -#### Core Relationship Types (5) - -##### 1. **RelatedTo** - Generic relationship (default) -```typescript -await brain.relate(entityA, entityB, "relatedTo") -``` +### Spatial (2) -##### 2. **Contains** - Containment relationship -```typescript -await brain.relate(folderId, fileId, "contains") -``` +| VerbType | Value | Meaning | +|----------|-------|---------| +| `LocatedAt` | `'locatedAt'` | Spatial location relationship | +| `AdjacentTo` | `'adjacentTo'` | Spatial proximity relationship | -##### 3. **PartOf** - Part-whole relationship -```typescript -await brain.relate(componentId, systemId, "partOf") -``` +### Temporal (3) -##### 4. **LocatedAt** - Spatial relationship -```typescript -await brain.relate(deviceId, locationId, "locatedAt") -``` +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Precedes` | `'precedes'` | Temporal sequence (before) | +| `During` | `'during'` | Temporal containment | +| `OccursAt` | `'occursAt'` | Temporal location | -##### 5. **References** - Citation relationship -```typescript -await brain.relate(paperId, sourceId, "references") -``` +### Causal & Dependency (5) -#### Temporal/Causal Types (5) +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Causes` | `'causes'` | Direct causal relationship | +| `Enables` | `'enables'` | Enablement without direct causation | +| `Prevents` | `'prevents'` | Prevention relationship | +| `DependsOn` | `'dependsOn'` | Dependency relationship | +| `Requires` | `'requires'` | Necessity relationship | + +### Creation & Transformation (5) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Creates` | `'creates'` | Creation relationship | +| `Transforms` | `'transforms'` | Transformation relationship | +| `Becomes` | `'becomes'` | State change relationship | +| `Modifies` | `'modifies'` | Modification relationship | +| `Consumes` | `'consumes'` | Consumption relationship | -##### 6. **Precedes** - Temporal sequence (before) -```typescript -await brain.relate(event1Id, event2Id, "precedes") -``` +### Lifecycle (1) -##### 7. **Succeeds** - Temporal sequence (after) -```typescript -await brain.relate(event2Id, event1Id, "succeeds") -``` +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Destroys` | `'destroys'` | Termination and destruction relationship | + +### Ownership & Attribution (2) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Owns` | `'owns'` | Ownership relationship | +| `AttributedTo` | `'attributedTo'` | Attribution relationship | -##### 8. **Causes** - Causal relationship -```typescript -await brain.relate(actionId, effectId, "causes") -``` +### Property & Quality (2) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `HasQuality` | `'hasQuality'` | Entity to quality attribution | +| `Realizes` | `'realizes'` | Function realization relationship | + +### Effects & Experience (1) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Affects` | `'affects'` | Patient/experiencer relationship | -##### 9. **DependsOn** - Dependency relationship -```typescript -await brain.relate(moduleId, libraryId, "dependsOn") -``` +### Composition (2) -##### 10. **Requires** - Necessity relationship -```typescript -await brain.relate(taskId, resourceId, "requires") -``` - -#### Creation/Transformation Types (5) - -##### 11. **Creates** - Creation relationship -```typescript -await brain.relate(authorId, documentId, "creates") -``` - -##### 12. **Transforms** - Transformation relationship -```typescript -await brain.relate(processId, dataId, "transforms") -``` - -##### 13. **Becomes** - State change relationship -```typescript -await brain.relate(caterpillarId, butterflyId, "becomes") -``` - -##### 14. **Modifies** - Modification relationship -```typescript -await brain.relate(editorId, fileId, "modifies") -``` - -##### 15. **Consumes** - Consumption relationship -```typescript -await brain.relate(processId, resourceId, "consumes") -``` - -#### Ownership/Attribution Types (4) - -##### 16. **Owns** - Ownership relationship -```typescript -await brain.relate(userId, assetId, "owns") -``` - -##### 17. **AttributedTo** - Attribution relationship -```typescript -await brain.relate(quoteId, authorId, "attributedTo") -``` - -##### 18. **CreatedBy** - Creation attribution -```typescript -await brain.relate(productId, teamId, "createdBy") -``` - -##### 19. **BelongsTo** - Belonging relationship -```typescript -await brain.relate(itemId, collectionId, "belongsTo") -``` - -#### Social/Organizational Types (9) - -##### 20. **MemberOf** - Membership relationship -```typescript -await brain.relate(userId, organizationId, "memberOf") -``` - -##### 21. **WorksWith** - Professional relationship -```typescript -await brain.relate(employee1Id, employee2Id, "worksWith") -``` - -##### 22. **FriendOf** - Friendship relationship -```typescript -await brain.relate(user1Id, user2Id, "friendOf") -``` - -##### 23. **Follows** - Following relationship -```typescript -await brain.relate(followerId, influencerId, "follows") -``` - -##### 24. **Likes** - Liking relationship -```typescript -await brain.relate(userId, postId, "likes") -``` - -##### 25. **ReportsTo** - Reporting relationship -```typescript -await brain.relate(employeeId, managerId, "reportsTo") -``` - -##### 26. **Supervises** - Supervisory relationship -```typescript -await brain.relate(managerId, employeeId, "supervises") -``` - -##### 27. **Mentors** - Mentorship relationship -```typescript -await brain.relate(seniorId, juniorId, "mentors") -``` - -##### 28. **Communicates** - Communication relationship -```typescript -await brain.relate(sender, receiver, "communicates") -``` - -#### Descriptive/Functional Types (8) - -##### 29. **Describes** - Descriptive relationship -```typescript -await brain.relate(documentId, conceptId, "describes") -``` - -##### 30. **Defines** - Definition relationship -```typescript -await brain.relate(glossaryId, termId, "defines") -``` - -##### 31. **Categorizes** - Categorization relationship -```typescript -await brain.relate(taxonomyId, itemId, "categorizes") -``` - -##### 32. **Measures** - Measurement relationship -```typescript -await brain.relate(sensorId, metricId, "measures") -``` - -##### 33. **Evaluates** - Evaluation relationship -```typescript -await brain.relate(reviewerId, proposalId, "evaluates") -``` - -##### 34. **Uses** - Utilization relationship -```typescript -await brain.relate(applicationId, libraryId, "uses") -``` - -##### 35. **Implements** - Implementation relationship -```typescript -await brain.relate(classId, interfaceId, "implements") -``` - -##### 36. **Extends** - Extension relationship -```typescript -await brain.relate(childClassId, parentClassId, "extends") -``` - -#### Enhanced Relationships (4) - -##### 37. **Inherits** - Inheritance relationship -```typescript -await brain.relate(childId, parentId, "inherits") -``` - -##### 38. **Conflicts** - Conflict relationship -```typescript -await brain.relate(policy1Id, policy2Id, "conflicts") -``` - -##### 39. **Synchronizes** - Synchronization relationship -```typescript -await brain.relate(service1Id, service2Id, "synchronizes") -``` - -##### 40. **Competes** - Competition relationship -```typescript -await brain.relate(company1Id, company2Id, "competes") -``` +| VerbType | Value | Meaning | +|----------|-------|---------| +| `ComposedOf` | `'composedOf'` | Material composition (distinct from partOf) | +| `Inherits` | `'inherits'` | Inheritance relationship | + +### Social & Organizational (8) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `MemberOf` | `'memberOf'` | Membership relationship | +| `WorksWith` | `'worksWith'` | Professional collaboration | +| `FriendOf` | `'friendOf'` | Friendship relationship | +| `Follows` | `'follows'` | Following/subscription relationship | +| `Likes` | `'likes'` | Liking/favoriting relationship | +| `ReportsTo` | `'reportsTo'` | Hierarchical reporting relationship | +| `Mentors` | `'mentors'` | Mentorship relationship | +| `Communicates` | `'communicates'` | Communication relationship | + +### Descriptive & Functional (8) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Describes` | `'describes'` | Descriptive relationship | +| `Defines` | `'defines'` | Definition relationship | +| `Categorizes` | `'categorizes'` | Categorization relationship | +| `Measures` | `'measures'` | Measurement relationship | +| `Evaluates` | `'evaluates'` | Evaluation relationship | +| `Uses` | `'uses'` | Utilization relationship | +| `Implements` | `'implements'` | Implementation relationship | +| `Extends` | `'extends'` | Extension relationship | + +### Advanced Relationships (5) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `EquivalentTo` | `'equivalentTo'` | Equivalence/identity relationship | +| `Believes` | `'believes'` | Epistemic relationship (cognitive state) | +| `Conflicts` | `'conflicts'` | Conflict relationship | +| `Synchronizes` | `'synchronizes'` | Synchronization relationship | +| `Competes` | `'competes'` | Competition relationship | + +### Modal (6) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `CanCause` | `'canCause'` | Potential causation (possibility) | +| `MustCause` | `'mustCause'` | Necessary causation (necessity) | +| `WouldCauseIf` | `'wouldCauseIf'` | Counterfactual causation | +| `CouldBe` | `'couldBe'` | Possible states | +| `MustBe` | `'mustBe'` | Necessary identity | +| `Counterfactual` | `'counterfactual'` | General counterfactual relationship | + +### Epistemic States (9) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Knows` | `'knows'` | Knowledge (justified true belief) | +| `Doubts` | `'doubts'` | Uncertainty/skepticism | +| `Desires` | `'desires'` | Want/preference | +| `Intends` | `'intends'` | Intentionality | +| `Fears` | `'fears'` | Fear/anxiety | +| `Loves` | `'loves'` | Strong positive emotional attitude | +| `Hates` | `'hates'` | Strong negative emotional attitude | +| `Hopes` | `'hopes'` | Hopeful expectation | +| `Perceives` | `'perceives'` | Sensory perception | + +### Learning & Cognition (1) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Learns` | `'learns'` | Cognitive acquisition and learning | + +### Uncertainty & Probability (4) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `ProbablyCauses` | `'probablyCauses'` | Probabilistic causation | +| `UncertainRelation` | `'uncertainRelation'` | Unknown relationship with confidence bounds | +| `CorrelatesWith` | `'correlatesWith'` | Statistical correlation (not causation) | +| `ApproximatelyEquals` | `'approximatelyEquals'` | Fuzzy equivalence | + +### Scalar Properties (5) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `GreaterThan` | `'greaterThan'` | Scalar comparison | +| `SimilarityDegree` | `'similarityDegree'` | Graded similarity | +| `MoreXThan` | `'moreXThan'` | Comparative property | +| `HasDegree` | `'hasDegree'` | Scalar property assignment | +| `PartiallyHas` | `'partiallyHas'` | Graded possession | + +### Information Theory (2) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Carries` | `'carries'` | Bearer carries content | +| `Encodes` | `'encodes'` | Encoding relationship | + +### Deontic (5) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `ObligatedTo` | `'obligatedTo'` | Moral/legal obligation | +| `PermittedTo` | `'permittedTo'` | Permission/authorization | +| `ProhibitedFrom` | `'prohibitedFrom'` | Prohibition/forbidden | +| `ShouldDo` | `'shouldDo'` | Normative expectation | +| `MustNotDo` | `'mustNotDo'` | Strong prohibition | + +### Context & Perspective (5) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `TrueInContext` | `'trueInContext'` | Context-dependent truth | +| `PerceivedAs` | `'perceivedAs'` | Subjective perception | +| `InterpretedAs` | `'interpretedAs'` | Interpretation relationship | +| `ValidInFrame` | `'validInFrame'` | Frame-dependent validity | +| `TrueFrom` | `'trueFrom'` | Perspective-dependent truth | + +### Advanced Temporal (6) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Overlaps` | `'overlaps'` | Partial temporal overlap | +| `ImmediatelyAfter` | `'immediatelyAfter'` | Direct temporal succession | +| `EventuallyLeadsTo` | `'eventuallyLeadsTo'` | Long-term consequence | +| `SimultaneousWith` | `'simultaneousWith'` | Exact temporal alignment | +| `HasDuration` | `'hasDuration'` | Temporal extent | +| `RecurringWith` | `'recurringWith'` | Cyclic temporal relationship | + +### Advanced Spatial (9) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `ContainsSpatially` | `'containsSpatially'` | Spatial containment | +| `OverlapsSpatially` | `'overlapsSpatially'` | Spatial overlap | +| `Surrounds` | `'surrounds'` | Encirclement | +| `ConnectedTo` | `'connectedTo'` | Topological connection | +| `Above` | `'above'` | Vertical (superior position) | +| `Below` | `'below'` | Vertical (inferior position) | +| `Inside` | `'inside'` | Within containment boundaries | +| `Outside` | `'outside'` | Beyond containment boundaries | +| `Facing` | `'facing'` | Directional orientation | + +### Social Structures (5) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Represents` | `'represents'` | Representative relationship | +| `Embodies` | `'embodies'` | Exemplification or personification | +| `Opposes` | `'opposes'` | Opposition relationship | +| `AlliesWith` | `'alliesWith'` | Alliance relationship | +| `ConformsTo` | `'conformsTo'` | Norm conformity | + +### Measurement (4) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `MeasuredIn` | `'measuredIn'` | Unit relationship | +| `ConvertsTo` | `'convertsTo'` | Unit conversion | +| `HasMagnitude` | `'hasMagnitude'` | Quantitative value | +| `DimensionallyEquals` | `'dimensionallyEquals'` | Dimensional analysis | + +### Change & Persistence (4) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `PersistsThrough` | `'persistsThrough'` | Persistence through change | +| `GainsProperty` | `'gainsProperty'` | Property acquisition | +| `LosesProperty` | `'losesProperty'` | Property loss | +| `RemainsSame` | `'remainsSame'` | Identity through time | + +### Parthood Variations (4) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `FunctionalPartOf` | `'functionalPartOf'` | Functional component | +| `TopologicalPartOf` | `'topologicalPartOf'` | Spatial part | +| `TemporalPartOf` | `'temporalPartOf'` | Temporal slice | +| `ConceptualPartOf` | `'conceptualPartOf'` | Abstract decomposition | + +### Dependency Variations (3) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `RigidlyDependsOn` | `'rigidlyDependsOn'` | Necessary dependency | +| `FunctionallyDependsOn` | `'functionallyDependsOn'` | Operational dependency | +| `HistoricallyDependsOn` | `'historicallyDependsOn'` | Causal history dependency | + +### Meta-Level (4) + +| VerbType | Value | Meaning | +|----------|-------|---------| +| `Endorses` | `'endorses'` | Second-order validation | +| `Contradicts` | `'contradicts'` | Logical contradiction | +| `Supports` | `'supports'` | Evidential support | +| `Supersedes` | `'supersedes'` | Replacement relationship | ## Coverage Completeness Analysis -### Is Anything Missing? +### Is Anything Missing? -While we could add more specific verb types (like "approves", "delegates", "shares"), our current taxonomy is **mathematically complete** for several reasons: +The taxonomy is intentionally bounded. When no type is an exact fit, three escape hatches keep it complete: #### 1. Generic Fallbacks -- **`Custom` noun type**: For any entity that doesn't fit standard types -- **`RelatedTo` verb type**: For any relationship not explicitly defined -- **Unlimited metadata**: Any additional semantics via properties -#### 2. Semantic Flexibility Through Metadata +- **`Custom` noun type**: For any entity that doesn't fit a standard type +- **`RelatedTo` verb type**: For any relationship not explicitly named +- **`subtype` + metadata**: Refine a base type with domain-specific semantics -Instead of adding dozens more verb types, we use metadata for specificity: +#### 2. Semantic Flexibility Through Subtype & Metadata + +Instead of adding ever more verb types, refine a base verb with a `subtype` and structured metadata: ```typescript -// Instead of adding "approves" verb: -await brain.relate(managerId, requestId, "evaluates", { - result: "approved", - timestamp: Date.now() +// "approves" → Evaluates with a result +await brain.relate({ + from: managerId, + to: requestId, + type: VerbType.Evaluates, + subtype: 'approval', + metadata: { result: 'approved', timestamp: Date.now() } }) -// Instead of adding "shares" verb: -await brain.relate(userId, documentId, "communicates", { - action: "shared", - permissions: "read-only" +// "shares" → Communicates with an action +await brain.relate({ + from: userId, + to: documentId, + type: VerbType.Communicates, + subtype: 'share', + metadata: { permissions: 'read-only' } }) -// Instead of adding "delegates" verb: -await brain.relate(managerId, taskId, "creates", { - delegatedTo: employeeId, - authority: "full" +// "delegates" → ParticipatesIn with a role + delegation metadata +await brain.relate({ + from: employeeId, + to: taskId, + type: VerbType.ParticipatesIn, + subtype: 'delegate', + metadata: { delegatedBy: managerId, authority: 'full' } }) ``` #### 3. Edge Cases Are Covered -Even exotic scenarios work with our current types: +Even exotic scenarios fit the standard types: ```typescript // Quantum computing -const qubitId = await brain.add("Qubit-1", { - type: "thing", - subtype: "quantum_bit", - superposition: [0.707, 0.707] +const qubitId = await brain.add({ + data: 'Qubit-1', + type: NounType.Thing, + subtype: 'quantum-bit', + metadata: { superposition: [0.707, 0.707] } }) // Cryptocurrency transactions -const txId = await brain.add("Bitcoin Transfer", { - type: "event", - subtype: "blockchain_transaction", - hash: "1A2B3C..." +const txId = await brain.add({ + data: 'Bitcoin Transfer', + type: NounType.Event, + subtype: 'blockchain-transaction', + metadata: { hash: '1A2B3C...' } }) // AI model training -const modelId = await brain.add("Neural Network", { - type: "process", - subtype: "ml_model", - architecture: "transformer" +const modelId = await brain.add({ + data: 'Neural Network', + type: NounType.Process, + subtype: 'ml-model', + metadata: { architecture: 'transformer' } }) ``` ### The Philosophy: Simplicity Over Specificity -We intentionally keep the type system minimal because: -1. **Fewer types = easier to learn** -2. **Metadata provides infinite extensibility** -3. **Consistent patterns across domains** -4. **Avoids taxonomy explosion** +A bounded type system stays learnable: +1. **A fixed vocabulary is easier to learn** than thousands of bespoke schemas +2. **Subtype + metadata provide open-ended extensibility** +3. **Consistent patterns** carry across domains +4. **No taxonomy explosion** as new use cases appear ## Industry-Specific Coverage Analysis -### Why 42 Nouns + 127 Verbs = Universal Coverage +### Why 42 Nouns + 127 Verbs = Broad Coverage -The combination of **31 noun types** and **40 verb types** creates **1,240 basic combinations**, but with metadata and multi-hop relationships, this expands to **infinite expressiveness**. Here's how it covers every industry: +The combination of **42 noun types** and **127 verb types** yields **5,334 base combinations**, and with subtypes, metadata, and multi-hop relationships that expands to cover essentially any domain. Here's how it maps onto common industries. ### Healthcare & Medical + ```typescript -// Patient records with medical history -const patientId = await brain.add("John Doe", { - type: "person", - subtype: "patient", - mrn: "12345" +const patientId = await brain.add({ + data: 'John Doe', + type: NounType.Person, + subtype: 'patient', + metadata: { mrn: '12345' } }) -const diagnosisId = await brain.add("Type 2 Diabetes", { - type: "state", - subtype: "diagnosis", - icd10: "E11.9" +const diagnosisId = await brain.add({ + data: 'Type 2 Diabetes', + type: NounType.State, + subtype: 'diagnosis', + metadata: { icd10: 'E11.9' } }) -const medicationId = await brain.add("Metformin", { - type: "thing", - subtype: "medication", - dosage: "500mg" +const medicationId = await brain.add({ + data: 'Metformin', + type: NounType.Substance, + subtype: 'medication', + metadata: { dosage: '500mg' } }) +const doctorId = await brain.add({ data: 'Dr. Reyes', type: NounType.Person, subtype: 'physician' }) + // Medical relationships -await brain.relate(patientId, diagnosisId, "diagnoses") -await brain.relate(medicationId, diagnosisId, "treats") -await brain.relate(doctorId, patientId, "treats") +await brain.relate({ from: patientId, to: diagnosisId, type: VerbType.HasQuality, subtype: 'diagnosis' }) +await brain.relate({ from: medicationId, to: diagnosisId, type: VerbType.Affects, subtype: 'treats' }) +await brain.relate({ from: doctorId, to: patientId, type: VerbType.RelatedTo, subtype: 'treats' }) ``` ### Finance & Banking + ```typescript -// Financial instruments and transactions -const accountId = await brain.add("Checking Account", { - type: "thing", - subtype: "account", - balance: 10000 +const accountId = await brain.add({ + data: 'Checking Account', + type: NounType.Thing, + subtype: 'account', + metadata: { balance: 10000 } }) -const transactionId = await brain.add("Wire Transfer", { - type: "event", - subtype: "transaction", - amount: 5000 +const transactionId = await brain.add({ + data: 'Wire Transfer', + type: NounType.Event, + subtype: 'transaction', + metadata: { amount: 5000 } }) -const regulationId = await brain.add("GDPR Compliance", { - type: "concept", - subtype: "regulation" +const regulationId = await brain.add({ + data: 'KYC Requirement', + type: NounType.Regulation, + subtype: 'compliance' }) +const customerId = await brain.add({ data: 'Account Holder', type: NounType.Person, subtype: 'customer' }) + // Financial relationships -await brain.relate(customerId, accountId, "owns") -await brain.relate(transactionId, accountId, "modifies") -await brain.relate(accountId, regulationId, "compliesWith") +await brain.relate({ from: customerId, to: accountId, type: VerbType.Owns }) +await brain.relate({ from: transactionId, to: accountId, type: VerbType.Modifies }) +await brain.relate({ from: accountId, to: regulationId, type: VerbType.ConformsTo }) ``` ### Manufacturing & Supply Chain + ```typescript -// Production and logistics -const factoryId = await brain.add("Plant #3", { - type: "location", - subtype: "facility" +const factoryId = await brain.add({ + data: 'Plant #3', + type: NounType.Location, + subtype: 'facility' }) -const assemblyLineId = await brain.add("Assembly Line A", { - type: "process", - subtype: "production" +const assemblyLineId = await brain.add({ + data: 'Assembly Line A', + type: NounType.Process, + subtype: 'production' }) -const componentId = await brain.add("Circuit Board v2", { - type: "thing", - subtype: "component" +const componentId = await brain.add({ + data: 'Circuit Board v2', + type: NounType.Thing, + subtype: 'component' }) +const productId = await brain.add({ data: 'Controller Unit', type: NounType.Product }) +const supplierId = await brain.add({ data: 'Acme Components', type: NounType.Organization, subtype: 'supplier' }) + // Manufacturing relationships -await brain.relate(assemblyLineId, componentId, "produces") -await brain.relate(componentId, productId, "partOf") -await brain.relate(supplierId, componentId, "supplies") +await brain.relate({ from: assemblyLineId, to: componentId, type: VerbType.Creates }) +await brain.relate({ from: componentId, to: productId, type: VerbType.PartOf }) +await brain.relate({ from: supplierId, to: componentId, type: VerbType.RelatedTo, subtype: 'supplies' }) ``` ### Education & Learning + ```typescript -// Educational content and progress -const courseId = await brain.add("Machine Learning 101", { - type: "collection", - subtype: "course" +const courseId = await brain.add({ + data: 'Machine Learning 101', + type: NounType.Collection, + subtype: 'course' }) -const lessonId = await brain.add("Neural Networks", { - type: "content", - subtype: "lesson" +const lessonId = await brain.add({ + data: 'Neural Networks', + type: NounType.Document, + subtype: 'lesson' }) -const assessmentId = await brain.add("Final Exam", { - type: "event", - subtype: "assessment" +const assessmentId = await brain.add({ + data: 'Final Exam', + type: NounType.Event, + subtype: 'assessment' }) +const studentId = await brain.add({ data: 'Student #88', type: NounType.Person, subtype: 'student' }) + // Educational relationships -await brain.relate(studentId, courseId, "enrolledIn") -await brain.relate(courseId, lessonId, "contains") -await brain.relate(studentId, assessmentId, "completed") +await brain.relate({ from: studentId, to: courseId, type: VerbType.MemberOf, subtype: 'enrolled' }) +await brain.relate({ from: courseId, to: lessonId, type: VerbType.Contains }) +await brain.relate({ from: studentId, to: assessmentId, type: VerbType.ParticipatesIn, subtype: 'completed' }) ``` ### Legal & Compliance + ```typescript -// Legal documents and cases -const contractId = await brain.add("Service Agreement", { - type: "document", - subtype: "contract" +const contractId = await brain.add({ + data: 'Service Agreement', + type: NounType.Contract, + subtype: 'service-agreement' }) -const clauseId = await brain.add("Liability Clause", { - type: "content", - subtype: "clause" +const clauseId = await brain.add({ + data: 'Liability Clause', + type: NounType.Document, + subtype: 'clause' }) -const caseId = await brain.add("Case #2024-1234", { - type: "event", - subtype: "legal_case" +const caseId = await brain.add({ + data: 'Case #2024-1234', + type: NounType.Event, + subtype: 'legal-case' }) +const partyId = await brain.add({ data: 'Counterparty LLC', type: NounType.Organization }) + // Legal relationships -await brain.relate(contractId, clauseId, "contains") -await brain.relate(party1Id, contractId, "signedBy") -await brain.relate(caseId, contractId, "references") +await brain.relate({ from: contractId, to: clauseId, type: VerbType.Contains }) +await brain.relate({ from: partyId, to: contractId, type: VerbType.RelatedTo, subtype: 'signatory' }) +await brain.relate({ from: caseId, to: contractId, type: VerbType.References }) ``` ### Retail & E-commerce + ```typescript -// Products and customer behavior -const productId = await brain.add("iPhone 15", { - type: "product", - sku: "IP15-128-BLK" +const productId = await brain.add({ + data: 'Wireless Earbuds', + type: NounType.Product, + metadata: { sku: 'WE-128-BLK' } }) -const cartId = await brain.add("Shopping Cart", { - type: "collection", - subtype: "cart" +const cartId = await brain.add({ + data: 'Shopping Cart', + type: NounType.Collection, + subtype: 'cart' }) -const promotionId = await brain.add("Black Friday Sale", { - type: "event", - subtype: "promotion" +const promotionId = await brain.add({ + data: 'Holiday Sale', + type: NounType.Event, + subtype: 'promotion' }) +const customerId = await brain.add({ data: 'Customer #5521', type: NounType.Person, subtype: 'customer' }) + // Retail relationships -await brain.relate(customerId, productId, "views") -await brain.relate(cartId, productId, "contains") -await brain.relate(promotionId, productId, "applies") +await brain.relate({ from: customerId, to: productId, type: VerbType.RelatedTo, subtype: 'view' }) +await brain.relate({ from: cartId, to: productId, type: VerbType.Contains }) +await brain.relate({ from: promotionId, to: productId, type: VerbType.Affects, subtype: 'applies' }) ``` ### Real Estate + ```typescript -// Properties and transactions -const propertyId = await brain.add("123 Main St", { - type: "location", - subtype: "property" +const propertyId = await brain.add({ + data: '123 Main St', + type: NounType.Location, + subtype: 'property' }) -const listingId = await brain.add("MLS #789", { - type: "document", - subtype: "listing" +const listingId = await brain.add({ + data: 'MLS #789', + type: NounType.Document, + subtype: 'listing' }) -const inspectionId = await brain.add("Home Inspection", { - type: "event", - subtype: "inspection" +const inspectionId = await brain.add({ + data: 'Home Inspection', + type: NounType.Event, + subtype: 'inspection' }) +const ownerId = await brain.add({ data: 'Property Owner', type: NounType.Person }) + // Real estate relationships -await brain.relate(ownerId, propertyId, "owns") -await brain.relate(listingId, propertyId, "describes") -await brain.relate(inspectionId, propertyId, "evaluates") +await brain.relate({ from: ownerId, to: propertyId, type: VerbType.Owns }) +await brain.relate({ from: listingId, to: propertyId, type: VerbType.Describes }) +await brain.relate({ from: inspectionId, to: propertyId, type: VerbType.Evaluates }) ``` ### Government & Public Sector + ```typescript -// Civic data and services -const citizenId = await brain.add("Citizen #123", { - type: "person", - subtype: "citizen" +const citizenId = await brain.add({ + data: 'Citizen #123', + type: NounType.Person, + subtype: 'citizen' }) -const permitId = await brain.add("Building Permit", { - type: "document", - subtype: "permit" +const permitId = await brain.add({ + data: 'Building Permit', + type: NounType.Document, + subtype: 'permit' }) -const departmentId = await brain.add("Planning Dept", { - type: "organization", - subtype: "government" +const departmentId = await brain.add({ + data: 'Planning Dept', + type: NounType.Organization, + subtype: 'government' }) +const propertyId = await brain.add({ data: '500 Oak Ave', type: NounType.Location, subtype: 'property' }) + // Government relationships -await brain.relate(citizenId, permitId, "requests") -await brain.relate(departmentId, permitId, "issues") -await brain.relate(permitId, propertyId, "authorizes") +await brain.relate({ from: citizenId, to: permitId, type: VerbType.RelatedTo, subtype: 'request' }) +await brain.relate({ from: departmentId, to: permitId, type: VerbType.Creates, subtype: 'issues' }) +await brain.relate({ from: permitId, to: propertyId, type: VerbType.PermittedTo, subtype: 'authorizes' }) ``` -### Why This Covers All Knowledge +### Why This Covers Most Knowledge -#### 1. **Mathematical Completeness** -The noun-verb model forms a **complete graph structure** where: +#### 1. Structural Completeness + +The noun-verb model forms a **graph structure** where: - Any entity can be represented as a noun - Any relationship can be represented as a verb - Complex knowledge emerges from simple combinations -#### 2. **Semantic Completeness** -Every piece of human knowledge falls into one of these categories: +#### 2. Semantic Coverage + +Most information falls into one of these categories: - **Entities** (who, what, where) → Nouns -- **Actions** (how, when, why) → Verbs +- **Actions/relations** (how, when, why) → Verbs - **Attributes** (properties) → Metadata - **Context** (conditions) → Graph structure -#### 3. **Compositional Power** +#### 3. Compositional Power + Simple types combine to represent complex knowledge: + ```typescript -// Complex knowledge from simple building blocks -const researchPaper = await brain.add("AI Ethics Study", { - type: "document" -}) +const researchPaper = await brain.add({ data: 'AI Ethics Study', type: NounType.Document }) +const researcher = await brain.add({ data: 'Dr. Smith', type: NounType.Person }) +const institution = await brain.add({ data: 'MIT', type: NounType.Organization }) +const concept = await brain.add({ data: 'AI Ethics', type: NounType.Concept }) -const researcher = await brain.add("Dr. Smith", { - type: "person" -}) - -const institution = await brain.add("MIT", { - type: "organization" -}) - -const concept = await brain.add("AI Ethics", { - type: "concept" -}) - -// Rich knowledge graph emerges -await brain.relate(researcher, researchPaper, "authors") -await brain.relate(researcher, institution, "affiliated") -await brain.relate(researchPaper, concept, "explores") -await brain.relate(institution, researchPaper, "publishes") +// A rich knowledge graph emerges from a handful of typed edges +await brain.relate({ from: researcher, to: researchPaper, type: VerbType.Creates }) +await brain.relate({ from: researcher, to: institution, type: VerbType.MemberOf }) +await brain.relate({ from: researchPaper, to: concept, type: VerbType.Describes }) +await brain.relate({ from: institution, to: researchPaper, type: VerbType.Creates, subtype: 'publishes' }) ``` -#### 4. **Domain Independence** -The same types work across all domains: +#### 4. Domain Independence + +The same types work across domains: **Science:** ```typescript -await brain.add("H2O", { type: "thing", category: "molecule" }) -await brain.add("Photosynthesis", { type: "process" }) -await brain.relate(moleculeId, processId, "participates") +const moleculeId = await brain.add({ data: 'H2O', type: NounType.Substance, metadata: { category: 'molecule' } }) +const processId = await brain.add({ data: 'Photosynthesis', type: NounType.Process }) +await brain.relate({ from: moleculeId, to: processId, type: VerbType.ParticipatesIn }) ``` **Business:** ```typescript -await brain.add("Q3 Revenue", { type: "metric", value: 10000000 }) -await brain.add("Sales Team", { type: "organization" }) -await brain.relate(teamId, metricId, "achieves") +const metricId = await brain.add({ data: 'Q3 Revenue', type: NounType.Measurement, metadata: { value: 10_000_000 } }) +const teamId = await brain.add({ data: 'Sales Team', type: NounType.Organization }) +await brain.relate({ from: teamId, to: metricId, type: VerbType.RelatedTo, subtype: 'achieves' }) ``` **Social:** ```typescript -await brain.add("John", { type: "person" }) -await brain.add("Community Group", { type: "organization" }) -await brain.relate(personId, groupId, "joins") +const personId = await brain.add({ data: 'John', type: NounType.Person }) +const groupId = await brain.add({ data: 'Community Group', type: NounType.SocialGroup }) +await brain.relate({ from: personId, to: groupId, type: VerbType.MemberOf }) ``` -#### 5. **Temporal Coverage** -Handles all temporal aspects: +#### 5. Temporal Coverage + +Time lives in edge metadata, so past, present, and future all fit: + ```typescript // Past -await brain.relate(personId, companyId, "worked", { - from: "2010", to: "2020" +await brain.relate({ + from: personId, + to: companyId, + type: VerbType.MemberOf, + subtype: 'past-employment', + metadata: { from: '2010', to: '2020' } }) // Present -await brain.relate(personId, projectId, "manages", { - since: "2024-01-01" +await brain.relate({ + from: personId, + to: projectId, + type: VerbType.ParticipatesIn, + subtype: 'manager', + metadata: { since: '2024-01-01' } }) // Future -await brain.relate(eventId, venueId, "scheduled", { - date: "2025-06-15" +await brain.relate({ + from: eventId, + to: venueId, + type: VerbType.LocatedAt, + metadata: { scheduledFor: '2025-06-15' } }) ``` -#### 6. **Hierarchical Representation** -Supports all levels of abstraction: +#### 6. Hierarchical Representation + +Every level of abstraction fits: + ```typescript // Micro level -await brain.add("Electron", { type: "thing", scale: "quantum" }) +await brain.add({ data: 'Electron', type: NounType.Thing, metadata: { scale: 'quantum' } }) // Macro level -await brain.add("Solar System", { type: "place", scale: "astronomical" }) +await brain.add({ data: 'Solar System', type: NounType.Location, metadata: { scale: 'astronomical' } }) // Abstract level -await brain.add("Justice", { type: "concept", domain: "philosophy" }) +await brain.add({ data: 'Justice', type: NounType.Concept, metadata: { domain: 'philosophy' } }) ``` ### Extensibility -While the core types cover all knowledge, you can extend with domain-specific subtypes: +While the core types cover most domains, you extend with `subtype` (and metadata) — never a schema migration: ```typescript -// Extend person for medical domain -await brain.add("Patient #12345", { - type: "person", - subtype: "patient", - medicalRecord: "MR-12345" +// Extend Person for the medical domain +await brain.add({ + data: 'Patient #12345', + type: NounType.Person, + subtype: 'patient', + metadata: { medicalRecord: 'MR-12345' } }) -// Extend document for legal domain -await brain.add("Contract ABC", { - type: "document", - subtype: "contract", - jurisdiction: "California" +// Extend Document for the legal domain +await brain.add({ + data: 'Contract ABC', + type: NounType.Document, + subtype: 'contract', + metadata: { jurisdiction: 'California' } }) -// Custom verb for specific domain -await brain.relate(lawyerId, contractId, "negotiates", { - verbSubtype: "legal-action", - billableHours: 10 +// Extend a verb with a domain-specific subtype + billing metadata +await brain.relate({ + from: lawyerId, + to: contractId, + type: VerbType.ParticipatesIn, + subtype: 'negotiator', + metadata: { billableHours: 10 } }) ``` -### Mathematical Proof of Universal Coverage +### How the Taxonomy Stays Complete -The noun-verb taxonomy achieves **Turing completeness** for knowledge representation: +The noun-verb model is designed to represent any knowledge that can be expressed as entities and relations: -1. **Storage Completeness**: Any data can be stored as nouns -2. **Relational Completeness**: Any relationship can be expressed as verbs -3. **Property Completeness**: Unlimited metadata captures all attributes -4. **Graph Completeness**: Multi-hop traversals express any complexity -5. **Temporal Completeness**: Time metadata handles all temporal aspects -6. **Semantic Completeness**: Vector embeddings capture meaning and similarity +1. **Storage**: Any data can be stored as nouns +2. **Relational**: Any relationship can be expressed as verbs +3. **Property**: Open-ended metadata captures all attributes +4. **Graph**: Multi-hop traversals express arbitrary complexity +5. **Temporal**: Date metadata handles all temporal aspects +6. **Semantic**: Vector embeddings capture meaning and similarity -#### The Infinity Formula +#### The Composition Formula ``` -Expressiveness = (31 nouns × 40 verbs) × ∞ metadata × ∞ graph depth - = 1,240 × ∞ × ∞ - = ∞ (Infinite Expressiveness) +Expressiveness = (42 nouns × 127 verbs) × metadata × graph depth + = 5,334 base combinations × open-ended refinement ``` -This mathematical infinity means Brainy can represent: -- **All Scientific Knowledge**: From quantum physics to molecular biology -- **All Business Data**: From transactions to supply chains -- **All Social Graphs**: From friendships to organizational hierarchies -- **All Historical Records**: From events to archaeological findings -- **All Creative Works**: From art metadata to story relationships -- **All Technical Systems**: From software architecture to network topology -- **All Personal Information**: From memories to preferences -- **Literally ANY Information That Can Exist** +That composition lets Brainy represent: +- **Scientific Knowledge**: From quantum physics to molecular biology +- **Business Data**: From transactions to supply chains +- **Social Graphs**: From friendships to organizational hierarchies +- **Historical Records**: From events to archaeological findings +- **Creative Works**: From media metadata to story relationships +- **Technical Systems**: From software architecture to network topology +- **Personal Information**: From memories to preferences ### Real-World Proof: Unmappable Becomes Mappable Even the most complex scenarios map naturally: ```typescript -// String Theory - 11-dimensional physics -const braneId = await brain.add("D3-Brane", { - type: "concept", - dimensions: 11, - vibrational_modes: ["0,1", "1,0", "2,1"] +// String Theory — high-dimensional physics +const braneId = await brain.add({ + data: 'D3-Brane', + type: NounType.Concept, + metadata: { dimensions: 11, vibrationalModes: ['0,1', '1,0', '2,1'] } }) -// Consciousness - The "hard problem" of philosophy -const qualiaId = await brain.add("Red Qualia", { - type: "concept", - subtype: "phenomenal_experience", - ineffable: true +// Consciousness — the "hard problem" of philosophy +const qualiaId = await brain.add({ + data: 'Red Qualia', + type: NounType.Concept, + subtype: 'phenomenal-experience', + metadata: { ineffable: true } }) -// Time Travel Paradoxes -const futureEvent = await brain.add("Future Effect", { - type: "event", - temporal_position: "future" +// Causal paradoxes +const futureEvent = await brain.add({ + data: 'Future Effect', + type: NounType.Event, + metadata: { temporalPosition: 'future' } }) -const pastCause = await brain.add("Past Cause", { - type: "event", - temporal_position: "past" +const pastCause = await brain.add({ + data: 'Past Cause', + type: NounType.Event, + metadata: { temporalPosition: 'past' } }) -await brain.relate(futureEvent, pastCause, "causes", { - paradox_type: "bootstrap" +await brain.relate({ + from: futureEvent, + to: pastCause, + type: VerbType.Causes, + metadata: { paradoxType: 'bootstrap' } }) ``` -If it exists, thinks, happens, or can be imagined—Brainy can model it. +If it exists, thinks, happens, or can be imagined — Brainy can model it. + +## Migration from Traditional Models + +### From Relational (SQL) + +```typescript +// Instead of JOIN queries: +// SELECT * FROM users JOIN orders ON users.id = orders.user_id + +// Use noun-verb relationships +const userId = await brain.add({ data: 'User', type: NounType.Person, metadata: { email: 'u@example.com' } }) +const orderId = await brain.add({ data: 'Order #1', type: NounType.Event, subtype: 'order' }) +await brain.relate({ from: userId, to: orderId, type: VerbType.Creates, subtype: 'placed' }) + +// Query naturally via the graph +const userOrders = await brain.find({ + type: NounType.Event, + connected: { from: userId, via: VerbType.Creates } +}) +``` + +### From Document (NoSQL) + +```typescript +// Instead of embedded documents: { user: { orders: [...] } } + +// Use explicit relationships +const userId = await brain.add({ data: 'User', type: NounType.Person }) +for (const order of orders) { + const orderId = await brain.add({ data: order.summary, type: NounType.Event, subtype: 'order' }) + await brain.relate({ from: userId, to: orderId, type: VerbType.Creates, subtype: 'placed' }) +} +``` + +### From Graph Databases + +```typescript +// Similar to a graph database, with added benefits: +// 1. Automatic vector embeddings for similarity +// 2. Natural language querying +// 3. Unified with metadata filtering + +// Vector + graph in one query +const results = await brain.find({ query: 'users who bought similar products' }) +``` ## Conclusion -The Noun-Verb taxonomy in Brainy 2.0 provides a natural, flexible, and powerful way to model any domain. By thinking in terms of entities and their relationships, you can build everything from simple data stores to complex knowledge graphs while maintaining code clarity and query simplicity. +The Noun-Verb Taxonomy gives Brainy a natural, flexible, and powerful way to model any domain. By thinking in terms of entities (42 `NounType`s) and their relationships (127 `VerbType`s) — refined with `subtype` and metadata — you can build everything from simple data stores to complex knowledge graphs while keeping code clear and queries simple. ## See Also -- [Triple Intelligence](./triple-intelligence.md) -- [Natural Language Queries](../guides/natural-language.md) -- [API Reference](../api/README.md) \ No newline at end of file +- [Triple Intelligence](/docs/concepts/triple-intelligence) +- [Subtypes & Facets](/docs/guides/subtypes-and-facets) +- [The Find System](/docs/guides/find-system) +- [API Reference](/docs/api/reference) diff --git a/docs/architecture/triple-intelligence.md b/docs/architecture/triple-intelligence.md index 0751444a..fbec56a9 100644 --- a/docs/architecture/triple-intelligence.md +++ b/docs/architecture/triple-intelligence.md @@ -23,29 +23,37 @@ Traditional databases force you to choose between vector search, graph traversal ### Unified Query Structure +`find()` accepts a single `FindParams` object (or a natural-language string). One +object combines all three intelligences: + ```typescript -interface TripleQuery { - // Vector/Semantic search - like?: string | Vector | any - similar?: string | Vector | any - - // Graph/Relationship search +interface FindParams { + // Vector intelligence — semantic similarity + query?: string // Natural-language / semantic query (embedded, matched via HNSW + text index) + vector?: number[] // Pre-computed embedding for direct vector search + + // Metadata intelligence — structured field filters + type?: NounType | NounType[] // Filter by entity type + subtype?: string | string[] // Filter by per-product subtype + where?: Record // Field predicates with bare operators (gte, lt, in, contains, exists…) + + // Graph intelligence — relationship traversal connected?: { - to?: string | string[] - from?: string | string[] - type?: string | string[] - depth?: number + to?: string // Reachable to this entity + from?: string // Reachable from this entity + via?: VerbType | VerbType[] // Relationship type(s) to traverse (alias: type) + depth?: number // Max traversal depth (default: 1) direction?: 'in' | 'out' | 'both' } - - // Field/Attribute search - where?: Record - - // Advanced options - limit?: number - boost?: 'recent' | 'popular' | 'verified' | string - explain?: boolean - threshold?: number + + // Proximity — nearest neighbours of a known entity + near?: { id: string; threshold?: number } + + // Control + limit?: number // Max results (default: 10) + offset?: number // Skip N results + orderBy?: string // Field to sort by (e.g. 'createdAt') + order?: 'asc' | 'desc' // Sort direction } ``` @@ -74,10 +82,10 @@ const results = await brain.find("machine learning concepts") #### Combined Intelligence Query ```typescript const results = await brain.find({ - like: "neural networks", + query: "neural networks", where: { category: "research", - year: { $gte: 2023 } + year: { gte: 2023 } }, connected: { to: "deep-learning-team", @@ -109,8 +117,8 @@ All three search types execute simultaneously: ```typescript // Parallel execution for balanced query const results = await brain.find({ - like: "AI research", // ~1000 potential matches - where: { type: "paper" }, // ~500 potential matches + query: "AI research", // ~1000 potential matches + where: { kind: "paper" }, // ~500 potential matches connected: { to: "stanford" } // ~200 potential matches }) // All three execute in parallel, results fused @@ -126,7 +134,7 @@ Operations chain for maximum efficiency: // Progressive execution for selective query const results = await brain.find({ where: { userId: "user123" }, // Very selective (1-10 matches) - like: "recent posts", // Applied to filtered set + query: "recent posts", // Applied to filtered set limit: 5 }) // Metadata filter first, then vector search on results @@ -166,10 +174,10 @@ const results = await brain.find( ) // Automatically converts to: // { -// like: "AI papers", -// where: { +// query: "AI papers", +// where: { // institution: "Stanford", -// published: { $gte: "2024-01-01" } +// published: { gte: "2024-01-01" } // } // } ``` @@ -188,10 +196,10 @@ The NLP processor identifies query intent: Successful execution plans are cached: ```typescript -// First query: 50ms (plan generation + execution) +// First call parses the natural-language query and builds an execution plan await brain.find("machine learning papers") -// Subsequent similar queries: 10ms (cached plan) +// A structurally similar query reuses that plan, skipping plan generation await brain.find("deep learning papers") ``` @@ -213,50 +221,45 @@ Triple Intelligence leverages all available indexes: ### Explain Mode -Understand how your query was executed: +Diagnose how a query's `where` fields map to the index. Run `brain.explain()` +first whenever `find()` returns surprising or empty results: ```typescript -const results = await brain.find({ - like: "quantum computing", - where: { category: "research" }, - explain: true +const plan = await brain.explain({ + query: "quantum computing", + where: { category: "research" } }) -console.log(results[0].explanation) -// { -// plan: "field-first-progressive", -// timing: { -// fieldFilter: 2, -// vectorSearch: 8, -// fusion: 1 -// }, -// selectivity: { -// field: 0.1, -// vector: 0.3 -// } -// } +console.log(plan.fieldPlan) +// [ +// { field: 'category', path: 'column-store', notes: '...' } +// ] + +console.log(plan.warnings) +// e.g. ['Field "category" has no index entries. find() will return [] silently...'] ``` -### Boosting +### Result Ordering -Apply custom ranking boosts: +Sort results by any stored field with `orderBy` / `order`: ```typescript const results = await brain.find({ - like: "news articles", - boost: 'recent', // Boost recent items - where: { verified: true } + query: "news articles", + where: { verified: true }, + orderBy: 'createdAt', // Newest first + order: 'desc' }) ``` -### Threshold Control +### Similarity Threshold -Set minimum similarity thresholds: +Find the nearest neighbours of a known entity and keep only close matches with +`near`: ```typescript const results = await brain.find({ - like: "exact match needed", - threshold: 0.9, // Only very similar results + near: { id: anchorId, threshold: 0.9 }, // Only results >= 0.9 similarity limit: 10 }) ``` @@ -283,8 +286,8 @@ const results = await brain.find({ ```typescript // Find similar content with constraints const results = await brain.find({ - like: query, - where: { + query: searchText, + where: { status: 'published', language: 'en' } @@ -295,10 +298,10 @@ const results = await brain.find({ ```typescript // Find items related to a specific item const results = await brain.find({ - connected: { + connected: { to: itemId, depth: 2, - type: 'similar' + via: VerbType.RelatedTo }, limit: 20 }) @@ -309,10 +312,11 @@ const results = await brain.find({ // Recent items matching criteria const results = await brain.find({ where: { - timestamp: { $gte: Date.now() - 86400000 } + timestamp: { gte: Date.now() - 86400000 } }, - like: "trending topics", - boost: 'recent' + query: "trending topics", + orderBy: 'timestamp', + order: 'desc' }) ``` diff --git a/docs/guides/quick-start.md b/docs/guides/quick-start.md index 2a689bd3..097c55fe 100644 --- a/docs/guides/quick-start.md +++ b/docs/guides/quick-start.md @@ -60,17 +60,17 @@ const nextId: string = await brain.add({ await brain.relate({ from: nextId, to: reactId, - type: VerbType.BuiltOn + type: VerbType.DependsOn }) ``` ## 5. Query with Triple Intelligence ```typescript -import type { FindResult } from '@soulcraft/brainy' +import type { Result } from '@soulcraft/brainy' // All three search paradigms in one call -const results: FindResult[] = await brain.find({ +const results: Result[] = await brain.find({ query: 'modern frontend frameworks', // Vector similarity search where: { year: { greaterThan: 2015 } }, // Metadata filtering connected: { to: reactId, depth: 2 } // Graph traversal diff --git a/tests/api/error-handling.test.ts b/tests/api/error-handling.test.ts deleted file mode 100644 index b8fd91f9..00000000 --- a/tests/api/error-handling.test.ts +++ /dev/null @@ -1,638 +0,0 @@ -/** - * Error Handling and Recovery Test Suite for Brainy v3.0 - * - * Comprehensive error scenario testing including: - * - Invalid input validation - * - Network failure recovery - * - Resource exhaustion handling - * - Concurrent error scenarios - * - Graceful degradation - * - Error message consistency - */ - -import { describe, it, expect, beforeEach, afterEach } from 'vitest' -import { Brainy } from '../../src/brainy' - -describe('Error Handling and Recovery', () => { - let brainy: Brainy - - beforeEach(async () => { - brainy = new Brainy({ requireSubtype: false, - storage: { type: 'memory' } - }) - await brainy.init() - }) - - afterEach(async () => { - await brainy.close() - }) - - describe('Input Validation Errors', () => { - describe('ADD operation validation', () => { - it('should reject null data', async () => { - await expect( - brainy.add({ - data: null as any, - type: 'document' - }) - ).rejects.toThrow() - }) - - it('should reject undefined data', async () => { - await expect( - brainy.add({ - data: undefined as any, - type: 'document' - }) - ).rejects.toThrow() - }) - - it('should reject empty string data', async () => { - await expect( - brainy.add({ - data: '', - type: 'document' - }) - ).rejects.toThrow(/data.*required|empty/i) - }) - - it('should reject invalid type values', async () => { - await expect( - brainy.add({ - data: 'Valid data', - type: 'invalid-type' as any - }) - ).rejects.toThrow(/invalid.*type|noun.*type/i) - }) - - it('should reject non-string data types', async () => { - const invalidData = [ - 42, - true, - [], - { text: 'object' }, - () => 'function' - ] - - for (const data of invalidData) { - await expect( - brainy.add({ - data: data as any, - type: 'document' - }) - ).rejects.toThrow() - } - }) - - it('should handle extremely large metadata gracefully', async () => { - const hugeMetadata: any = {} - for (let i = 0; i < 10000; i++) { - hugeMetadata[`field${i}`] = `value${i}`.repeat(100) - } - - // Should either succeed or throw meaningful error - try { - const id = await brainy.add({ - data: 'Test with huge metadata', - type: 'document', - metadata: hugeMetadata - }) - expect(id).toBeDefined() - } catch (error: any) { - expect(error.message).toMatch(/size|memory|large/i) - } - }) - }) - - describe('GET operation validation', () => { - it('should return null for non-existent IDs', async () => { - const result = await brainy.get('non-existent-id-12345') - expect(result).toBeNull() - }) - - it('should handle invalid ID formats gracefully', async () => { - const invalidIds = [ - null, - undefined, - '', - ' ', - '../etc/passwd', - '' - ] - - for (const id of invalidIds) { - const result = await brainy.get(id as any) - expect(result).toBeNull() - } - }) - }) - - describe('UPDATE operation validation', () => { - it('should reject updates without ID', async () => { - await expect( - brainy.update({ - id: undefined as any, - metadata: { updated: true } - }) - ).rejects.toThrow(/id.*required/i) - }) - - it('should reject updates to non-existent entities', async () => { - await expect( - brainy.update({ - id: 'non-existent-id', - metadata: { updated: true } - }) - ).rejects.toThrow(/not found|doesn't exist/i) - }) - - it('should handle circular references in metadata', async () => { - const id = await brainy.add({ - data: 'Test entity', - type: 'document' - }) - - const metadata: any = { level: 1 } - metadata.self = metadata // Circular reference - - // Should either handle or throw meaningful error - try { - await brainy.update({ - id, - metadata - }) - // If successful, verify it was handled - const updated = await brainy.get(id) - expect(updated).toBeDefined() - } catch (error: any) { - expect(error.message).toMatch(/circular|cyclic/i) - } - }) - }) - - describe('DELETE operation validation', () => { - it('should handle deletion of non-existent entities', async () => { - // Should not throw, just return void - await expect( - brainy.remove('non-existent-id') - ).resolves.toBeUndefined() - }) - - it('should handle double deletion gracefully', async () => { - const id = await brainy.add({ - data: 'To be deleted', - type: 'document' - }) - - await brainy.remove(id) - // Second delete should not throw - await expect(brainy.remove(id)).resolves.toBeUndefined() - }) - }) - - describe('RELATE operation validation', () => { - it('should reject relationships without source', async () => { - await expect( - brainy.relate({ - from: undefined as any, - to: 'some-id', - type: 'relatedTo' - }) - ).rejects.toThrow(/from.*required/i) - }) - - it('should reject relationships without target', async () => { - await expect( - brainy.relate({ - from: 'some-id', - to: undefined as any, - type: 'relatedTo' - }) - ).rejects.toThrow(/to.*required/i) - }) - - it('should reject invalid relationship types', async () => { - const id1 = await brainy.add({ - data: 'Entity 1', - type: 'thing' - }) - const id2 = await brainy.add({ - data: 'Entity 2', - type: 'thing' - }) - - await expect( - brainy.relate({ - from: id1, - to: id2, - type: 'invalid-verb' as any - }) - ).rejects.toThrow(/invalid.*type|verb.*type/i) - }) - - it('should reject self-relationships by default', async () => { - const id = await brainy.add({ - data: 'Self entity', - type: 'thing' - }) - - // Some systems reject self-relationships - try { - await brainy.relate({ - from: id, - to: id, - type: 'relatedTo' - }) - } catch (error: any) { - expect(error.message).toMatch(/self|same.*entity/i) - } - }) - }) - - describe('FIND operation validation', () => { - it('should handle empty queries gracefully', async () => { - const results = await brainy.find({}) - expect(Array.isArray(results)).toBe(true) - }) - - it('should handle invalid where clauses', async () => { - const results = await brainy.find({ - where: { - 'metadata.field': { $invalidOp: 'value' } as any - } - }) - // Should return empty array or handle gracefully - expect(Array.isArray(results)).toBe(true) - }) - - it('should handle negative limits', async () => { - const results = await brainy.find({ - limit: -10 - }) - expect(Array.isArray(results)).toBe(true) - expect(results.length).toBeGreaterThanOrEqual(0) - }) - - it('should handle huge limits reasonably', async () => { - const results = await brainy.find({ - limit: Number.MAX_SAFE_INTEGER - }) - expect(Array.isArray(results)).toBe(true) - // Should cap at reasonable limit - expect(results.length).toBeLessThanOrEqual(10000) - }) - }) - }) - - describe('Batch Operation Error Handling', () => { - it('should handle partial failures in addMany', async () => { - const items = [ - { data: 'Valid 1', type: 'document' as const }, - { data: '', type: 'document' as const }, // Invalid - { data: 'Valid 2', type: 'document' as const }, - { data: null as any, type: 'document' as const }, // Invalid - { data: 'Valid 3', type: 'document' as const } - ] - - const result = await brainy.addMany({ - items, - continueOnError: true - }) - - expect(result.successful.length).toBeGreaterThanOrEqual(3) - expect(result.failed.length).toBeGreaterThanOrEqual(0) - expect(result.total).toBe(5) - }) - - it('should rollback batch on critical failure', async () => { - const items = Array(100).fill(null).map((_, i) => ({ - data: `Item ${i}`, - type: 'document' as const - })) - - // Inject a failure midway - items[50] = { - data: null as any, - type: 'document' as const - } - - const result = await brainy.addMany({ - items, - continueOnError: false - }) - - // Should either complete all or rollback - if (result.successful.length > 0) { - expect(result.successful.length).toBe(100) - } else { - expect(result.failed.length).toBeGreaterThan(0) - } - }) - }) - - describe('Concurrent Operation Errors', () => { - it('should handle concurrent updates to same entity', async () => { - const id = await brainy.add({ - data: 'Concurrent test', - type: 'document', - metadata: { version: 1 } - }) - - // Attempt concurrent updates - const updates = Array(10).fill(null).map((_, i) => - brainy.update({ - id, - metadata: { version: i + 2 } - }) - ) - - const results = await Promise.allSettled(updates) - - // At least one should succeed - const succeeded = results.filter(r => r.status === 'fulfilled') - expect(succeeded.length).toBeGreaterThanOrEqual(1) - - // Final state should be consistent - const final = await brainy.get(id) - expect(final?.metadata?.version).toBeGreaterThanOrEqual(2) - }) - - it('should handle concurrent deletes gracefully', async () => { - const id = await brainy.add({ - data: 'To be deleted concurrently', - type: 'document' - }) - - // Attempt concurrent deletes - const deletes = Array(5).fill(null).map(() => brainy.remove(id)) - - // Should not throw - await expect(Promise.all(deletes)).resolves.toBeDefined() - - // Entity should be deleted - const result = await brainy.get(id) - expect(result).toBeNull() - }) - }) - - describe('Resource Exhaustion', () => { - it('should handle memory pressure gracefully', async () => { - const largeData = 'x'.repeat(1024 * 1024) // 1MB - let successCount = 0 - let errorCount = 0 - - // Try to add many large entities - for (let i = 0; i < 100; i++) { - try { - await brainy.add({ - data: largeData + ` Entity ${i}`, - type: 'document', - metadata: { index: i, size: largeData.length } - }) - successCount++ - } catch (error) { - errorCount++ - // Should get meaningful error - expect(error).toBeDefined() - } - } - - // Should handle at least some operations - expect(successCount).toBeGreaterThan(0) - }) - - it('should handle rapid-fire operations', async () => { - const operations = 1000 - const promises: Promise[] = [] - - for (let i = 0; i < operations; i++) { - const op = i % 4 - switch (op) { - case 0: - promises.push( - brainy.add({ - data: `Rapid ${i}`, - type: 'document' - }).catch(() => null) - ) - break - case 1: - promises.push(brainy.get(`rapid-${i}`).catch(() => null)) - break - case 2: - promises.push( - brainy.update({ - id: `rapid-${i}`, - metadata: { updated: true } - }).catch(() => null) - ) - break - case 3: - promises.push(brainy.remove(`rapid-${i}`).catch(() => null)) - break - } - } - - const results = await Promise.allSettled(promises) - - // Most should complete - const completed = results.filter(r => r.status === 'fulfilled') - expect(completed.length).toBeGreaterThan(operations * 0.5) - }) - }) - - describe('Error Message Quality', () => { - it('should provide clear error messages for common mistakes', async () => { - // Missing required field - try { - await brainy.add({} as any) - } catch (error: any) { - expect(error.message).toBeDefined() - expect(error.message.length).toBeGreaterThan(10) - // Should mention what's missing - expect(error.message).toMatch(/data|required/i) - } - - // Invalid type - try { - await brainy.add({ - data: 'Valid data', - type: 'not-a-valid-type' as any - }) - } catch (error: any) { - expect(error.message).toMatch(/type|invalid|noun/i) - } - - // Entity not found - try { - await brainy.update({ - id: 'definitely-does-not-exist', - metadata: { test: true } - }) - } catch (error: any) { - expect(error.message).toMatch(/not found|doesn't exist|does not exist/i) - } - }) - - it('should not leak sensitive information in errors', async () => { - try { - await brainy.add({ - data: 'Test', - type: 'document', - metadata: { - password: 'secret123', - apiKey: 'sk-1234567890' - } - }) - - await brainy.update({ - id: 'non-existent', - metadata: { password: 'should-not-appear' } - }) - } catch (error: any) { - // Error message should not contain sensitive data - expect(error.message).not.toMatch(/secret123|sk-1234567890|should-not-appear/i) - } - }) - }) - - describe('Recovery Mechanisms', () => { - it('should recover from transient failures', async () => { - let attemptCount = 0 - const maxAttempts = 3 - - async function retryableAdd(data: string): Promise { - attemptCount++ - if (attemptCount < maxAttempts) { - throw new Error('Transient failure') - } - return brainy.add({ - data, - type: 'document' - }) - } - - // Should eventually succeed with retry - let result: string | null = null - for (let i = 0; i < maxAttempts; i++) { - try { - result = await retryableAdd('Test with retry') - break - } catch (error) { - if (i === maxAttempts - 1) throw error - } - } - - expect(result).toBeDefined() - expect(attemptCount).toBe(maxAttempts) - }) - - it('should maintain consistency after errors', async () => { - // Add some valid data - const validIds: string[] = [] - for (let i = 0; i < 5; i++) { - const id = await brainy.add({ - data: `Valid entity ${i}`, - type: 'document' - }) - validIds.push(id) - } - - // Cause some errors - try { - await brainy.add({ data: null as any, type: 'document' }) - } catch {} - - try { - await brainy.update({ id: 'non-existent', metadata: {} }) - } catch {} - - try { - await brainy.relate({ - from: 'non-existent-1', - to: 'non-existent-2', - type: 'relatedTo' - }) - } catch {} - - // Valid data should still be intact - for (const id of validIds) { - const entity = await brainy.get(id) - expect(entity).toBeDefined() - expect(entity?.type).toBe('document') - } - - // Should still be able to add new data - const newId = await brainy.add({ - data: 'Added after errors', - type: 'document' - }) - expect(newId).toBeDefined() - }) - }) - - describe('Edge Cases', () => { - it('should handle operations on just-deleted entities', async () => { - const id = await brainy.add({ - data: 'About to be deleted', - type: 'document' - }) - - await brainy.remove(id) - - // Operations on deleted entity should handle gracefully - const getResult = await brainy.get(id) - expect(getResult).toBeNull() - - await expect( - brainy.update({ - id, - metadata: { attempt: 'update-after-delete' } - }) - ).rejects.toThrow(/not found/i) - - // Should not throw for delete - await expect(brainy.remove(id)).resolves.toBeUndefined() - }) - - it('should handle rapid create-delete cycles', async () => { - const cycles = 50 - - for (let i = 0; i < cycles; i++) { - const id = await brainy.add({ - data: `Cycle ${i}`, - type: 'document' - }) - - // Immediately delete - await brainy.remove(id) - - // Verify it's gone - const result = await brainy.get(id) - expect(result).toBeNull() - } - }) - - it('should handle maximum field lengths', async () => { - const maxFieldLength = 1024 * 1024 // 1MB - const longString = 'x'.repeat(maxFieldLength) - - try { - const id = await brainy.add({ - data: longString, - type: 'document', - metadata: { - description: longString.substring(0, 1000) - } - }) - - // If successful, should be retrievable - const entity = await brainy.get(id) - expect(entity).toBeDefined() - } catch (error: any) { - // If it fails, should have meaningful error - expect(error.message).toMatch(/size|large|limit/i) - } - }) - }) -}) \ No newline at end of file diff --git a/tests/critical-error-handling.test.ts b/tests/critical-error-handling.test.ts deleted file mode 100644 index d19ae3c6..00000000 --- a/tests/critical-error-handling.test.ts +++ /dev/null @@ -1,441 +0,0 @@ -import { describe, it, expect, beforeAll, afterAll } from 'vitest' -import { Brainy } from '../src/brainy' - -describe('CRITICAL: Error Handling and Edge Cases', () => { - let brainy: Brainy - - beforeAll(async () => { - brainy = new Brainy({ requireSubtype: false, - storage: { type: 'memory' } - }) - await brainy.init() - }) - - afterAll(async () => { - await brainy.close() - }) - - describe('Invalid Input Handling', () => { - it('should handle null and undefined gracefully', async () => { - await expect(brainy.add({ - data: null as any, - type: 'document' - })).rejects.toThrow() - - await expect(brainy.add({ - data: undefined as any, - type: 'document' - })).rejects.toThrow() - - const result = await brainy.get(null as any) - expect(result).toBeNull() - }) - - it('should validate noun types', async () => { - await expect(brainy.add({ - data: { content: 'test' }, - type: 'invalid-type' as any - })).rejects.toThrow() - }) - - it('should validate verb types in relationships', async () => { - await brainy.add({ id: 'node1', data: { name: 'Node 1' }, type: 'entity' }) - await brainy.add({ id: 'node2', data: { name: 'Node 2' }, type: 'entity' }) - - await expect(brainy.relate({ - from: 'node1', - to: 'node2', - type: 'invalid-verb' as any - })).rejects.toThrow() - }) - - it('should handle extremely long strings', async () => { - const longString = 'x'.repeat(1000000) - - const id = await brainy.add({ - data: { content: longString }, - type: 'document' - }) - - expect(id).toBeDefined() - - const retrieved = await brainy.get(id) - expect(retrieved).toBeDefined() - }) - - it('should handle circular references in data', async () => { - const circular: any = { name: 'test' } - circular.self = circular - - await expect(brainy.add({ - data: circular, - type: 'entity' - })).rejects.toThrow() - }) - - it('should handle special characters in IDs', async () => { - const specialIds = [ - 'id with spaces', - 'id/with/slashes', - 'id\\with\\backslashes', - 'id:with:colons', - 'id|with|pipes', - 'id"with"quotes', - 'idbrackets' - ] - - for (const id of specialIds) { - const result = await brainy.add({ - id, - data: { content: `Test ${id}` }, - type: 'document' - }) - - expect(result).toBe(id) - - const retrieved = await brainy.get(id) - expect(retrieved).toBeDefined() - } - }) - }) - - describe('Concurrent Operation Safety', () => { - it('should handle concurrent writes to same ID', async () => { - const id = 'concurrent-test' - - const promises = Array.from({ length: 10 }, (_, i) => - brainy.add({ - id, - data: { content: `Version ${i}` }, - type: 'document' - }) - ) - - await Promise.all(promises) - - const final = await brainy.get(id) - expect(final).toBeDefined() - }) - - it('should handle concurrent updates', async () => { - const id = await brainy.add({ - data: { counter: 0 }, - type: 'entity' - }) - - const updates = Array.from({ length: 10 }, (_, i) => - brainy.update({ - id, - data: { counter: i } - }) - ) - - await Promise.all(updates) - - const final = await brainy.get(id) - expect(final).toBeDefined() - }) - - it('should handle concurrent deletes', async () => { - const ids = await Promise.all( - Array.from({ length: 10 }, (_, i) => - brainy.add({ - data: { index: i }, - type: 'entity' - }) - ) - ) - - const deletes = ids.map(id => brainy.remove(id)) - await Promise.all(deletes) - - for (const id of ids) { - const result = await brainy.get(id) - expect(result).toBeNull() - } - }) - }) - - describe('Memory and Resource Management', () => { - it('should not leak memory on failed operations', async () => { - const memBefore = process.memoryUsage().heapUsed - - for (let i = 0; i < 1000; i++) { - try { - await brainy.add({ - data: null as any, - type: 'invalid' as any - }) - } catch { - // Expected to fail - } - } - - global.gc?.() - const memAfter = process.memoryUsage().heapUsed - const leak = memAfter - memBefore - - expect(leak).toBeLessThan(10 * 1024 * 1024) // Less than 10MB - }) - - it('should handle out of memory scenarios gracefully', async () => { - const hugeArray = Array.from({ length: 100000 }, (_, i) => ({ - id: `huge-${i}`, - data: { - content: 'x'.repeat(10000), - metadata: Array.from({ length: 100 }, () => Math.random()) - }, - type: 'document' as const - })) - - try { - await brainy.addMany({ items: hugeArray }) - } catch (error: any) { - expect(error).toBeDefined() - } - }) - }) - - describe('Query Edge Cases', () => { - it('should handle empty search queries', async () => { - const results = await brainy.find('') - expect(Array.isArray(results)).toBe(true) - }) - - it('should handle queries with only special characters', async () => { - const specialQueries = [ - '!!!', - '???', - '...', - '---', - '___', - '@#$%^&*()', - '{}[]|\\', - '<>?/~`' - ] - - for (const query of specialQueries) { - const results = await brainy.find(query) - expect(Array.isArray(results)).toBe(true) - } - }) - - it('should handle metadata filters with undefined values', async () => { - await brainy.add({ - data: { name: 'Test', value: undefined }, - type: 'entity' - }) - - const results = await brainy.find({ - where: { value: undefined as any } - }) - - expect(Array.isArray(results)).toBe(true) - }) - - it('should handle complex nested metadata queries', async () => { - await brainy.add({ - data: { - nested: { - deep: { - deeper: { - value: 42 - } - } - } - }, - type: 'entity' - }) - - const results = await brainy.find({ - where: { - 'nested.deep.deeper.value': 42 - } - }) - - expect(results.length).toBeGreaterThan(0) - }) - }) - - describe('Relationship Error Cases', () => { - it('should handle relationships to non-existent nodes', async () => { - await expect(brainy.relate({ - from: 'non-existent-1', - to: 'non-existent-2', - type: 'relatedTo' - })).rejects.toThrow() - }) - - it('should handle self-referential relationships', async () => { - const id = await brainy.add({ - data: { name: 'Self' }, - type: 'entity' - }) - - const relId = await brainy.relate({ - from: id, - to: id, - type: 'relatedTo' - }) - - expect(relId).toBeDefined() - - const relations = await brainy.related({ from: id }) - expect(relations.length).toBeGreaterThan(0) - }) - - it('should handle duplicate relationships', async () => { - const id1 = await brainy.add({ data: { name: 'A' }, type: 'entity' }) - const id2 = await brainy.add({ data: { name: 'B' }, type: 'entity' }) - - const rel1 = await brainy.relate({ - from: id1, - to: id2, - type: 'relatedTo' - }) - - const rel2 = await brainy.relate({ - from: id1, - to: id2, - type: 'relatedTo' - }) - - expect(rel1).not.toBe(rel2) - }) - }) - - describe('Data Validation', () => { - it('should validate vector dimensions', async () => { - const wrongDimVector = new Array(100).fill(0) - - await brainy.add({ - data: { test: 'first' }, - type: 'entity' - }) - - await expect(brainy.add({ - vector: wrongDimVector, - type: 'entity' - })).rejects.toThrow() - }) - - it('should handle NaN and Infinity in vectors', async () => { - const badVector = [NaN, Infinity, -Infinity, 0.5] - - await expect(brainy.add({ - vector: badVector as any, - type: 'entity' - })).rejects.toThrow() - }) - - it('should validate metadata field names', async () => { - const invalidMetadata = { - '$invalid': 'value', - 'also.invalid': 'value', - 'normal': 'value' - } - - const id = await brainy.add({ - data: invalidMetadata, - type: 'entity' - }) - - const retrieved = await brainy.get(id) - expect(retrieved).toBeDefined() - }) - }) - - describe('Recovery and Cleanup', () => { - it('should recover from partial batch failures', async () => { - const items = [ - { id: 'valid1', data: { content: 'Valid 1' }, type: 'document' as const }, - { id: 'invalid', data: null as any, type: 'invalid' as any }, - { id: 'valid2', data: { content: 'Valid 2' }, type: 'document' as const } - ] - - const result = await brainy.addMany({ items }) - - expect(result.successful).toBeGreaterThan(0) - expect(result.failed).toBeGreaterThan(0) - expect(result.errors.length).toBeGreaterThan(0) - }) - - it('should clean up after failed operations', async () => { - const memBefore = process.memoryUsage().heapUsed - - for (let i = 0; i < 100; i++) { - try { - await brainy.add({ - id: `cleanup-${i}`, - data: { content: 'test' }, - type: 'entity' - }) - - await brainy.remove(`cleanup-${i}`) - } catch { - // Ignore errors - } - } - - global.gc?.() - const memAfter = process.memoryUsage().heapUsed - const diff = memAfter - memBefore - - expect(diff).toBeLessThan(5 * 1024 * 1024) // Less than 5MB growth - }) - }) - - describe('Boundary Conditions', () => { - it('should handle zero-length arrays and strings', async () => { - const id = await brainy.add({ - data: { - emptyString: '', - emptyArray: [], - zero: 0, - false: false, - null: null - }, - type: 'entity' - }) - - const retrieved = await brainy.get(id) - expect(retrieved?.data.emptyString).toBe('') - expect(retrieved?.data.emptyArray).toEqual([]) - }) - - it('should handle maximum safe integers', async () => { - const id = await brainy.add({ - data: { - maxSafe: Number.MAX_SAFE_INTEGER, - minSafe: Number.MIN_SAFE_INTEGER, - maxValue: Number.MAX_VALUE, - minValue: Number.MIN_VALUE - }, - type: 'entity' - }) - - const retrieved = await brainy.get(id) - expect(retrieved?.data.maxSafe).toBe(Number.MAX_SAFE_INTEGER) - }) - - it('should handle Unicode and emoji correctly', async () => { - const unicodeData = { - emoji: '😀🎉🚀❤️', - chinese: '你好世界', - arabic: 'مرحبا بالعالم', - hebrew: 'שלום עולם', - special: '™®©℠', - zalgo: 'ẗ̸̢̼͉͈́̏͊̈ë̵̺́͊s̴̱̈́ẗ̸͎̆' - } - - const id = await brainy.add({ - data: unicodeData, - type: 'entity' - }) - - const retrieved = await brainy.get(id) - expect(retrieved?.data.emoji).toBe(unicodeData.emoji) - expect(retrieved?.data.chinese).toBe(unicodeData.chinese) - }) - }) -}) \ No newline at end of file