soulcraft/brainy
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

docs(8.0): correct public docs to the real 8.0 API + honest perf claims

Browse source 
The GA-readiness audit found the public docs had drifted from the shipped
surface and presented uncited performance numbers as measured fact.

- quick-start: `FindResult`→`Result`, `VerbType.BuiltOn`→`DependsOn` (the
  canonical getting-started example now compiles).
- noun-verb-taxonomy: rewrote every sample off removed/fictional APIs
  (`augment`/`connectModel`/`getVerbs`/two-arg `add`/`like`/`$gte`) onto the
  real single-object `add`/`find`/`relate`/`related`; replaced the stale
  31-noun/40-verb catalogs with accurate, complete tables (42 nouns, 127 verbs).
- triple-intelligence: `like:`→`query:`, dollar-operators→bare operators, and
  several other fictional keys swept to the real `FindParams`.
- FIND_SYSTEM / PERFORMANCE / index-architecture / BATCHING: replaced
  fabricated, mutually-inconsistent latency tables and uncited speedup
  multipliers with Big-O characterizations, qualitative mechanism descriptions,
  and the one genuinely-measured benchmark (graph O(1) neighbor lookup), per the
  evidence-based-claims rule.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
David Snelling 2026-06-29 10:03:02 -07:00
parent 3d116190f3
commit 40d2cd5419
9 changed files with 1209 additions and 2283 deletions
Download patch file Download diff file Expand all files Collapse all files

52
docs/BATCHING.md
View file

@ -78,10 +78,10 @@ for (const [id, metadata] of metadataMap) {
- ✅ Direct O(1) path construction from ID (no type lookup needed!)
- ✅ Sharding preservation (all paths include `{shard}/{id}`)
- ✅ Write-cache coherent (read-after-write consistency)
- ✅ 40x faster than v5.x type-first architecture
- ✅ O(1) path construction — eliminates the per-entity type search of the old type-first layout
**Performance:**
- ~1ms per 100 entities (consistent, no cache misses!)
- Constant-time path construction per ID — no type-cache misses
- Filesystem: parallel reads bounded by IOPS
- No type search delays — every ID maps directly to storage path
@ -121,7 +121,7 @@ for (const [sourceId, verbs] of results) {
- Bulk export of relationship data
**Performance:**
- Memory storage: <10ms for 1000 relationships
- Memory storage: single in-memory pass over the metadata index
- Filesystem storage: parallel reads through the metadata index
---
@ -171,7 +171,7 @@ entities/verbs/{SHARD}/{ID}/metadata.json
**Direct O(1) Path Construction:**
```typescript
// Every ID maps directly to exactly ONE path - 40x faster!
// Every ID maps directly to exactly ONE path - O(1), no type search
const id = 'abc-123'
const shard = getShardIdFromUuid(id) // → 'ab' (first 2 hex chars)
const path = `entities/nouns/${shard}/${id}/metadata.json`
@ -183,9 +183,9 @@ const path = `entities/nouns/${shard}/${id}/metadata.json`
```
**Benefits:**
- **40x faster** path lookups (eliminates 42-type sequential search)
- **O(1)** path lookups (eliminates the 42-type sequential search the old type-first layout required)
- **Simpler code** - removed 500+ lines of type cache complexity
- **Scalable** - works at billion-scale without type tracking overhead
- **Scalable** - works at large scale without type tracking overhead
---
@ -220,28 +220,22 @@ await db.release()
---
## Performance Benchmarks
## Why Batching Is Faster
### VFS Operations (12 Files)
Batching's advantage is structural, not a fixed multiplier (the actual speedup
depends on storage backend, IOPS, and batch size):
| Storage | Before optimization | After optimization | Improvement |
|---------|---------------------|--------------------|-------------|
| **Memory** | 150ms | 50ms | **67% faster** |
| **Filesystem** | ~500ms | ~80ms | **84% faster** |
- **N+1 elimination** — N sequential reads collapse into a single parallel pass
(`Promise.all` over the batch).
- **O(1) path construction** — every ID maps directly to one storage path, with
no per-type cache lookup.
- **One metadata round-trip** — relationship batches fetch all sources' metadata
in a single pass instead of one query per source.
### Entity Batch Retrieval (100 Entities)
| Storage | Individual Gets | Batch Get | Improvement |
|---------|-----------------|-----------|-------------|
| **Memory** | 180ms | 15ms | **92% faster** |
| **Filesystem** | ~1.2s | ~120ms | **90% faster** |
### Throughput (Entities/Second)
| Storage | Individual | Batch | Improvement |
|---------|------------|-------|-------------|
| **Memory** | 556 ent/s | 6667 ent/s | **12x** |
| **Filesystem** | ~80 ent/s | ~800 ent/s | **10x** |
The integration test `tests/integration/storage-batch-operations.test.ts`
exercises batch vs. individual reads and asserts that batch retrieval is not
slower than the per-entity loop for large batches; it does not pin a specific
multiplier, since that is hardware- and IOPS-dependent.
---
@ -306,7 +300,7 @@ const results = await brain.batchGet(ids)
const entities = Array.from(results.values())
```
**Performance Gain:** 10-20x faster on filesystem storage.
**Performance Gain:** Replaces N sequential reads with a single batched pass — no fixed multiplier, it scales with storage IOPS.
---
@ -332,7 +326,7 @@ for (const verbs of results.values()) {
}
```
**Performance Gain:** 5-10x faster due to batched metadata fetches.
**Performance Gain:** One batched metadata fetch instead of one query per source entity.
---
@ -450,8 +444,8 @@ return await Promise.all(resolvedPaths.map(path => this.read(path)))
- `storage.getVerbsBySourceBatch(sourceIds, verbType?)` - Batch relationship queries
**Performance Improvements:**
- VFS operations: 90%+ faster than the naive per-entity loop
- Entity retrieval: 10-20x throughput improvement
- VFS operations: single batched pass instead of N sequential reads
- Entity retrieval: N+1 reads collapsed into one batched pass
- Zero N+1 query patterns
**Compatibility:**

48
docs/FIND_SYSTEM.md
View file

@ -22,7 +22,7 @@ Brainy's `find()` method is the most advanced query system in any vector databas
### 1. Vector Intelligence (HNSW Index)
- **Purpose**: Semantic similarity search using embeddings
- **Algorithm**: Hierarchical Navigable Small World (HNSW)
- **Performance**: O(log n) search, ~1.8ms typical
- **Performance**: O(log n) search
- **Data Structure**: Multi-layer graph with 16 connections per node
- **Use Cases**: "Find similar documents", "Content like this"
@ -36,14 +36,14 @@ Brainy's `find()` method is the most advanced query system in any vector databas
### 3. Metadata Intelligence (Incremental Indices)
- **Purpose**: Fast filtering on structured data
- **Algorithm**: HashMap for exact matches, Sorted arrays for ranges
- **Performance**: O(1) exact, O(log n) ranges, <1ms typical
- **Performance**: O(1) exact, O(log n) ranges
- **Data Structure**: `Map<field:value, Set<id>>` + sorted value arrays
- **Use Cases**: "Documents from 2023", "Status equals active"
### 4. Graph Intelligence (Adjacency Maps)
- **Purpose**: Relationship traversal and connection analysis
- **Algorithm**: Pure O(1) neighbor lookups via Map operations
- **Performance**: O(1) per hop, ~0.1ms typical
- **Performance**: O(1) per hop (measured <1 ms per neighbor lookup, validated up to 1M relationships — `tests/performance/graph-scale-performance.test.ts:238`)
- **Data Structure**: `Map<sourceId, Set<targetId>>`
- **Use Cases**: "Papers connected to MIT", "Authors who collaborated"
@ -373,36 +373,40 @@ return results.slice(offset, offset + limit)
### Query Performance by Type
| Query Type | Index Used | Performance | Example |
|------------|------------|-------------|---------|
| **Semantic Search** | HNSW Vector | O(log n), ~1.8ms | `"AI research papers"` |
| **Exact Metadata** | HashMap | O(1), ~0.8ms | `{status: "published"}` |
| **Range Metadata** | Sorted Array | O(log n), ~0.6ms | `{year: {greaterThan: 2020}}` |
| **Graph Traversal** | Adjacency Map | O(1), ~0.1ms | `{connected: {to: "mit"}}` |
| **Type Detection** | Pre-embedded Types | O(t), ~0.3ms | `"documents"` → `NounType.Document` |
| **Field Matching** | Field Embeddings | O(f), ~0.1ms | `"by"` → `"author"` |
| **Combined Query** | All Indices | O(log n), ~1.8ms | NLP + filters + graph |
| Query Type | Index Used | Complexity | Example |
|------------|------------|------------|---------|
| **Semantic Search** | HNSW Vector | O(log n) | `"AI research papers"` |
| **Exact Metadata** | HashMap | O(1) | `{status: "published"}` |
| **Range Metadata** | Sorted Array | O(log n) | `{year: {greaterThan: 2020}}` |
| **Graph Traversal** | Adjacency Map | O(1) | `{connected: {to: "mit"}}` |
| **Type Detection** | Pre-embedded Types | O(t) | `"documents"` → `NounType.Document` |
| **Field Matching** | Field Embeddings | O(f) | `"by"` → `"author"` |
| **Combined Query** | All Indices | O(log n) | NLP + filters + graph |
Where:
- n = number of entities in database
- t = number of types (169 total: 42 noun + 127 verb)
- f = number of fields for detected entity type (typically 5-15)
Absolute latencies depend on hardware, embedding model, and storage backend; the columns above describe how each stage scales. The graph stage is the one path with a committed scale benchmark (see [Scalability](#scalability)).
### Scalability
| Database Size | Vector Search | Metadata Filter | Graph Query | Combined |
|---------------|---------------|-----------------|-------------|----------|
| **1K entities** | 0.8ms | 0.3ms | 0.05ms | 1.1ms |
| **10K entities** | 1.2ms | 0.5ms | 0.08ms | 1.5ms |
| **100K entities** | 1.8ms | 0.8ms | 0.1ms | 2.1ms |
| **1M entities** | 2.5ms | 1.2ms | 0.1ms | 2.8ms |
| **10M entities** | 3.8ms | 1.8ms | 0.1ms | 4.2ms |
The cost of each query stage is governed by its algorithmic complexity, not a fixed millisecond figure — absolute latency depends on hardware, embedding model, and storage backend. Only the graph adjacency index carries a committed scale assertion:
| Query stage | Complexity | Scaling behavior |
|-------------|------------|------------------|
| Metadata filter (exact) | O(1) | Constant — independent of dataset size |
| Metadata filter (range) | O(log n) + O(k) | Sub-linear; k = matching results |
| Vector search (HNSW) | O(log n) | Degrades gracefully via hierarchical layers |
| Graph hop | O(1) | Measured <1 ms per neighbor lookup, validated up to 1M relationships (`tests/performance/graph-scale-performance.test.ts:238`) |
| Combined query | O(log n) | Bounded by the vector stage; metadata and graph stages stay O(1)/O(log n) |
**Key Performance Notes:**
- Graph queries stay O(1) regardless of scale
- Metadata ranges scale as O(log n), not O(n)
- Vector search degrades gracefully due to HNSW
- Type-aware NLP adds minimal overhead (~0.4ms)
- Type-aware NLP adds minimal overhead (single embedding pass, no full scan)
## Example Query Flows
@ -455,7 +459,7 @@ await brain.find({
},
limit: 10
})
// Total performance: ~1.2ms for 100K entities
// Executes as O(1) type filter + O(1)/O(log n) metadata + O(1) graph lookup — no full scan
```
## Filter Syntax Reference
@ -1001,7 +1005,7 @@ const results = await brain.find({
})
```
**Performance**: O(log n) vector search + O(log n) metadata filters + O(1) graph traversal = ~2-3ms total.
**Performance**: O(log n) vector search + O(log n) metadata filters + O(1) graph traversal — bounded by the vector stage.
### Excluding Soft-Deleted Entities

77
docs/PERFORMANCE.md
View file

@ -2,11 +2,11 @@
## Performance Characteristics
Brainy achieves industry-leading performance through carefully optimized data structures and algorithms. All performance claims are verified through actual benchmarks on production code.
Brainy achieves high performance through carefully optimized data structures and algorithms. The tables below describe each component by its **algorithmic complexity** — the durable, defensible guarantee. The example latencies are figures from a single 100-item run on one machine (see [Benchmarks](#benchmarks)); they are illustrative, not a committed benchmark, and vary with hardware, embedding model, and storage backend. The one component with a committed scale assertion is the graph adjacency index (`tests/performance/graph-scale-performance.test.ts:238`).
### Core Performance Summary
| Component | Operation | Time Complexity | Measured Performance | Data Structure |
| Component | Operation | Time Complexity | Example latency (100-item run)\* | Data Structure |
|-----------|-----------|-----------------|---------------------|----------------|
| **Metadata Index** | Exact match | **O(1)** | 0.8ms | `Map<string, Set<string>>` |
| **Metadata Index** | Range query | **O(log n) + O(k)** | 0.6ms | Sorted array + binary search |
@ -17,6 +17,8 @@ Brainy achieves industry-leading performance through carefully optimized data st
| **Type Detection** | Noun/Verb matching | **O(t)** | 0.3ms | Pre-embedded type vectors |
| **Triple Intelligence** | Combined query | **O(1) to O(log n)** | 1.8ms | Parallel execution |
\* Illustrative single-run figures at 100 items on one machine — not a committed benchmark. Only the graph index carries an asserted scale bound (measured <1 ms per neighbor lookup up to 1M relationships, `tests/performance/graph-scale-performance.test.ts:238`).
Where:
- `n` = number of items in index
- `k` = number of results returned
@ -26,25 +28,32 @@ Where:
### brain.get() Metadata-Only Optimization
✨ **Massive Performance Improvement**: `brain.get()` is now **76-81% faster** by default!
`brain.get()` returns **metadata only by default**, skipping the 384-dimensional
embedding — the bulk of an entity's payload. Callers that need the vector opt in
with `{ includeVectors: true }`.
| Operation | Before | After | Speedup | Use Case |
|-----------|------------------|-----------------|---------|----------|
| **brain.get() (metadata-only)** | 43ms, 6KB | **10ms, 300 bytes** | **76-81%** | VFS, existence checks, metadata |
| **brain.get({ includeVectors: true })** | 43ms, 6KB | 43ms, 6KB | 0% | Similarity calculations |
| **VFS readFile()** | 53ms | **~13ms** | **75%** | File operations |
| **VFS readdir(100 files)** | 5.3s | **~1.3s** | **75%** | Directory listings |
| Operation | Default (metadata-only) | With `includeVectors: true` | Use Case |
|-----------|-------------------------|-----------------------------|----------|
| **brain.get()** | Skips vector load | Loads full vector | VFS, existence checks, metadata |
| **VFS readFile() / readdir()** | Inherits metadata-only path | n/a | File operations, directory listings |
**Key Innovation**: Lazy vector loading - only load 384-dimensional embeddings when explicitly needed.
**Key Innovation**: Lazy vector loading — only load the 384-dimensional embedding when explicitly needed.
The integration test `tests/integration/metadata-only-comprehensive.test.ts:306`
asserts metadata-only `get()` is faster than the full-entity `get()`
(`metadataTime < fullTime`). The *magnitude* of the speedup is
environment-dependent (the percentage assertion in
`tests/integration/vfs-performance-v5.11.1.test.ts` is intentionally skipped on
CI for that reason), so no fixed percentage is quoted here.
**Why this matters**:
- **94% of brain.get() calls** don't need vectors (VFS, admin tools, import utilities, data APIs)
- **Vector data is 95% of entity size** (6KB vectors vs 300 bytes metadata)
- **Zero code changes** for most applications - automatic speedup!
- Most `brain.get()` calls don't need vectors (VFS, admin tools, import utilities, data APIs)
- The embedding dominates an entity's serialized size, so skipping it is the largest win
- **Zero code changes** for most applications — automatic by default
**When to use what**:
```typescript
// DEFAULT: Metadata-only (76-81% faster) - use for:
// DEFAULT: Metadata-only (skips the vector load) - use for:
const entity = await brain.get(id)
// - VFS operations (readFile, stat, readdir)
// - Existence checks: if (await brain.get(id)) ...
@ -252,28 +261,34 @@ const results = await Promise.all(searchPromises)
## Benchmarks
### Real-world Performance Test (100 items)
### Illustrative Single Run (100 items, one machine)
Example output from a single 100-item run — illustrative only, not a committed
benchmark; absolute numbers vary by hardware. The values feed the
[Core Performance Summary](#core-performance-summary) example-latency column.
```
📊 Metadata exact match: 0.818ms (50 items matched)
📊 Metadata range query: 0.631ms (40 items in range)
🔗 Graph neighbor lookup: 0.092ms (2 connections)
🎯 Vector k-NN search: 1.773ms (10 nearest neighbors)
🧠 NLP query parsing: 8.906ms (full natural language)
⚡ Triple Intelligence: 1.830ms (combined query)
Metadata exact match: 0.818ms (50 items matched)
Metadata range query: 0.631ms (40 items in range)
Graph neighbor lookup: 0.092ms (2 connections)
Vector k-NN search: 1.773ms (10 nearest neighbors)
NLP query parsing: 8.906ms (full natural language)
Triple Intelligence: 1.830ms (combined query)
```
### Scaling Characteristics
| Items | Metadata O(1) | Range O(log n) | Graph O(1) | Vector O(log n) |
|-------|---------------|----------------|------------|-----------------|
| 100 | 0.8ms | 0.6ms | 0.09ms | 1.8ms |
| 1,000 | 0.8ms | 0.9ms | 0.09ms | 2.5ms |
| 10,000 | 0.8ms | 1.2ms | 0.09ms | 3.2ms |
| 100,000 | 0.8ms | 1.5ms | 0.09ms | 4.1ms |
| 1,000,000 | 0.8ms | 1.8ms | 0.09ms | 5.0ms |
Each stage scales by its algorithmic complexity, not a fixed millisecond figure
— absolute latency depends on hardware, embedding model, and storage backend.
Only the graph adjacency index carries a committed scale assertion:
*Note: O(1) operations maintain constant time regardless of scale*
| Query stage | Complexity | Scaling behavior |
|-------------|------------|------------------|
| Metadata filter (exact) | O(1) | Constant — independent of dataset size |
| Metadata filter (range) | O(log n) + O(k) | Sub-linear; k = matching results |
| Vector search (HNSW) | O(log n) | Degrades gracefully via hierarchical layers |
| Graph hop | O(1) | Measured <1 ms per neighbor lookup, validated up to 1M relationships (`tests/performance/graph-scale-performance.test.ts:238`) |
| Combined query | O(log n) | Bounded by the vector stage; metadata and graph stages stay O(1)/O(log n) |
## Comparison with Other Systems
@ -470,8 +485,8 @@ For off-site replication, snapshot `path` from your scheduler (`gsutil rsync`, `
- **Auto-configuration system** with environment detection
- **Zero-config operation** with intelligent defaults
- **Auto-flush and auto-optimize** in indices
- **Sub-2ms response times** for complex queries
- **Low-latency Triple Intelligence queries** (O(log n) vector + O(1) metadata/graph)
## Conclusion
Brainy delivers on its promise of **production-ready Triple Intelligence** with measured, verified performance characteristics. All listed features are fully implemented, tested, and benchmarked. No stubs, no mocks, no theoretical claims - just real, working code with measured performance.
Brainy delivers on its promise of **production-ready Triple Intelligence** with documented algorithmic-complexity guarantees and a committed graph-scale benchmark (`tests/performance/graph-scale-performance.test.ts`). All listed features are fully implemented and tested. No stubs, no mocks — just real, working code with characterized performance.

17
docs/architecture/index-architecture.md
View file

@ -183,7 +183,7 @@ async getIdsForMultipleFields(pairs: [{field, value}, ...]): Promise<string[]> {
- Memory usage: **90% reduction** (17.17 MB → 2.01 MB for 100K entities)
- Hardware acceleration: SIMD instructions make bitmap operations nearly free
**Benchmark Results** (1,000 queries on various dataset sizes):
**Benchmark Results** — example output from a single run of `tests/performance/roaring-bitmap-benchmark.ts` (1,000 queries per size, one machine; absolute times vary by hardware, the relative speedup and memory savings are the durable signal):
| Dataset Size | Operation | Set Time | Roaring Time | Speedup | Memory Savings |
|--------------|-----------|----------|--------------|---------|----------------|
| 10,000 entities | 3-field intersection | 3.74ms | 1.14ms | **3.3x faster** | 90% |
@ -842,14 +842,15 @@ Where:
### Scalability
All indexes scale gracefully:
All indexes scale gracefully. The cost of each stage is governed by its algorithmic complexity, not a fixed millisecond figure — absolute latency depends on hardware, embedding model, and storage backend. Only the graph adjacency index carries a committed scale assertion:
| Database Size | Metadata Filter | Vector Search | Graph Hop | Combined Query |
|---------------|----------------|---------------|-----------|----------------|
| **1K entities** | 0.3ms | 0.8ms | 0.05ms | 1.1ms |
| **10K entities** | 0.5ms | 1.2ms | 0.08ms | 1.5ms |
| **100K entities** | 0.8ms | 1.8ms | 0.1ms | 2.1ms |
| **1M entities** | 1.2ms | 2.5ms | 0.1ms | 2.8ms |
| Query stage | Complexity | Scaling behavior |
|-------------|------------|------------------|
| Metadata filter (exact) | O(1) | Constant — independent of dataset size |
| Metadata filter (range) | O(log n) + O(k) | Sub-linear; k = matching results |
| Vector search (HNSW) | O(log n) | Degrades gracefully via hierarchical layers |
| Graph hop | O(1) | Measured <1 ms per neighbor lookup, validated up to 1M relationships (`tests/performance/graph-scale-performance.test.ts:238`) |
| Combined query | O(log n) | Bounded by the vector stage; metadata and graph stages stay O(1)/O(log n) |
**Key observations**:
- Graph queries stay O(1) regardless of scale

2083
docs/architecture/noun-verb-taxonomy.md
View file

File diff suppressed because it is too large Load diff

130
docs/architecture/triple-intelligence.md
View file

@ -23,29 +23,37 @@ Traditional databases force you to choose between vector search, graph traversal
### Unified Query Structure
`find()` accepts a single `FindParams` object (or a natural-language string). One
object combines all three intelligences:
```typescript
interface TripleQuery {
// Vector/Semantic search
like?: string | Vector | any
similar?: string | Vector | any
// Graph/Relationship search
interface FindParams {
// Vector intelligence — semantic similarity
query?: string // Natural-language / semantic query (embedded, matched via HNSW + text index)
vector?: number[] // Pre-computed embedding for direct vector search
// Metadata intelligence — structured field filters
type?: NounType | NounType[] // Filter by entity type
subtype?: string | string[] // Filter by per-product subtype
where?: Record<string, any> // Field predicates with bare operators (gte, lt, in, contains, exists…)
// Graph intelligence — relationship traversal
connected?: {
to?: string | string[]
from?: string | string[]
type?: string | string[]
depth?: number
to?: string // Reachable to this entity
from?: string // Reachable from this entity
via?: VerbType | VerbType[] // Relationship type(s) to traverse (alias: type)
depth?: number // Max traversal depth (default: 1)
direction?: 'in' | 'out' | 'both'
}
// Field/Attribute search
where?: Record<string, any>
// Advanced options
limit?: number
boost?: 'recent' | 'popular' | 'verified' | string
explain?: boolean
threshold?: number
// Proximity — nearest neighbours of a known entity
near?: { id: string; threshold?: number }
// Control
limit?: number // Max results (default: 10)
offset?: number // Skip N results
orderBy?: string // Field to sort by (e.g. 'createdAt')
order?: 'asc' | 'desc' // Sort direction
}
```
@ -74,10 +82,10 @@ const results = await brain.find("machine learning concepts")
#### Combined Intelligence Query
```typescript
const results = await brain.find({
like: "neural networks",
query: "neural networks",
where: {
category: "research",
year: { $gte: 2023 }
year: { gte: 2023 }
},
connected: {
to: "deep-learning-team",
@ -109,8 +117,8 @@ All three search types execute simultaneously:
```typescript
// Parallel execution for balanced query
const results = await brain.find({
like: "AI research", // ~1000 potential matches
where: { type: "paper" }, // ~500 potential matches
query: "AI research", // ~1000 potential matches
where: { kind: "paper" }, // ~500 potential matches
connected: { to: "stanford" } // ~200 potential matches
})
// All three execute in parallel, results fused
@ -126,7 +134,7 @@ Operations chain for maximum efficiency:
// Progressive execution for selective query
const results = await brain.find({
where: { userId: "user123" }, // Very selective (1-10 matches)
like: "recent posts", // Applied to filtered set
query: "recent posts", // Applied to filtered set
limit: 5
})
// Metadata filter first, then vector search on results
@ -166,10 +174,10 @@ const results = await brain.find(
)
// Automatically converts to:
// {
// like: "AI papers",
// where: {
// query: "AI papers",
// where: {
// institution: "Stanford",
// published: { $gte: "2024-01-01" }
// published: { gte: "2024-01-01" }
// }
// }
```
@ -188,10 +196,10 @@ The NLP processor identifies query intent:
Successful execution plans are cached:
```typescript
// First query: 50ms (plan generation + execution)
// First call parses the natural-language query and builds an execution plan
await brain.find("machine learning papers")
// Subsequent similar queries: 10ms (cached plan)
// A structurally similar query reuses that plan, skipping plan generation
await brain.find("deep learning papers")
```
@ -213,50 +221,45 @@ Triple Intelligence leverages all available indexes:
### Explain Mode
Understand how your query was executed:
Diagnose how a query's `where` fields map to the index. Run `brain.explain()`
first whenever `find()` returns surprising or empty results:
```typescript
const results = await brain.find({
like: "quantum computing",
where: { category: "research" },
explain: true
const plan = await brain.explain({
query: "quantum computing",
where: { category: "research" }
})
console.log(results[0].explanation)
// {
// plan: "field-first-progressive",
// timing: {
// fieldFilter: 2,
// vectorSearch: 8,
// fusion: 1
// },
// selectivity: {
// field: 0.1,
// vector: 0.3
// }
// }
console.log(plan.fieldPlan)
// [
// { field: 'category', path: 'column-store', notes: '...' }
// ]
console.log(plan.warnings)
// e.g. ['Field "category" has no index entries. find() will return [] silently...']
```
### Boosting
### Result Ordering
Apply custom ranking boosts:
Sort results by any stored field with `orderBy` / `order`:
```typescript
const results = await brain.find({
like: "news articles",
boost: 'recent', // Boost recent items
where: { verified: true }
query: "news articles",
where: { verified: true },
orderBy: 'createdAt', // Newest first
order: 'desc'
})
```
### Threshold Control
### Similarity Threshold
Set minimum similarity thresholds:
Find the nearest neighbours of a known entity and keep only close matches with
`near`:
```typescript
const results = await brain.find({
like: "exact match needed",
threshold: 0.9, // Only very similar results
near: { id: anchorId, threshold: 0.9 }, // Only results >= 0.9 similarity
limit: 10
})
```
@ -283,8 +286,8 @@ const results = await brain.find({
```typescript
// Find similar content with constraints
const results = await brain.find({
like: query,
where: {
query: searchText,
where: {
status: 'published',
language: 'en'
}
@ -295,10 +298,10 @@ const results = await brain.find({
```typescript
// Find items related to a specific item
const results = await brain.find({
connected: {
connected: {
to: itemId,
depth: 2,
type: 'similar'
via: VerbType.RelatedTo
},
limit: 20
})
@ -309,10 +312,11 @@ const results = await brain.find({
// Recent items matching criteria
const results = await brain.find({
where: {
timestamp: { $gte: Date.now() - 86400000 }
timestamp: { gte: Date.now() - 86400000 }
},
like: "trending topics",
boost: 'recent'
query: "trending topics",
orderBy: 'timestamp',
order: 'desc'
})
```

6
docs/guides/quick-start.md
View file

@ -60,17 +60,17 @@ const nextId: string = await brain.add({
await brain.relate({
from: nextId,
to: reactId,
type: VerbType.BuiltOn
type: VerbType.DependsOn
})
```
## 5. Query with Triple Intelligence
```typescript
import type { FindResult } from '@soulcraft/brainy'
import type { Result } from '@soulcraft/brainy'
// All three search paradigms in one call
const results: FindResult[] = await brain.find({
const results: Result[] = await brain.find({
query: 'modern frontend frameworks', // Vector similarity search
where: { year: { greaterThan: 2015 } }, // Metadata filtering
connected: { to: reactId, depth: 2 } // Graph traversal