docs(8.0): correct public docs to the real 8.0 API + honest perf claims
The GA-readiness audit found the public docs had drifted from the shipped surface and presented uncited performance numbers as measured fact. - quick-start: `FindResult`→`Result`, `VerbType.BuiltOn`→`DependsOn` (the canonical getting-started example now compiles). - noun-verb-taxonomy: rewrote every sample off removed/fictional APIs (`augment`/`connectModel`/`getVerbs`/two-arg `add`/`like`/`$gte`) onto the real single-object `add`/`find`/`relate`/`related`; replaced the stale 31-noun/40-verb catalogs with accurate, complete tables (42 nouns, 127 verbs). - triple-intelligence: `like:`→`query:`, dollar-operators→bare operators, and several other fictional keys swept to the real `FindParams`. - FIND_SYSTEM / PERFORMANCE / index-architecture / BATCHING: replaced fabricated, mutually-inconsistent latency tables and uncited speedup multipliers with Big-O characterizations, qualitative mechanism descriptions, and the one genuinely-measured benchmark (graph O(1) neighbor lookup), per the evidence-based-claims rule. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
3d116190f3
commit
40d2cd5419
9 changed files with 1209 additions and 2283 deletions
|
|
@ -22,7 +22,7 @@ Brainy's `find()` method is the most advanced query system in any vector databas
|
|||
### 1. Vector Intelligence (HNSW Index)
|
||||
- **Purpose**: Semantic similarity search using embeddings
|
||||
- **Algorithm**: Hierarchical Navigable Small World (HNSW)
|
||||
- **Performance**: O(log n) search, ~1.8ms typical
|
||||
- **Performance**: O(log n) search
|
||||
- **Data Structure**: Multi-layer graph with 16 connections per node
|
||||
- **Use Cases**: "Find similar documents", "Content like this"
|
||||
|
||||
|
|
@ -36,14 +36,14 @@ Brainy's `find()` method is the most advanced query system in any vector databas
|
|||
### 3. Metadata Intelligence (Incremental Indices)
|
||||
- **Purpose**: Fast filtering on structured data
|
||||
- **Algorithm**: HashMap for exact matches, Sorted arrays for ranges
|
||||
- **Performance**: O(1) exact, O(log n) ranges, <1ms typical
|
||||
- **Performance**: O(1) exact, O(log n) ranges
|
||||
- **Data Structure**: `Map<field:value, Set<id>>` + sorted value arrays
|
||||
- **Use Cases**: "Documents from 2023", "Status equals active"
|
||||
|
||||
### 4. Graph Intelligence (Adjacency Maps)
|
||||
- **Purpose**: Relationship traversal and connection analysis
|
||||
- **Algorithm**: Pure O(1) neighbor lookups via Map operations
|
||||
- **Performance**: O(1) per hop, ~0.1ms typical
|
||||
- **Performance**: O(1) per hop (measured <1 ms per neighbor lookup, validated up to 1M relationships — `tests/performance/graph-scale-performance.test.ts:238`)
|
||||
- **Data Structure**: `Map<sourceId, Set<targetId>>`
|
||||
- **Use Cases**: "Papers connected to MIT", "Authors who collaborated"
|
||||
|
||||
|
|
@ -373,36 +373,40 @@ return results.slice(offset, offset + limit)
|
|||
|
||||
### Query Performance by Type
|
||||
|
||||
| Query Type | Index Used | Performance | Example |
|
||||
|------------|------------|-------------|---------|
|
||||
| **Semantic Search** | HNSW Vector | O(log n), ~1.8ms | `"AI research papers"` |
|
||||
| **Exact Metadata** | HashMap | O(1), ~0.8ms | `{status: "published"}` |
|
||||
| **Range Metadata** | Sorted Array | O(log n), ~0.6ms | `{year: {greaterThan: 2020}}` |
|
||||
| **Graph Traversal** | Adjacency Map | O(1), ~0.1ms | `{connected: {to: "mit"}}` |
|
||||
| **Type Detection** | Pre-embedded Types | O(t), ~0.3ms | `"documents"` → `NounType.Document` |
|
||||
| **Field Matching** | Field Embeddings | O(f), ~0.1ms | `"by"` → `"author"` |
|
||||
| **Combined Query** | All Indices | O(log n), ~1.8ms | NLP + filters + graph |
|
||||
| Query Type | Index Used | Complexity | Example |
|
||||
|------------|------------|------------|---------|
|
||||
| **Semantic Search** | HNSW Vector | O(log n) | `"AI research papers"` |
|
||||
| **Exact Metadata** | HashMap | O(1) | `{status: "published"}` |
|
||||
| **Range Metadata** | Sorted Array | O(log n) | `{year: {greaterThan: 2020}}` |
|
||||
| **Graph Traversal** | Adjacency Map | O(1) | `{connected: {to: "mit"}}` |
|
||||
| **Type Detection** | Pre-embedded Types | O(t) | `"documents"` → `NounType.Document` |
|
||||
| **Field Matching** | Field Embeddings | O(f) | `"by"` → `"author"` |
|
||||
| **Combined Query** | All Indices | O(log n) | NLP + filters + graph |
|
||||
|
||||
Where:
|
||||
- n = number of entities in database
|
||||
- t = number of types (169 total: 42 noun + 127 verb)
|
||||
- f = number of fields for detected entity type (typically 5-15)
|
||||
|
||||
Absolute latencies depend on hardware, embedding model, and storage backend; the columns above describe how each stage scales. The graph stage is the one path with a committed scale benchmark (see [Scalability](#scalability)).
|
||||
|
||||
### Scalability
|
||||
|
||||
| Database Size | Vector Search | Metadata Filter | Graph Query | Combined |
|
||||
|---------------|---------------|-----------------|-------------|----------|
|
||||
| **1K entities** | 0.8ms | 0.3ms | 0.05ms | 1.1ms |
|
||||
| **10K entities** | 1.2ms | 0.5ms | 0.08ms | 1.5ms |
|
||||
| **100K entities** | 1.8ms | 0.8ms | 0.1ms | 2.1ms |
|
||||
| **1M entities** | 2.5ms | 1.2ms | 0.1ms | 2.8ms |
|
||||
| **10M entities** | 3.8ms | 1.8ms | 0.1ms | 4.2ms |
|
||||
The cost of each query stage is governed by its algorithmic complexity, not a fixed millisecond figure — absolute latency depends on hardware, embedding model, and storage backend. Only the graph adjacency index carries a committed scale assertion:
|
||||
|
||||
| Query stage | Complexity | Scaling behavior |
|
||||
|-------------|------------|------------------|
|
||||
| Metadata filter (exact) | O(1) | Constant — independent of dataset size |
|
||||
| Metadata filter (range) | O(log n) + O(k) | Sub-linear; k = matching results |
|
||||
| Vector search (HNSW) | O(log n) | Degrades gracefully via hierarchical layers |
|
||||
| Graph hop | O(1) | Measured <1 ms per neighbor lookup, validated up to 1M relationships (`tests/performance/graph-scale-performance.test.ts:238`) |
|
||||
| Combined query | O(log n) | Bounded by the vector stage; metadata and graph stages stay O(1)/O(log n) |
|
||||
|
||||
**Key Performance Notes:**
|
||||
- Graph queries stay O(1) regardless of scale
|
||||
- Metadata ranges scale as O(log n), not O(n)
|
||||
- Vector search degrades gracefully due to HNSW
|
||||
- Type-aware NLP adds minimal overhead (~0.4ms)
|
||||
- Type-aware NLP adds minimal overhead (single embedding pass, no full scan)
|
||||
|
||||
## Example Query Flows
|
||||
|
||||
|
|
@ -455,7 +459,7 @@ await brain.find({
|
|||
},
|
||||
limit: 10
|
||||
})
|
||||
// Total performance: ~1.2ms for 100K entities
|
||||
// Executes as O(1) type filter + O(1)/O(log n) metadata + O(1) graph lookup — no full scan
|
||||
```
|
||||
|
||||
## Filter Syntax Reference
|
||||
|
|
@ -1001,7 +1005,7 @@ const results = await brain.find({
|
|||
})
|
||||
```
|
||||
|
||||
**Performance**: O(log n) vector search + O(log n) metadata filters + O(1) graph traversal = ~2-3ms total.
|
||||
**Performance**: O(log n) vector search + O(log n) metadata filters + O(1) graph traversal — bounded by the vector stage.
|
||||
|
||||
### Excluding Soft-Deleted Entities
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue