docs(8.0): correct public docs to the real 8.0 API + honest perf claims
The GA-readiness audit found the public docs had drifted from the shipped surface and presented uncited performance numbers as measured fact. - quick-start: `FindResult`→`Result`, `VerbType.BuiltOn`→`DependsOn` (the canonical getting-started example now compiles). - noun-verb-taxonomy: rewrote every sample off removed/fictional APIs (`augment`/`connectModel`/`getVerbs`/two-arg `add`/`like`/`$gte`) onto the real single-object `add`/`find`/`relate`/`related`; replaced the stale 31-noun/40-verb catalogs with accurate, complete tables (42 nouns, 127 verbs). - triple-intelligence: `like:`→`query:`, dollar-operators→bare operators, and several other fictional keys swept to the real `FindParams`. - FIND_SYSTEM / PERFORMANCE / index-architecture / BATCHING: replaced fabricated, mutually-inconsistent latency tables and uncited speedup multipliers with Big-O characterizations, qualitative mechanism descriptions, and the one genuinely-measured benchmark (graph O(1) neighbor lookup), per the evidence-based-claims rule. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
3d116190f3
commit
40d2cd5419
9 changed files with 1209 additions and 2283 deletions
|
|
@ -2,11 +2,11 @@
|
|||
|
||||
## Performance Characteristics
|
||||
|
||||
Brainy achieves industry-leading performance through carefully optimized data structures and algorithms. All performance claims are verified through actual benchmarks on production code.
|
||||
Brainy achieves high performance through carefully optimized data structures and algorithms. The tables below describe each component by its **algorithmic complexity** — the durable, defensible guarantee. The example latencies are figures from a single 100-item run on one machine (see [Benchmarks](#benchmarks)); they are illustrative, not a committed benchmark, and vary with hardware, embedding model, and storage backend. The one component with a committed scale assertion is the graph adjacency index (`tests/performance/graph-scale-performance.test.ts:238`).
|
||||
|
||||
### Core Performance Summary
|
||||
|
||||
| Component | Operation | Time Complexity | Measured Performance | Data Structure |
|
||||
| Component | Operation | Time Complexity | Example latency (100-item run)\* | Data Structure |
|
||||
|-----------|-----------|-----------------|---------------------|----------------|
|
||||
| **Metadata Index** | Exact match | **O(1)** | 0.8ms | `Map<string, Set<string>>` |
|
||||
| **Metadata Index** | Range query | **O(log n) + O(k)** | 0.6ms | Sorted array + binary search |
|
||||
|
|
@ -17,6 +17,8 @@ Brainy achieves industry-leading performance through carefully optimized data st
|
|||
| **Type Detection** | Noun/Verb matching | **O(t)** | 0.3ms | Pre-embedded type vectors |
|
||||
| **Triple Intelligence** | Combined query | **O(1) to O(log n)** | 1.8ms | Parallel execution |
|
||||
|
||||
\* Illustrative single-run figures at 100 items on one machine — not a committed benchmark. Only the graph index carries an asserted scale bound (measured <1 ms per neighbor lookup up to 1M relationships, `tests/performance/graph-scale-performance.test.ts:238`).
|
||||
|
||||
Where:
|
||||
- `n` = number of items in index
|
||||
- `k` = number of results returned
|
||||
|
|
@ -26,25 +28,32 @@ Where:
|
|||
|
||||
### brain.get() Metadata-Only Optimization
|
||||
|
||||
✨ **Massive Performance Improvement**: `brain.get()` is now **76-81% faster** by default!
|
||||
`brain.get()` returns **metadata only by default**, skipping the 384-dimensional
|
||||
embedding — the bulk of an entity's payload. Callers that need the vector opt in
|
||||
with `{ includeVectors: true }`.
|
||||
|
||||
| Operation | Before | After | Speedup | Use Case |
|
||||
|-----------|------------------|-----------------|---------|----------|
|
||||
| **brain.get() (metadata-only)** | 43ms, 6KB | **10ms, 300 bytes** | **76-81%** | VFS, existence checks, metadata |
|
||||
| **brain.get({ includeVectors: true })** | 43ms, 6KB | 43ms, 6KB | 0% | Similarity calculations |
|
||||
| **VFS readFile()** | 53ms | **~13ms** | **75%** | File operations |
|
||||
| **VFS readdir(100 files)** | 5.3s | **~1.3s** | **75%** | Directory listings |
|
||||
| Operation | Default (metadata-only) | With `includeVectors: true` | Use Case |
|
||||
|-----------|-------------------------|-----------------------------|----------|
|
||||
| **brain.get()** | Skips vector load | Loads full vector | VFS, existence checks, metadata |
|
||||
| **VFS readFile() / readdir()** | Inherits metadata-only path | n/a | File operations, directory listings |
|
||||
|
||||
**Key Innovation**: Lazy vector loading - only load 384-dimensional embeddings when explicitly needed.
|
||||
**Key Innovation**: Lazy vector loading — only load the 384-dimensional embedding when explicitly needed.
|
||||
|
||||
The integration test `tests/integration/metadata-only-comprehensive.test.ts:306`
|
||||
asserts metadata-only `get()` is faster than the full-entity `get()`
|
||||
(`metadataTime < fullTime`). The *magnitude* of the speedup is
|
||||
environment-dependent (the percentage assertion in
|
||||
`tests/integration/vfs-performance-v5.11.1.test.ts` is intentionally skipped on
|
||||
CI for that reason), so no fixed percentage is quoted here.
|
||||
|
||||
**Why this matters**:
|
||||
- **94% of brain.get() calls** don't need vectors (VFS, admin tools, import utilities, data APIs)
|
||||
- **Vector data is 95% of entity size** (6KB vectors vs 300 bytes metadata)
|
||||
- **Zero code changes** for most applications - automatic speedup!
|
||||
- Most `brain.get()` calls don't need vectors (VFS, admin tools, import utilities, data APIs)
|
||||
- The embedding dominates an entity's serialized size, so skipping it is the largest win
|
||||
- **Zero code changes** for most applications — automatic by default
|
||||
|
||||
**When to use what**:
|
||||
```typescript
|
||||
// DEFAULT: Metadata-only (76-81% faster) - use for:
|
||||
// DEFAULT: Metadata-only (skips the vector load) - use for:
|
||||
const entity = await brain.get(id)
|
||||
// - VFS operations (readFile, stat, readdir)
|
||||
// - Existence checks: if (await brain.get(id)) ...
|
||||
|
|
@ -252,28 +261,34 @@ const results = await Promise.all(searchPromises)
|
|||
|
||||
## Benchmarks
|
||||
|
||||
### Real-world Performance Test (100 items)
|
||||
### Illustrative Single Run (100 items, one machine)
|
||||
|
||||
Example output from a single 100-item run — illustrative only, not a committed
|
||||
benchmark; absolute numbers vary by hardware. The values feed the
|
||||
[Core Performance Summary](#core-performance-summary) example-latency column.
|
||||
|
||||
```
|
||||
📊 Metadata exact match: 0.818ms (50 items matched)
|
||||
📊 Metadata range query: 0.631ms (40 items in range)
|
||||
🔗 Graph neighbor lookup: 0.092ms (2 connections)
|
||||
🎯 Vector k-NN search: 1.773ms (10 nearest neighbors)
|
||||
🧠 NLP query parsing: 8.906ms (full natural language)
|
||||
⚡ Triple Intelligence: 1.830ms (combined query)
|
||||
Metadata exact match: 0.818ms (50 items matched)
|
||||
Metadata range query: 0.631ms (40 items in range)
|
||||
Graph neighbor lookup: 0.092ms (2 connections)
|
||||
Vector k-NN search: 1.773ms (10 nearest neighbors)
|
||||
NLP query parsing: 8.906ms (full natural language)
|
||||
Triple Intelligence: 1.830ms (combined query)
|
||||
```
|
||||
|
||||
### Scaling Characteristics
|
||||
|
||||
| Items | Metadata O(1) | Range O(log n) | Graph O(1) | Vector O(log n) |
|
||||
|-------|---------------|----------------|------------|-----------------|
|
||||
| 100 | 0.8ms | 0.6ms | 0.09ms | 1.8ms |
|
||||
| 1,000 | 0.8ms | 0.9ms | 0.09ms | 2.5ms |
|
||||
| 10,000 | 0.8ms | 1.2ms | 0.09ms | 3.2ms |
|
||||
| 100,000 | 0.8ms | 1.5ms | 0.09ms | 4.1ms |
|
||||
| 1,000,000 | 0.8ms | 1.8ms | 0.09ms | 5.0ms |
|
||||
Each stage scales by its algorithmic complexity, not a fixed millisecond figure
|
||||
— absolute latency depends on hardware, embedding model, and storage backend.
|
||||
Only the graph adjacency index carries a committed scale assertion:
|
||||
|
||||
*Note: O(1) operations maintain constant time regardless of scale*
|
||||
| Query stage | Complexity | Scaling behavior |
|
||||
|-------------|------------|------------------|
|
||||
| Metadata filter (exact) | O(1) | Constant — independent of dataset size |
|
||||
| Metadata filter (range) | O(log n) + O(k) | Sub-linear; k = matching results |
|
||||
| Vector search (HNSW) | O(log n) | Degrades gracefully via hierarchical layers |
|
||||
| Graph hop | O(1) | Measured <1 ms per neighbor lookup, validated up to 1M relationships (`tests/performance/graph-scale-performance.test.ts:238`) |
|
||||
| Combined query | O(log n) | Bounded by the vector stage; metadata and graph stages stay O(1)/O(log n) |
|
||||
|
||||
## Comparison with Other Systems
|
||||
|
||||
|
|
@ -470,8 +485,8 @@ For off-site replication, snapshot `path` from your scheduler (`gsutil rsync`, `
|
|||
- **Auto-configuration system** with environment detection
|
||||
- **Zero-config operation** with intelligent defaults
|
||||
- **Auto-flush and auto-optimize** in indices
|
||||
- **Sub-2ms response times** for complex queries
|
||||
- **Low-latency Triple Intelligence queries** (O(log n) vector + O(1) metadata/graph)
|
||||
|
||||
## Conclusion
|
||||
|
||||
Brainy delivers on its promise of **production-ready Triple Intelligence** with measured, verified performance characteristics. All listed features are fully implemented, tested, and benchmarked. No stubs, no mocks, no theoretical claims - just real, working code with measured performance.
|
||||
Brainy delivers on its promise of **production-ready Triple Intelligence** with documented algorithmic-complexity guarantees and a committed graph-scale benchmark (`tests/performance/graph-scale-performance.test.ts`). All listed features are fully implemented and tested. No stubs, no mocks — just real, working code with characterized performance.
|
||||
Loading…
Add table
Add a link
Reference in a new issue