docs(8.0): Phase F — deep clean across 21 docs
Aligned every public doc to the 8.0 contract: filesystem + memory adapters
only, vector index provider terminology (config.vector with recall +
quantization + persistMode knobs), no cloud storage adapters, no closed-
source product names.
Tier 1 — heavier rewrites:
- docs/architecture/storage-architecture.md
- docs/architecture/data-storage-architecture.md
- docs/architecture/distributed-storage.md DELETED — content was 100%
cloud-coordination examples with no 8.0 substance.
- docs/guides/distributed-system.md DELETED — same reason; no inbound refs.
- docs/SCALING.md rewritten for single-node guidance.
- docs/PLUGINS.md, docs/augmentations/{COMPLETE-REFERENCE,README}.md:
HnswProvider→VectorIndexProvider, hnsw→vector key.
- docs/PERFORMANCE.md, docs/BATCHING.md cloud-detection + sharding
sections replaced with single-node vector tuning + filesystem framing.
Tier 2 — surgical renames + cloud-section deletions:
- architecture/{index,initialization-and-rebuild,overview}.md
- transactions.md, DEVELOPER_LEARNING_PATH.md
- vfs/{VFS_API_GUIDE,COMMON_PATTERNS}.md
- api/README.md, guides/{inspection,import-flow}.md
Tier 3 — light edits:
- docs/README.md, architecture/augmentation-system-audit.md
MIGRATION-V3-TO-V4.md untouched (internal migration doc, no stale terms).
This commit is contained in:
parent
2626ab8d62
commit
adda1570f3
22 changed files with 570 additions and 2658 deletions
|
|
@ -5,7 +5,7 @@ public: true
|
|||
category: guides
|
||||
template: guide
|
||||
order: 4
|
||||
description: Replace any Brainy subsystem — distance functions, embeddings, HNSW index, metadata index, aggregation — with a custom implementation or native Rust via Cortex.
|
||||
description: Replace any Brainy subsystem — distance functions, embeddings, vector index, metadata index, aggregation — with a custom implementation or optional native acceleration.
|
||||
next:
|
||||
- cortex/comparison
|
||||
- guides/storage-adapters
|
||||
|
|
@ -13,7 +13,7 @@ next:
|
|||
|
||||
# Plugin Development Guide
|
||||
|
||||
Brainy has a plugin system that allows third-party packages to replace internal subsystems with custom implementations. This is how `@soulcraft/cortex` provides native Rust acceleration, and it's the same system available to any developer.
|
||||
Brainy has a plugin system that allows third-party packages to replace internal subsystems with custom implementations. This is how `@soulcraft/cortex` provides optional native acceleration, and it's the same system available to any developer.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
|
|
@ -28,7 +28,7 @@ Plugins are **opt-in** — brainy never auto-imports packages. You must explicit
|
|||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
plugins: ['@soulcraft/cortex'] // explicitly load cortex
|
||||
plugins: ['@soulcraft/cortex'] // explicitly load the native acceleration package
|
||||
})
|
||||
```
|
||||
|
||||
|
|
@ -109,7 +109,7 @@ Each key has a specific expected signature. Brainy checks for these during `init
|
|||
#### `distance`
|
||||
**Type:** `(a: number[], b: number[]) => number`
|
||||
|
||||
Replaces the default cosine distance function used in HNSW search and neural APIs. This is the highest-impact single provider — it's called for every vector comparison.
|
||||
Replaces the default cosine distance function used in vector search and neural APIs. This is the highest-impact single provider — it's called for every vector comparison.
|
||||
|
||||
```typescript
|
||||
context.registerProvider('distance', (a: number[], b: number[]): number => {
|
||||
|
|
@ -151,10 +151,10 @@ context.registerProvider('embedBatch', async (texts: string[]) => {
|
|||
|
||||
### Index Providers
|
||||
|
||||
#### `hnsw`
|
||||
**Type:** `(config: object, distanceFunction: Function, options: object) => HNSWIndex-compatible`
|
||||
#### `vector`
|
||||
**Type:** `(config: object, distanceFunction: Function, options: object) => VectorIndexProvider-compatible`
|
||||
|
||||
Factory function that creates an HNSW index instance. The returned object must implement the `HNSWIndex` public API:
|
||||
Factory function that creates a vector index instance. The returned object must implement the `VectorIndexProvider` public API:
|
||||
|
||||
- `addItem(item: { id: string, vector: number[] }): Promise<string>`
|
||||
- `search(queryVector: number[], k: number, filter?, options?): Promise<Array<[string, number]>>`
|
||||
|
|
@ -174,12 +174,12 @@ Factory function that creates an HNSW index instance. The returned object must i
|
|||
- `setUseParallelization(boolean): void`
|
||||
|
||||
For type-aware indexes (separate graph per noun type), also implement:
|
||||
- `getIndexForType(type: string): HNSWIndex` (duck-typed detection)
|
||||
- `getIndexForType(type: string): VectorIndexProvider` (duck-typed detection)
|
||||
- `search(queryVector, k, type?, filter?, options?): Promise<Array<[string, number]>>`
|
||||
|
||||
```typescript
|
||||
context.registerProvider('hnsw', (config, distanceFn, options) => {
|
||||
return new MyNativeHNSWIndex(config, distanceFn, options)
|
||||
context.registerProvider('vector', (config, distanceFn, options) => {
|
||||
return new MyNativeVectorIndex(config, distanceFn, options)
|
||||
})
|
||||
```
|
||||
|
||||
|
|
@ -216,7 +216,7 @@ context.registerProvider('aggregation', (storage) => {
|
|||
})
|
||||
```
|
||||
|
||||
When provided by a native plugin like `@soulcraft/cortex`, this enables:
|
||||
When provided by an optional native acceleration plugin (such as `@soulcraft/cortex`), this enables:
|
||||
- Compiled source filters (vs per-entity JS object traversal)
|
||||
- Precise MIN/MAX via sorted data structures (vs lazy recompute)
|
||||
- Parallel aggregate rebuild across CPU cores
|
||||
|
|
@ -227,7 +227,7 @@ When provided by a native plugin like `@soulcraft/cortex`, this enables:
|
|||
#### `cache`
|
||||
**Type:** `UnifiedCache`
|
||||
|
||||
Replaces the global `UnifiedCache` singleton used for VFS path resolution, semantic caching, and HNSW vector caching. Must implement the `UnifiedCache` interface (available from `@soulcraft/brainy/internals`).
|
||||
Replaces the global `UnifiedCache` singleton used for VFS path resolution, semantic caching, and vector index caching. Must implement the `UnifiedCache` interface (available from `@soulcraft/brainy/internals`).
|
||||
|
||||
```typescript
|
||||
import type { UnifiedCache } from '@soulcraft/brainy/internals'
|
||||
|
|
@ -252,7 +252,7 @@ Native msgpack encode/decode for SSTable serialization.
|
|||
|
||||
### Analytics Providers (Native-Only)
|
||||
|
||||
These provider keys have **no JavaScript fallback** — they represent capabilities that require native code (SIMD, mmap, sub-microsecond latency). They are available when a native plugin like `@soulcraft/cortex` is installed.
|
||||
These provider keys have **no JavaScript fallback** — they represent capabilities that require native code (SIMD, mmap, sub-microsecond latency). They are available when an optional native acceleration plugin (such as `@soulcraft/cortex`) is installed.
|
||||
|
||||
Use `brain.getProvider('analytics:hyperloglog')` to check availability. Returns `undefined` if no plugin provides it.
|
||||
|
||||
|
|
@ -338,11 +338,11 @@ console.log(diag)
|
|||
// embeddings: { source: 'plugin' },
|
||||
// embedBatch: { source: 'plugin' },
|
||||
// distance: { source: 'plugin' },
|
||||
// hnsw: { source: 'default' },
|
||||
// vector: { source: 'default' },
|
||||
// ...
|
||||
// },
|
||||
// indexes: {
|
||||
// hnsw: { size: 0, type: 'TypeAwareHNSWIndex' },
|
||||
// vector: { size: 0, type: 'JsHnswVectorIndex' },
|
||||
// metadata: { type: 'MetadataIndexManager', initialized: true },
|
||||
// graph: { type: 'GraphAdjacencyIndex', initialized: true, wiredToStorage: true }
|
||||
// }
|
||||
|
|
@ -361,7 +361,7 @@ When a plugin is active, brainy automatically logs a provider summary after `ini
|
|||
|
||||
```
|
||||
[brainy] Plugin activated: @soulcraft/cortex
|
||||
[brainy] Providers: 8/10 native (@soulcraft/cortex) | default: hnsw, cache
|
||||
[brainy] Providers: 8/10 native (@soulcraft/cortex) | default: vector, cache
|
||||
```
|
||||
|
||||
This tells you at a glance how many subsystems are accelerated and which ones are falling back to JavaScript. The log respects `config.silent`.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue