docs: rename the native provider to @soulcraft/cor across public docs and JSDoc
The 3.0 native engine ships as @soulcraft/cor (brainy 8.x <-> cor 3.x are a version-matched pair); the old @soulcraft/cortex package stays on the 2.x/7.x line. Public docs and .d.ts-visible comments now name the correct package. Historical 2.x contract references (e.g. the 2.3.1 read-side fallback) keep the old name deliberately.
This commit is contained in:
parent
a3c2717ddb
commit
bf4a333f9b
38 changed files with 104 additions and 104 deletions
|
|
@ -417,7 +417,7 @@ const brain = new Brainy({
|
|||
})
|
||||
```
|
||||
|
||||
The default JS index is `JsHnswVectorIndex`. An optional native acceleration package (`@soulcraft/cortex`) can replace it with a higher-performing implementation; the public knobs stay the same.
|
||||
The default JS index is `JsHnswVectorIndex`. An optional native acceleration package (`@soulcraft/cor`) can replace it with a higher-performing implementation; the public knobs stay the same.
|
||||
|
||||
### Scale Scenarios
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ next:
|
|||
|
||||
# Plugin Development Guide
|
||||
|
||||
Brainy has a plugin system that allows third-party packages to replace internal subsystems with custom implementations. This is how `@soulcraft/cortex` provides optional native acceleration, and it's the same system available to any developer.
|
||||
Brainy has a plugin system that allows third-party packages to replace internal subsystems with custom implementations. This is how `@soulcraft/cor` provides optional native acceleration, and it's the same system available to any developer.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
|
|
@ -27,7 +27,7 @@ Plugins are **opt-in** — brainy never auto-imports packages. You must explicit
|
|||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
plugins: ['@soulcraft/cortex'] // explicitly load the native acceleration package
|
||||
plugins: ['@soulcraft/cor'] // explicitly load the native acceleration package
|
||||
})
|
||||
```
|
||||
|
||||
|
|
@ -36,7 +36,7 @@ const brain = new Brainy({
|
|||
| `undefined` (default) | No plugins loaded |
|
||||
| `false` | No plugins loaded |
|
||||
| `[]` | No plugins loaded |
|
||||
| `['@soulcraft/cortex']` | Load only the listed packages |
|
||||
| `['@soulcraft/cor']` | Load only the listed packages |
|
||||
|
||||
Plugins registered programmatically via `brain.use(plugin)` are always activated regardless of the `plugins` config.
|
||||
|
||||
|
|
@ -215,7 +215,7 @@ context.registerProvider('aggregation', (storage) => {
|
|||
})
|
||||
```
|
||||
|
||||
When provided by an optional native acceleration plugin (such as `@soulcraft/cortex`), this enables:
|
||||
When provided by an optional native acceleration plugin (such as `@soulcraft/cor`), this enables:
|
||||
- Compiled source filters (vs per-entity JS object traversal)
|
||||
- Precise MIN/MAX via sorted data structures (vs lazy recompute)
|
||||
- Parallel aggregate rebuild across CPU cores
|
||||
|
|
@ -251,7 +251,7 @@ Native msgpack encode/decode for SSTable serialization.
|
|||
|
||||
### Analytics Providers (Native-Only)
|
||||
|
||||
These provider keys have **no JavaScript fallback** — they represent capabilities that require native code (SIMD, mmap, sub-microsecond latency). They are available when an optional native acceleration plugin (such as `@soulcraft/cortex`) is installed.
|
||||
These provider keys have **no JavaScript fallback** — they represent capabilities that require native code (SIMD, mmap, sub-microsecond latency). They are available when an optional native acceleration plugin (such as `@soulcraft/cor`) is installed.
|
||||
|
||||
Use `brain.getProvider('analytics:hyperloglog')` to check availability. Returns `undefined` if no plugin provides it.
|
||||
|
||||
|
|
@ -359,8 +359,8 @@ brainy diagnostics
|
|||
When a plugin is active, brainy automatically logs a provider summary after `init()`:
|
||||
|
||||
```
|
||||
[brainy] Plugin activated: @soulcraft/cortex
|
||||
[brainy] Providers: 8/10 native (@soulcraft/cortex) | default: vector, cache
|
||||
[brainy] Plugin activated: @soulcraft/cor
|
||||
[brainy] Providers: 8/10 native (@soulcraft/cor) | default: vector, cache
|
||||
```
|
||||
|
||||
This tells you at a glance how many subsystems are accelerated and which ones are falling back to JavaScript. The log respects `config.silent`.
|
||||
|
|
@ -381,7 +381,7 @@ If a required provider is missing, the error message tells you exactly what's wr
|
|||
|
||||
```
|
||||
[brainy] Required providers using JS fallback: graphIndex.
|
||||
Active plugins: @soulcraft/cortex.
|
||||
Active plugins: @soulcraft/cor.
|
||||
These providers must be supplied by a plugin for this deployment.
|
||||
Check plugin installation, license, and native module availability.
|
||||
```
|
||||
|
|
|
|||
|
|
@ -234,7 +234,7 @@ const all = await brain.related({
|
|||
})
|
||||
|
||||
// Graph traversal — subtype filters traversal edges (depth-1 in 7.30 JS path;
|
||||
// multi-hop subtype filtering lands on Cortex native)
|
||||
// multi-hop subtype filtering lands on Cor native)
|
||||
const reports = await brain.find({
|
||||
connected: {
|
||||
from: ceoId,
|
||||
|
|
|
|||
|
|
@ -37,7 +37,7 @@ The three knobs that matter most:
|
|||
1. **`config.vector.recall`** — `'fast'`, `'balanced'`, or `'accurate'` (default `'balanced'`)
|
||||
2. **`config.vector.persistMode`** — `'immediate'` for durability, `'deferred'` for throughput
|
||||
|
||||
The native vector provider (via the optional `@soulcraft/cortex` package) extends this with a higher-performing index — and its own at-scale acceleration such as on-disk compressed indexing — when installed.
|
||||
The native vector provider (via the optional `@soulcraft/cor` package) extends this with a higher-performing index — and its own at-scale acceleration such as on-disk compressed indexing — when installed.
|
||||
|
||||
## Measured Performance
|
||||
|
||||
|
|
@ -77,7 +77,7 @@ graph, metadata index, and 100k edges).
|
|||
**Scale ceiling (open-core).** The pure-JS HNSW *build* cost (~100 inserts/s at 384-dim on
|
||||
one core) makes the in-process open-core path most appropriate up to ~10⁵–10⁶ entities.
|
||||
*Query* latency stays low well beyond that, but for the 10⁸–10¹⁰ regime install the native
|
||||
provider (`@soulcraft/cortex`, on-disk DiskANN) — same API, no code change. _Projected from
|
||||
provider (`@soulcraft/cor`, on-disk DiskANN) — same API, no code change. _Projected from
|
||||
the two measured points, vector p50 at 1M is ~2 ms; metadata-heavy composition grows with
|
||||
match-set size and is the path to move onto the native provider first._
|
||||
|
||||
|
|
@ -213,12 +213,12 @@ const stats = await brain.stats()
|
|||
### Issue: Slow queries
|
||||
1. Switch to `vector.recall: 'fast'`
|
||||
2. Increase the read cache (`storage.cache.maxSize`)
|
||||
3. Consider the optional native vector provider via `@soulcraft/cortex`
|
||||
3. Consider the optional native vector provider via `@soulcraft/cor`
|
||||
|
||||
### Issue: Memory pressure
|
||||
1. Reduce `storage.cache.maxSize`
|
||||
2. Move to `vector.persistMode: 'deferred'` to batch writes
|
||||
3. Consider the optional native vector provider via `@soulcraft/cortex` for at-scale index acceleration
|
||||
3. Consider the optional native vector provider via `@soulcraft/cor` for at-scale index acceleration
|
||||
|
||||
### Issue: Slow startup after a crash
|
||||
1. Use `vector.persistMode: 'immediate'` so the index file stays in sync with storage
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@
|
|||
│ ▼ │
|
||||
│ ┌──────────────────────┐ │
|
||||
│ │ AggregationProvider │ (optional, registered by │
|
||||
│ │ ├─ incrementalUpdate│ plugin like @soulcraft/cortex)│
|
||||
│ │ ├─ incrementalUpdate│ plugin like @soulcraft/cor) │
|
||||
│ │ ├─ rebuildAggregate │ │
|
||||
│ │ ├─ queryAggregate │ │
|
||||
│ │ └─ serialize/restore│ │
|
||||
|
|
@ -135,7 +135,7 @@ M2 is clamped to zero on remove to prevent floating-point drift from producing n
|
|||
|
||||
The TypeScript engine uses simple comparison for add operations and marks MIN/MAX as potentially stale on delete (since removing the current min/max value requires a rescan). Stale values are lazily recomputed on the next query.
|
||||
|
||||
The Cortex native engine uses a `BTreeMap<OrderedFloat<f64>, u64>` that tracks the exact frequency of every value, providing precise MIN/MAX after any sequence of operations without rescanning.
|
||||
The Cor native engine uses a `BTreeMap<OrderedFloat<f64>, u64>` that tracks the exact frequency of every value, providing precise MIN/MAX after any sequence of operations without rescanning.
|
||||
|
||||
### Time Window Bucketing
|
||||
|
||||
|
|
|
|||
|
|
@ -402,7 +402,7 @@ const DEFAULT_EXCLUDE_FIELDS = [
|
|||
|
||||
**Purpose**: O(log n) semantic similarity search using vector embeddings.
|
||||
|
||||
The default JS implementation is `JsHnswVectorIndex`; an optional native acceleration package (`@soulcraft/cortex`) can register a higher-performing `VectorIndexProvider` through the plugin system. The public API stays the same either way.
|
||||
The default JS implementation is `JsHnswVectorIndex`; an optional native acceleration package (`@soulcraft/cor`) can register a higher-performing `VectorIndexProvider` through the plugin system. The public API stays the same either way.
|
||||
|
||||
### Internal Architecture
|
||||
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ requestFlushOverFilesystem(timeoutMs)
|
|||
|
||||
They live on `BaseStorage` as no-op defaults and are overridden on
|
||||
`FileSystemStorage` with real implementations. Any adapter extending
|
||||
`FileSystemStorage` (e.g. Cortex's `MmapFileSystemStorage`) inherits the
|
||||
`FileSystemStorage` (e.g. Cor's `MmapFileSystemStorage`) inherits the
|
||||
real ones for free.
|
||||
|
||||
This works correctly today. The question is whether the methods *belong*
|
||||
|
|
@ -83,7 +83,7 @@ Benefits:
|
|||
## The case against doing it now
|
||||
|
||||
- Breaking change for any adapter that already overrides these methods.
|
||||
`FileSystemStorage` is the only one in-tree, but Cortex's
|
||||
`FileSystemStorage` is the only one in-tree, but Cor's
|
||||
`MmapFileSystemStorage` inherits from it — interface relocation would
|
||||
ripple through the plugin ecosystem.
|
||||
- The current state works. The real failure modes seen in the field
|
||||
|
|
|
|||
|
|
@ -48,7 +48,7 @@ Pluggable vector index (`VectorIndexProvider`) for efficient nearest-neighbor se
|
|||
- **Configurable recall**: `fast` / `balanced` / `accurate` presets trade recall for latency
|
||||
- **Scalable**: Handles millions of vectors per process
|
||||
- **Persistent**: Serializable to storage
|
||||
- **Swappable**: Replace with a native implementation (such as `@soulcraft/cortex`) via the plugin system without changing application code
|
||||
- **Swappable**: Replace with a native implementation (such as `@soulcraft/cor`) via the plugin system without changing application code
|
||||
|
||||
### Metadata Index Manager
|
||||
High-performance field indexing system:
|
||||
|
|
|
|||
|
|
@ -84,7 +84,7 @@ const brain = new Brainy({
|
|||
```
|
||||
|
||||
The default JS index is `JsHnswVectorIndex`. An optional native acceleration
|
||||
provider (the `@soulcraft/cortex` package) can replace it with a
|
||||
provider (the `@soulcraft/cor` package) can replace it with a
|
||||
higher-performing implementation; the public knobs stay the same. Quantization
|
||||
and other index-internal acceleration are the native provider's concern, not a
|
||||
Brainy configuration option.
|
||||
|
|
|
|||
|
|
@ -55,20 +55,20 @@ process opening the same directory:
|
|||
The fix is the lock: refuse to open a second writer. SQLite has done the same
|
||||
since the late 1990s (`SQLITE_BUSY`).
|
||||
|
||||
## What about Cortex?
|
||||
## What about Cor?
|
||||
|
||||
Brainy + Cortex compose cleanly under this model:
|
||||
Brainy + Cor compose cleanly under this model:
|
||||
|
||||
- Cortex stores its column-index segments inside the same `rootDir` (under
|
||||
- Cor stores its column-index segments inside the same `rootDir` (under
|
||||
`indexes/_column_index/{field}/`).
|
||||
- Segments (`*.cidx` files) are **immutable** once written. Cortex mmaps them
|
||||
- Segments (`*.cidx` files) are **immutable** once written. Cor mmaps them
|
||||
read-only.
|
||||
- The `MANIFEST.json` per field is updated via atomic rename — readers see
|
||||
either the old or new manifest, never a torn file.
|
||||
|
||||
A reader process can safely mmap Cortex segments alongside a live writer
|
||||
A reader process can safely mmap Cor segments alongside a live writer
|
||||
without coordination. The single Brainy writer lock at
|
||||
`<rootDir>/locks/_writer.lock` covers Cortex too, because Cortex segment
|
||||
`<rootDir>/locks/_writer.lock` covers Cor too, because Cor segment
|
||||
writes happen on the writer's side.
|
||||
|
||||
## Stale-lock detection
|
||||
|
|
@ -141,7 +141,7 @@ The CLI `brainy inspect` subcommands all do this for you by default
|
|||
processes can both succeed at `init()` in writer mode and clobber each
|
||||
other's writes. A best-effort warning is logged in writer mode against a
|
||||
non-filesystem backend.
|
||||
- **Long-running readers** do not automatically pick up new Cortex segments
|
||||
- **Long-running readers** do not automatically pick up new Cor segments
|
||||
the writer publishes. One-shot inspector calls re-open the store and see
|
||||
fresh segments; a reader that stays open for hours sees its column store
|
||||
as-of the time it opened.
|
||||
|
|
@ -150,4 +150,4 @@ The CLI `brainy inspect` subcommands all do this for you by default
|
|||
|
||||
- `Brainy.openReadOnly()` — [API reference](../api/brainy.md)
|
||||
- `brainy inspect` — [inspection guide](../guides/inspection.md)
|
||||
- Cortex columnar storage — see `node_modules/@soulcraft/cortex/README.md`
|
||||
- Cor columnar storage — see `node_modules/@soulcraft/cor/README.md`
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@ BaseStorage (type-statistics, lifecycle helpers, generation
|
|||
FileSystemStorage (real filesystem I/O, writer-lock implementation,
|
||||
flush-request watcher, atomic writes)
|
||||
↑
|
||||
<your plugin's storage> (Cortex's MmapFileSystemStorage, etc.)
|
||||
<your plugin's storage> (Cor's MmapFileSystemStorage, etc.)
|
||||
```
|
||||
|
||||
When you `extend FileSystemStorage`, your adapter inherits every method on
|
||||
|
|
|
|||
14
docs/eli5.md
14
docs/eli5.md
|
|
@ -11,7 +11,7 @@ next:
|
|||
- getting-started/quick-start
|
||||
---
|
||||
|
||||
# Brainy and Cortex — Explained Simply
|
||||
# Brainy and Cor — Explained Simply
|
||||
|
||||
*A plain-language guide for anyone who wants to understand what this thing actually does.*
|
||||
|
||||
|
|
@ -65,13 +65,13 @@ Brainy can narrow any result set down by exact labels or ranges in the same brea
|
|||
|
||||
---
|
||||
|
||||
## What is Cortex?
|
||||
## What is Cor?
|
||||
|
||||
Cortex is a turbocharger for Brainy.
|
||||
Cor is a turbocharger for Brainy.
|
||||
|
||||
Same car. Same controls. Same fuel. You just swap in a faster engine under the hood, and everything that used to take a moment now happens instantly.
|
||||
|
||||
Technically, Cortex is an optional plugin written in Rust — a lower-level language that runs much closer to the raw metal of your processor. It plugs into Brainy and takes over the most compute-intensive work: the distance calculations that power meaning search, the number-crunching behind live aggregates, and the set operations that drive label filtering.
|
||||
Technically, Cor is an optional plugin written in Rust — a lower-level language that runs much closer to the raw metal of your processor. It plugs into Brainy and takes over the most compute-intensive work: the distance calculations that power meaning search, the number-crunching behind live aggregates, and the set operations that drive label filtering.
|
||||
|
||||
You install it with one line, register it with one call, and Brainy automatically uses it everywhere it can help.
|
||||
|
||||
|
|
@ -83,9 +83,9 @@ Plain language:
|
|||
|
||||
- **Searches** go from "the blink of an eye" to "faster than a blink." The overall speedup is **5.2× on average** across all operations.
|
||||
- **Live aggregates** are rebuilt using all CPU cores in parallel, so re-indexing large datasets takes a fraction of the time.
|
||||
- **Analytics** that aren't even possible in pure JavaScript — real-time anomaly detection, streaming percentile estimates, approximate unique counts — become available because Cortex brings the native capabilities required to run them efficiently.
|
||||
- **Analytics** that aren't even possible in pure JavaScript — real-time anomaly detection, streaming percentile estimates, approximate unique counts — become available because Cor brings the native capabilities required to run them efficiently.
|
||||
|
||||
If Brainy is what makes knowledge fast, Cortex is what makes Brainy feel instant.
|
||||
If Brainy is what makes knowledge fast, Cor is what makes Brainy feel instant.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -135,7 +135,7 @@ Brainy is the only row with every box checked. And it runs all of them in a sing
|
|||
|
||||
Brainy scales from a quick experiment to serious production datasets without changing a line of code. Small datasets live entirely in memory. Larger ones spill to disk, where Brainy shards and compresses everything automatically. Need a backup or a copy? Snapshots are instant — the same API the whole way.
|
||||
|
||||
Add Cortex and you also unlock memory-mapped storage — aggregate state lives directly in the operating system's memory with zero serialization overhead, as fast as the hardware allows.
|
||||
Add Cor and you also unlock memory-mapped storage — aggregate state lives directly in the operating system's memory with zero serialization overhead, as fast as the hardware allows.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -487,7 +487,7 @@ Aggregate definitions and running state are automatically persisted:
|
|||
|
||||
## Native Acceleration
|
||||
|
||||
When [Cortex](https://github.com/soulcraftlabs/cortex) is installed as a plugin, the aggregation engine automatically uses Rust-accelerated computation:
|
||||
When [Cor](https://www.npmjs.com/package/@soulcraft/cor) is installed as a plugin, the aggregation engine automatically uses Rust-accelerated computation:
|
||||
|
||||
- Incremental updates run in Rust with BTreeMap-backed precise MIN/MAX
|
||||
- Welford's online stddev/variance computed natively
|
||||
|
|
@ -496,7 +496,7 @@ When [Cortex](https://github.com/soulcraftlabs/cortex) is installed as a plugin,
|
|||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
plugins: ['@soulcraft/cortex']
|
||||
plugins: ['@soulcraft/cor']
|
||||
})
|
||||
await brain.init()
|
||||
|
||||
|
|
@ -577,7 +577,7 @@ brain.defineAggregate({
|
|||
|
||||
Aggregation complexity per write is O(A x G x M) where A = matching aggregates, G = groupBy dimensions, M = metrics. For typical configurations (2-5 aggregates, 1-3 dimensions, 3-5 metrics), this is effectively O(1).
|
||||
|
||||
With Cortex native acceleration:
|
||||
With Cor native acceleration:
|
||||
|
||||
| Operation | Throughput | Latency |
|
||||
|-----------|-----------|---------|
|
||||
|
|
|
|||
|
|
@ -174,13 +174,13 @@ await brain.syncWith({
|
|||
const brain = new Brainy()
|
||||
|
||||
// 1 → ~1M vectors: pure-JS HNSW, zero extra setup
|
||||
// 1M → 10B+ vectors: install @soulcraft/cortex for the native DiskANN provider
|
||||
// 1M → 10B+ vectors: install @soulcraft/cor for the native DiskANN provider
|
||||
```
|
||||
|
||||
**Scaling model:**
|
||||
- **Single process, no cluster**: Brainy runs in one process — no coordinator,
|
||||
no peer discovery, no consensus to operate
|
||||
- **Optional native provider**: install `@soulcraft/cortex` to back the index with
|
||||
- **Optional native provider**: install `@soulcraft/cor` to back the index with
|
||||
on-disk DiskANN that scales to 10B+ vectors on one machine
|
||||
- **Per-tenant pools**: isolate tenants by giving each its own Brainy instance and
|
||||
storage directory
|
||||
|
|
|
|||
|
|
@ -45,20 +45,20 @@ console.log('Brainy ready.')
|
|||
|
||||
## Native Acceleration (Optional)
|
||||
|
||||
For production workloads, add Cortex for Rust-accelerated SIMD distance calculations and native embeddings:
|
||||
For production workloads, add Cor for Rust-accelerated SIMD distance calculations and native embeddings:
|
||||
|
||||
```bash
|
||||
npm install @soulcraft/cortex
|
||||
npm install @soulcraft/cor
|
||||
```
|
||||
|
||||
```typescript
|
||||
import { Brainy } from '@soulcraft/brainy'
|
||||
|
||||
const brain = new Brainy({ plugins: ['@soulcraft/cortex'] })
|
||||
const brain = new Brainy({ plugins: ['@soulcraft/cor'] })
|
||||
await brain.init() // native providers registered during init
|
||||
```
|
||||
|
||||
Cortex registers native (Rust/SIMD) vector, metadata, and graph engines behind the same Brainy `find()` API — no code changes, an optional dependency for production-scale workloads.
|
||||
Cor registers native (Rust/SIMD) vector, metadata, and graph engines behind the same Brainy `find()` API — no code changes, an optional dependency for production-scale workloads.
|
||||
|
||||
## Server-only since 8.0
|
||||
|
||||
|
|
|
|||
|
|
@ -332,7 +332,7 @@ await brain.updateRelation({ id: relationId, weight: 0.5, confidence: 0.9 })
|
|||
`find({ connected, subtype })` filters traversal edges by their subtype. Composes with `via` (verb-type filter):
|
||||
|
||||
```typescript
|
||||
// All direct reports two hops deep (will be supported in Cortex; for now depth-1)
|
||||
// All direct reports two hops deep (will be supported in Cor; for now depth-1)
|
||||
const directChain = await brain.find({
|
||||
connected: {
|
||||
from: ceoId,
|
||||
|
|
@ -343,7 +343,7 @@ const directChain = await brain.find({
|
|||
})
|
||||
```
|
||||
|
||||
Multi-hop subtype filtering (`depth > 1`) lights up on the Cortex native path; the JS path throws today rather than return incorrect partial results.
|
||||
Multi-hop subtype filtering (`depth > 1`) lights up on the Cor native path; the JS path throws today rather than return incorrect partial results.
|
||||
|
||||
### Counts
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue