feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0
8.0 RC cleanup toward "one place per thing, zero-config, no deprecation":
- Remove the `brain.neural()` clustering namespace (ImprovedNeuralAPI + the dead
legacy NeuralAPI + the neural CLI + neural-only types). Similarity is `find({vector})`
/ `similar({to})`; attribute grouping is the aggregation `GROUP BY` engine. The separate
entity-extraction / smart-import feature (NeuralImport, NeuralEntityExtractor, SmartExtractor,
NaturalLanguageProcessor, `brain.extract()`/`brain.nlp()`) is kept.
- Remove `Db.search()`; `find()` is the one query verb (accepts a bare string or FindParams).
Fix the bundled MCP client, which called a non-existent `brain.search(query, limit)` →
now `find({ query, limit })`.
- Storage config: collapse to one canonical top-level `path` key. The pre-8.0 aliases
(`rootDirectory`, `options.*`, `fileSystemStorage.*`) are removed and now THROW with the
exact rename instead of silently defaulting to `./brainy-data` on upgrade. A single resolver
feeds createStorage, the 7.x→8.0 migration probe, and the plugin-factory handoff, so a native
storage provider resolves the identical root (no split-brain).
- Fix `similar({ threshold })`: the min-similarity filter was silently dropped; it is now
applied as a post-filter on `result.score` (the documented way to bound semantic results).
- Fix `vfs.rename()` on a directory: child path updates spread the entity vector into `update()`
and failed dimension validation; they are metadata-only updates now.
- Fix `vfs.move()`: copy+delete orphaned the content-addressed content blob (the destination
shared the source hash, then unlink removed it). `move()` now delegates to `rename()` — an
in-place path change that preserves the blob and the entity id, for files and directories.
- Fix streaming import: the bulk fast path never flushed mid-import nor signalled queryability.
Entity writes are now chunked by a progressive flush interval (100 → 1000 → 5000); each chunk
flushes and emits `progress.queryable`, so imported data is queryable during the import.
- Sweep all docs, comments, and JSDoc for the removed/changed APIs.
Integration suite: 49 files / 588 passed / 0 failed. Unit: 80 files / 1456 passed, no type errors.
This commit is contained in:
parent
0c4a51c24e
commit
606445cd61
74 changed files with 712 additions and 7470 deletions
|
|
@ -20,7 +20,7 @@ const brain = new Brainy({ storage: { type: 'memory' } })
|
|||
### On-Disk (Default for Node)
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'filesystem', rootDirectory: './brainy-data' }
|
||||
storage: { type: 'filesystem', path: './brainy-data' }
|
||||
})
|
||||
```
|
||||
|
||||
|
|
@ -88,11 +88,11 @@ match-set size and is the path to move onto the native provider first._
|
|||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'filesystem',
|
||||
rootDirectory: '/var/lib/brainy'
|
||||
path: '/var/lib/brainy'
|
||||
}
|
||||
})
|
||||
```
|
||||
- Stores everything in a sharded JSON tree under `rootDirectory`
|
||||
- Stores everything in a sharded JSON tree under `path`
|
||||
- Atomic writes via rename
|
||||
- Survives process restarts
|
||||
- Snapshot it off-site with `gsutil rsync`, `aws s3 sync`, `rclone`, or `tar` from your scheduler
|
||||
|
|
@ -108,10 +108,10 @@ const brain = new Brainy({ storage: { type: 'memory' } })
|
|||
### Auto
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'auto', rootDirectory: './data' }
|
||||
storage: { type: 'auto', path: './data' }
|
||||
})
|
||||
```
|
||||
- Picks `filesystem` when running on Node with a writable `rootDirectory`
|
||||
- Picks `filesystem` when running on Node with a writable `path`
|
||||
- Falls back to `memory` otherwise
|
||||
|
||||
## Scaling Patterns
|
||||
|
|
@ -125,7 +125,7 @@ const brain = new Brainy({ storage: { type: 'memory' } })
|
|||
### Stage 2: Production (Filesystem)
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'filesystem', rootDirectory: '/var/lib/brainy' }
|
||||
storage: { type: 'filesystem', path: '/var/lib/brainy' }
|
||||
})
|
||||
// Most production workloads up to ~10M entities on a single host
|
||||
```
|
||||
|
|
@ -133,7 +133,7 @@ const brain = new Brainy({
|
|||
### Stage 3: Higher Throughput (Tune the Vector Index)
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'filesystem', rootDirectory: '/var/lib/brainy' },
|
||||
storage: { type: 'filesystem', path: '/var/lib/brainy' },
|
||||
vector: {
|
||||
recall: 'fast', // Trade recall for latency
|
||||
persistMode: 'deferred' // Batch persistence
|
||||
|
|
@ -142,14 +142,14 @@ const brain = new Brainy({
|
|||
```
|
||||
|
||||
### Stage 4: Multi-Instance (Operator-Layer)
|
||||
Run multiple Brainy processes behind your own routing/service layer. Each process owns its own `rootDirectory`. Sync each artifact off-site independently. Brainy itself does not coordinate between processes.
|
||||
Run multiple Brainy processes behind your own routing/service layer. Each process owns its own `path`. Sync each artifact off-site independently. Brainy itself does not coordinate between processes.
|
||||
|
||||
## Real World Examples
|
||||
|
||||
### Example 1: Single-Node App With Backup
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'filesystem', rootDirectory: '/var/lib/brainy' }
|
||||
storage: { type: 'filesystem', path: '/var/lib/brainy' }
|
||||
})
|
||||
```
|
||||
Schedule (cron / systemd timer):
|
||||
|
|
@ -170,7 +170,7 @@ function brainForTenant(tenantId: string) {
|
|||
return new Brainy({
|
||||
storage: {
|
||||
type: 'filesystem',
|
||||
rootDirectory: `/var/lib/brainy/${tenantId}`
|
||||
path: `/var/lib/brainy/${tenantId}`
|
||||
}
|
||||
})
|
||||
}
|
||||
|
|
@ -180,7 +180,7 @@ Your service layer handles routing and isolation; Brainy stays simple.
|
|||
### Example 4: Higher Recall at Scale
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'filesystem', rootDirectory: '/var/lib/brainy' },
|
||||
storage: { type: 'filesystem', path: '/var/lib/brainy' },
|
||||
vector: {
|
||||
recall: 'accurate'
|
||||
}
|
||||
|
|
@ -226,7 +226,7 @@ const stats = await brain.stats()
|
|||
|
||||
## Best Practices
|
||||
|
||||
1. **One process = one `rootDirectory`** — never share a directory between processes
|
||||
1. **One process = one `path`** — never share a directory between processes
|
||||
2. **Snapshot from your scheduler** — Brainy doesn't ship cloud SDKs; use `rclone` / `aws s3 sync` / `gsutil`
|
||||
3. **Profile before tuning** — `recall: 'balanced'` is right for most workloads
|
||||
4. **Install the native vector provider only when measured profiling shows it pays off**
|
||||
|
|
@ -236,4 +236,4 @@ const stats = await brain.stats()
|
|||
- Brainy 8.0 is a **library**, not a cluster
|
||||
- Storage adapters: `filesystem`, `memory`, `auto`
|
||||
- Vector tuning: `recall`, `persistMode`
|
||||
- Backup is an operator-layer concern — snapshot `rootDirectory`
|
||||
- Backup is an operator-layer concern — snapshot `path`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue