feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0

8.0 RC cleanup toward "one place per thing, zero-config, no deprecation":

- Remove the `brain.neural()` clustering namespace (ImprovedNeuralAPI + the dead
  legacy NeuralAPI + the neural CLI + neural-only types). Similarity is `find({vector})`
  / `similar({to})`; attribute grouping is the aggregation `GROUP BY` engine. The separate
  entity-extraction / smart-import feature (NeuralImport, NeuralEntityExtractor, SmartExtractor,
  NaturalLanguageProcessor, `brain.extract()`/`brain.nlp()`) is kept.
- Remove `Db.search()`; `find()` is the one query verb (accepts a bare string or FindParams).
  Fix the bundled MCP client, which called a non-existent `brain.search(query, limit)` →
  now `find({ query, limit })`.
- Storage config: collapse to one canonical top-level `path` key. The pre-8.0 aliases
  (`rootDirectory`, `options.*`, `fileSystemStorage.*`) are removed and now THROW with the
  exact rename instead of silently defaulting to `./brainy-data` on upgrade. A single resolver
  feeds createStorage, the 7.x→8.0 migration probe, and the plugin-factory handoff, so a native
  storage provider resolves the identical root (no split-brain).
- Fix `similar({ threshold })`: the min-similarity filter was silently dropped; it is now
  applied as a post-filter on `result.score` (the documented way to bound semantic results).
- Fix `vfs.rename()` on a directory: child path updates spread the entity vector into `update()`
  and failed dimension validation; they are metadata-only updates now.
- Fix `vfs.move()`: copy+delete orphaned the content-addressed content blob (the destination
  shared the source hash, then unlink removed it). `move()` now delegates to `rename()` — an
  in-place path change that preserves the blob and the entity id, for files and directories.
- Fix streaming import: the bulk fast path never flushed mid-import nor signalled queryability.
  Entity writes are now chunked by a progressive flush interval (100 → 1000 → 5000); each chunk
  flushes and emits `progress.queryable`, so imported data is queryable during the import.
- Sweep all docs, comments, and JSDoc for the removed/changed APIs.

Integration suite: 49 files / 588 passed / 0 failed. Unit: 80 files / 1456 passed, no type errors.
This commit is contained in:
David Snelling 2026-06-20 13:31:11 -07:00
parent 0c4a51c24e
commit 606445cd61
74 changed files with 712 additions and 7470 deletions

View file

@ -0,0 +1,68 @@
/**
* @module tests/unit/brainy/similar-threshold
* @description Pins `brain.similar({ threshold })` the min-similarity filter.
*
* Before 8.0 `similar()` accepted a `threshold` but silently DROPPED it (it was
* never forwarded to the query, so callers got unfiltered results). 8.0 applies
* it as a post-filter on `result.score` the canonical way to impose a minimum
* score on plain semantic results (top-level vector search does not honor a
* `threshold`; see the `find({ near })` guidance). These tests use explicit
* vectors so the embedder is never invoked.
*/
import { describe, it, expect, afterEach } from 'vitest'
import { Brainy } from '../../../src/brainy.js'
import { NounType } from '../../../src/types/graphTypes.js'
/** Deterministic 384-dim vectors — no embedder, distinct per seed. */
function vec(seed: number): number[] {
return Array.from({ length: 384 }, (_, i) => ((seed * 31 + i * 13) % 100) / 100)
}
describe('brain.similar() — threshold post-filter (8.0)', () => {
const brains: Brainy[] = []
afterEach(async () => {
for (const b of brains.splice(0)) await b.close().catch(() => {})
})
it('honors the min-similarity threshold (was silently dropped before 8.0)', async () => {
const brain = new Brainy({ requireSubtype: false, storage: { type: 'memory' } })
brains.push(brain)
await brain.init()
for (let i = 0; i < 12; i++) {
await brain.add({ data: `e${i}`, type: NounType.Thing, vector: vec(i) })
}
const target = vec(0)
const all = await brain.similar({ to: target, limit: 100 })
expect(all.length).toBe(12)
const scores = all.map((r) => r.score)
const min = Math.min(...scores)
const max = Math.max(...scores)
// The corpus has a real score spread (one vector is identical to the target).
expect(max).toBeGreaterThan(min)
const threshold = (min + max) / 2
const filtered = await brain.similar({ to: target, limit: 100, threshold })
// THE invariant the fix guarantees: every result meets the threshold.
expect(filtered.every((r) => r.score >= threshold)).toBe(true)
// The threshold is actually applied — weaker matches dropped, strong kept.
expect(filtered.length).toBeGreaterThan(0)
expect(filtered.length).toBeLessThan(all.length)
})
it('returns the full set when no threshold is given (unchanged behavior)', async () => {
const brain = new Brainy({ requireSubtype: false, storage: { type: 'memory' } })
brains.push(brain)
await brain.init()
for (let i = 0; i < 6; i++) {
await brain.add({ data: `n${i}`, type: NounType.Thing, vector: vec(i + 50) })
}
const all = await brain.similar({ to: vec(50), limit: 100 })
expect(all.length).toBe(6)
})
})