refactor(8.0)!: remove orphaned zero-config subsystem + dead cloud/progressive-init storage vestige
The old config-generation subsystem (src/config/ + autoConfiguration.ts) was
superseded during the 8.0 rework and never wired into init(): it emitted settings
for a partitioning subsystem that no longer exists and probed deleted cloud env
vars. The live zero-config path is inline — recall preset → HNSW knobs, storage
auto-detect, auto persistMode, container-memory-aware cache sizing.
The storage progressive-init / cloud-detection cluster was equally dead after the
cloud adapters were dropped: isCloudStorage() is permanently false (no overriders),
scheduleBackgroundInit/runBackgroundInit were never called (the latter an empty
body), initMode was never assigned, and Brainy.isFullyInitialized()/
awaitBackgroundInit() were always-trivial with zero callers. scheduleCountPersist()
collapses to its only-ever-taken immediate write-through path.
Removed:
- src/config/{index,zeroConfig,storageAutoConfig,modelAutoConfig,sharedConfigManager}.ts
- src/utils/autoConfiguration.ts + the inert BrainyZeroConfig export
- Brainy.isFullyInitialized()/awaitBackgroundInit() (+ BrainyInterface decls)
- InitMode type, isCloudStorage/detectCloudEnvironment/resolveInitMode,
scheduleBackgroundInit/runBackgroundInit/ensureValidatedForWrite and their state
- Dead cloud env-var probes (K_SERVICE/K_REVISION/AWS_LAMBDA_FUNCTION_NAME/
FUNCTIONS_TARGET/AZURE_FUNCTIONS_ENVIRONMENT)
Kept (verified live): production-detection logging (environment.ts), container-
memory cache sizing (memoryDetection/paramValidation), on-disk hash bucketing
(sharding.ts).
Docs: scrubbed deleted-subsystem references (JS quantization knobs, cloud/OPFS
adapters, partitioning, old zero-config API) across 14 files; deleted two wholly-
obsolete feature docs (complete-feature-list, v3-features); rewrote
architecture/zero-config for 8.0.
~3,700 LOC removed. Build clean; 1392 unit + 24 db-mvcc green.
This commit is contained in:
parent
00d3203d68
commit
35b9d7ef43
28 changed files with 596 additions and 3752 deletions
|
|
@ -35,10 +35,9 @@ Brainy 8.0 is a **single-node library**. There is no cluster, no peer discovery,
|
|||
The three knobs that matter most:
|
||||
|
||||
1. **`config.vector.recall`** — `'fast'`, `'balanced'`, or `'accurate'` (default `'balanced'`)
|
||||
2. **`config.vector.quantization`** — `{ bits: 4 | 8 }` for memory savings on the open-core JS vector index
|
||||
3. **`config.vector.persistMode`** — `'immediate'` for durability, `'deferred'` for throughput
|
||||
2. **`config.vector.persistMode`** — `'immediate'` for durability, `'deferred'` for throughput
|
||||
|
||||
The native vector provider (via the optional `@soulcraft/cortex` package) extends this with a higher-performing index when installed.
|
||||
The native vector provider (via the optional `@soulcraft/cortex` package) extends this with a higher-performing index — and its own at-scale acceleration such as on-disk compressed indexing — when installed.
|
||||
|
||||
## Storage Configurations
|
||||
|
||||
|
|
@ -95,7 +94,6 @@ const brain = new Brainy({
|
|||
storage: { type: 'filesystem', rootDirectory: '/var/lib/brainy' },
|
||||
vector: {
|
||||
recall: 'fast', // Trade recall for latency
|
||||
quantization: { bits: 8 }, // SQ8 quantization
|
||||
persistMode: 'deferred' // Batch persistence
|
||||
}
|
||||
})
|
||||
|
|
@ -142,8 +140,7 @@ Your service layer handles routing and isolation; Brainy stays simple.
|
|||
const brain = new Brainy({
|
||||
storage: { type: 'filesystem', rootDirectory: '/var/lib/brainy' },
|
||||
vector: {
|
||||
recall: 'accurate',
|
||||
quantization: { bits: 8 }
|
||||
recall: 'accurate'
|
||||
}
|
||||
})
|
||||
```
|
||||
|
|
@ -153,7 +150,6 @@ const brain = new Brainy({
|
|||
| Setting | Values | When to change |
|
||||
|---------|--------|----------------|
|
||||
| `vector.recall` | `'fast'` / `'balanced'` / `'accurate'` | Trade recall for latency |
|
||||
| `vector.quantization.bits` | `4` / `8` | Smaller index, lower memory |
|
||||
| `vector.persistMode` | `'immediate'` / `'deferred'` | Throughput vs. durability |
|
||||
| `storage.cache.maxSize` | integer | Hot-path read cache size |
|
||||
| `storage.cache.ttl` | ms | Cache freshness |
|
||||
|
|
@ -174,14 +170,13 @@ const stats = await brain.stats()
|
|||
|
||||
### Issue: Slow queries
|
||||
1. Switch to `vector.recall: 'fast'`
|
||||
2. Enable SQ8 quantization
|
||||
3. Increase the read cache (`storage.cache.maxSize`)
|
||||
4. Consider the optional native vector provider via `@soulcraft/cortex`
|
||||
2. Increase the read cache (`storage.cache.maxSize`)
|
||||
3. Consider the optional native vector provider via `@soulcraft/cortex`
|
||||
|
||||
### Issue: Memory pressure
|
||||
1. Enable `vector.quantization: { bits: 4 }` or `{ bits: 8 }`
|
||||
2. Reduce `storage.cache.maxSize`
|
||||
3. Move to `vector.persistMode: 'deferred'` to batch writes
|
||||
1. Reduce `storage.cache.maxSize`
|
||||
2. Move to `vector.persistMode: 'deferred'` to batch writes
|
||||
3. Consider the optional native vector provider via `@soulcraft/cortex` for at-scale index acceleration
|
||||
|
||||
### Issue: Slow startup after a crash
|
||||
1. Use `vector.persistMode: 'immediate'` so the index file stays in sync with storage
|
||||
|
|
@ -198,5 +193,5 @@ const stats = await brain.stats()
|
|||
|
||||
- Brainy 8.0 is a **library**, not a cluster
|
||||
- Storage adapters: `filesystem`, `memory`, `auto`
|
||||
- Vector tuning: `recall`, `quantization`, `persistMode`
|
||||
- Vector tuning: `recall`, `persistMode`
|
||||
- Backup is an operator-layer concern — snapshot `rootDirectory`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue