refactor(8.0): simplify config.vector to 3 knobs + fold persistMode (scaffold step 8)
Per user direction and BRAINY-8.0-RENAME-COORDINATION § G.2: the
`config.vector.advanced.{hnsw, diskann}` escape-hatch surface was
over-engineered. The 8.0 config surface collapses to **three knobs**:
```
config.vector = {
recall?: 'fast' | 'balanced' | 'accurate'
quantization?: { enabled?, bits?: 8 | 4 }
persistMode?: 'immediate' | 'deferred'
}
```
The algorithm-internal HNSW knobs (`M`, `efConstruction`, `efSearch`,
`ml`) and DiskANN knobs (`pqM`, `searchListSize`, `alpha`, …) are no
longer exposed at the public surface. The `recall` preset covers the
legitimate quality/latency tradeoff; algorithm-internal tuning is
intentionally hidden. If users need it later, we can add it back —
shipping with too few knobs is easier to evolve than shipping with too
many.
CHANGES
src/types/brainy.types.ts
- BrainyConfig.hnswPersistMode (7.x top-level) → BrainyConfig.vector.persistMode.
- BrainyConfig.vector — dropped `advanced.{hnsw, diskann}` sub-block.
- BrainyConfig.vector.quantization — dropped `rerankMultiplier` (hardcoded to 3).
- BrainyConfig.vector — dropped `vectorStorage: 'memory' | 'lazy'`. Brainy
always keeps vectors resident in 8.0; the lazy-eviction path was a
cloud-storage optimisation that's no longer needed (cloud adapters are gone).
- JSDoc tightened to reflect the closed-form contract.
src/utils/recallPreset.ts
- resolveJsHnswConfig() signature narrowed: accepts `{ recall? }` only
(no more `advanced` parameter).
src/brainy.ts
- normalizeConfig() — no longer carries `hnswPersistMode` on the resolved
config. Dropped the deprecated 'gcs-native' warning, the gcsStorage
HMAC-key warning, and the lenient storage-pairing block (dead code now
that cloud adapters are gone).
- resolveHNSWPersistMode() — collapsed to a one-liner:
`return this.config.vector?.persistMode ?? 'immediate'`. The cloud-vs-local
smart-default branching is no longer relevant (filesystem is the only
persistent backend in 8.0).
- setupIndex() — reads quantization fields from config.vector directly;
rerankMultiplier hardcoded to 3.
NO-OP scope
The recall preset's 'balanced' = brainy 7.x DEFAULT_CONFIG byte-for-byte,
so users who don't set recall get the same behavior. Users who set
config.hnsw.* in 7.x will need to migrate to config.vector.* per the
8.0 release notes; pure 7.x defaults users see no change.
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1408 / 1409 (same pre-existing test-isolation race condition
from step 7, not related to step 8)
This commit is contained in:
parent
0e6263a1bd
commit
8e767408d9
3 changed files with 68 additions and 105 deletions
|
|
@ -9068,16 +9068,17 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
}
|
||||
|
||||
/**
|
||||
* Setup index — single unified HNSW graph.
|
||||
* Setup index — single unified vector graph.
|
||||
*
|
||||
* Smart defaults for HNSW persistence mode:
|
||||
* - Cloud storage (GCS/S3/R2/Azure): 'deferred' for 30-50x faster adds
|
||||
* - Local storage (FileSystem/Memory/OPFS): 'immediate' (already fast)
|
||||
* Brainy 8.0 ships filesystem-only storage (no cloud adapters), so persistence
|
||||
* mode defaults to `'immediate'` everywhere. Operators who want bulk-ingest
|
||||
* speed can set `config.vector.persistMode = 'deferred'`.
|
||||
*/
|
||||
private setupIndex(): JsHnswVectorIndex {
|
||||
// 8.0 config surface: config.vector.{quantization, vectorStorage, recall, advanced}.
|
||||
// The recall preset translates to HNSW knobs (M / efConstruction / efSearch) via
|
||||
// resolveJsHnswConfig; explicit advanced.hnsw overrides win.
|
||||
// 8.0 config surface: config.vector.{recall, quantization, persistMode}.
|
||||
// The recall preset translates to HNSW knobs (M / efConstruction / efSearch)
|
||||
// via resolveJsHnswConfig. No algorithm-internal knobs are exposed at the
|
||||
// public surface.
|
||||
const recallKnobs = resolveJsHnswConfig(this.config.vector)
|
||||
const vectorCfg = this.config.vector
|
||||
const indexConfig = {
|
||||
|
|
@ -9087,9 +9088,8 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
quantization: vectorCfg?.quantization ? {
|
||||
enabled: vectorCfg.quantization.enabled ?? false,
|
||||
bits: vectorCfg.quantization.bits ?? 8,
|
||||
rerankMultiplier: vectorCfg.quantization.rerankMultiplier ?? 3
|
||||
} : undefined,
|
||||
vectorStorage: vectorCfg?.vectorStorage
|
||||
rerankMultiplier: 3
|
||||
} : undefined
|
||||
}
|
||||
|
||||
const persistMode = this.resolveHNSWPersistMode()
|
||||
|
|
@ -9211,49 +9211,28 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
* Resolve HNSW persistence mode.
|
||||
* Extracted so both setupIndex() and the HNSW plugin factory path can use it.
|
||||
*
|
||||
* User config > smart default:
|
||||
* - Cloud storage (GCS/S3/R2/Azure): 'deferred' for 30-50x faster adds
|
||||
* - Local storage (FileSystem/Memory/OPFS): 'immediate' (already fast)
|
||||
* Brainy 8.0 ships filesystem-only; the cloud-storage smart-default
|
||||
* (deferred for cloud, immediate for local) collapses to "always
|
||||
* immediate unless the user overrides via config.vector.persistMode".
|
||||
*/
|
||||
private resolveHNSWPersistMode(): 'immediate' | 'deferred' {
|
||||
let persistMode: 'immediate' | 'deferred' = this.config.hnswPersistMode || 'immediate'
|
||||
|
||||
if (!this.config.hnswPersistMode) {
|
||||
const storageType = this.config.storage?.type || this.getStorageType()
|
||||
if (['gcs', 's3', 'r2', 'azure'].includes(storageType)) {
|
||||
persistMode = 'deferred'
|
||||
}
|
||||
}
|
||||
|
||||
return persistMode
|
||||
return this.config.vector?.persistMode ?? 'immediate'
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalize and validate configuration
|
||||
*/
|
||||
private normalizeConfig(config?: BrainyConfig): Required<BrainyConfig> {
|
||||
// Validate storage configuration
|
||||
if (config?.storage?.type && !['auto', 'memory', 'filesystem', 'opfs', 'remote', 's3', 'r2', 'gcs', 'gcs-native', 'azure'].includes(config.storage.type)) {
|
||||
throw new Error(`Invalid storage type: ${config.storage.type}. Must be one of: auto, memory, filesystem, opfs, remote, s3, r2, gcs, gcs-native, azure`)
|
||||
}
|
||||
|
||||
// Warn about deprecated gcs-native
|
||||
if (config?.storage?.type === ('gcs-native' as any)) {
|
||||
console.warn('⚠️ DEPRECATED: type "gcs-native" is deprecated. Use type "gcs" instead.')
|
||||
console.warn(' This will continue to work but may be removed in a future version.')
|
||||
}
|
||||
|
||||
// Validate storage type/config pairing (now more lenient)
|
||||
if (config?.storage) {
|
||||
const storage = config.storage as any
|
||||
|
||||
// Warn about legacy gcsStorage config with HMAC keys
|
||||
if (storage.gcsStorage && storage.gcsStorage.accessKeyId && storage.gcsStorage.secretAccessKey) {
|
||||
console.warn('⚠️ GCS with HMAC keys (gcsStorage) is legacy. Consider migrating to native GCS (gcsNativeStorage) with ADC.')
|
||||
}
|
||||
|
||||
// No longer throw errors for mismatches - storageFactory now handles this intelligently
|
||||
// Both 'gcs' and 'gcs-native' can now use either gcsStorage or gcsNativeStorage
|
||||
// Validate storage configuration. Brainy 8.0 ships two adapters only —
|
||||
// FileSystemStorage and MemoryStorage — per BR-BRAINY-80-STORAGE-SIMPLIFY.
|
||||
// Cloud backup remains supported via operator tooling (db.persist() +
|
||||
// gsutil / aws s3 cp / rclone / azcopy).
|
||||
if (config?.storage?.type && !['auto', 'memory', 'filesystem'].includes(config.storage.type)) {
|
||||
throw new Error(
|
||||
`Invalid storage type: ${config.storage.type}. Brainy 8.0 ships 'auto', 'memory', and 'filesystem' only. ` +
|
||||
`Cloud storage adapters (GCS / S3 / R2 / Azure) and OPFS were removed in 8.0; ` +
|
||||
`back up locally with db.persist() and sync the on-disk artefact with your tool of choice.`
|
||||
)
|
||||
}
|
||||
|
||||
// Validate numeric configurations
|
||||
|
|
@ -9292,10 +9271,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
// Memory management options
|
||||
maxQueryLimit: config?.maxQueryLimit ?? undefined as any,
|
||||
reservedQueryMemory: config?.reservedQueryMemory ?? undefined as any,
|
||||
// HNSW persistence mode - undefined = smart default in setupIndex
|
||||
hnswPersistMode: config?.hnswPersistMode ?? undefined as any,
|
||||
// Vector index configuration (8.0) — algorithm-neutral surface with
|
||||
// `recall` preset. See BRAINY-8.0-RENAME-COORDINATION § A.2.
|
||||
// `recall` preset + `persistMode` (folded in from 7.x's hnswPersistMode).
|
||||
// See BRAINY-8.0-RENAME-COORDINATION § A.2 + § G.2.
|
||||
vector: config?.vector ?? undefined as any,
|
||||
// Embedding initialization - false = lazy init on first embed()
|
||||
eagerEmbeddings: config?.eagerEmbeddings ?? false,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue