brainy/src/utils/recallPreset.ts

91 lines
3.4 KiB
TypeScript
Raw Normal View History

refactor(8.0): add config.vector path + 'vectors' cache category (scaffold step 2) Step 2 of the brainy 8.0 rename scaffolding. Adds the new algorithm-neutral config surface alongside the legacy config.hnsw path (which becomes a compat shim removed in the final cleanup commit). CHANGES src/utils/recallPreset.ts (NEW) - DEFAULT_RECALL = 'balanced' - HNSW_PRESETS table — fast/balanced/accurate → (M, efConstruction, efSearch). 'balanced' equals brainy 7.x DEFAULT_CONFIG byte-for-byte so a 7.x → 8.0 upgrade with no explicit recall value is a no-op. - resolveRecallHnswKnobs(preset) — preset → knobs. - resolveJsHnswConfig(vectorConfig) — preset first, then `advanced.hnsw` overrides win. Explicit knobs always beat the preset. src/types/brainy.types.ts - New optional `vector` field on BrainyConfig: vector?: { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4, rerankMultiplier? } vectorStorage?: 'memory' | 'lazy' advanced?: { hnsw?: { M?, efConstruction?, efSearch?, ml? } diskann?: { pqM?, pqKsub?, maxDegree?, searchListSize?, alpha?, ... } } } - `hnsw` field marked @deprecated with note that it stays as a compat shim during the 8.0 rename and is removed in the final cleanup commit. src/utils/unifiedCache.ts - Cache-category union widened from 'hnsw' | 'metadata' | 'embedding' | 'other' to 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other' - 5 union sites + 4 typed-counter maps (typeAccessCounts, checkFairness typeSizes/typeCounts/accessRatios/sizeRatios, getStats typeSizes/typeCounts, evictType loop) all gained the 'vectors' key with initial value 0. - The hard-coded fairness-check loop now iterates both keys. - Final 8.0 cleanup will narrow back to 'vectors' | 'metadata' | 'embedding' | 'other'. src/brainy.ts (normalizeConfig) - Adds the new `vector` field to the Required<BrainyConfig> return shape so TS compiles. Reads config?.vector ?? undefined as any (same pattern as the other optional config fields). NO-OP scope No behavioural change. The new config.vector path is silently ignored at runtime in this commit — wired up in step 4 (storage + index resolution). Pure surface plumbing. Tests + build green. VERIFICATION - npx tsc --noEmit: clean - npm test: 1468 / 1468 unit
2026-06-09 13:06:10 -07:00
/**
* @module utils/recallPreset
* @description Maps Brainy 8.0's algorithm-neutral `recall` preset to the
* underlying index-specific knobs. The same user-facing word (`'fast'` /
* `'balanced'` / `'accurate'`) means "the right thing happens" whether
* Brainy's open-core JS HNSW path is in play or a native acceleration
* provider (DiskANN-style) has taken over the `'vector'` provider key.
*
* **Public contract** locked in handoff thread BRAINY-8.0-RENAME-COORDINATION
* § A.2 (cortex-confirmed 2026-06-09):
* - `'fast'` minimum-latency search, accepts lower recall
* - `'balanced'` Brainy 7.x defaults, the right pick for almost everyone
* - `'accurate'` maximum recall, accepts higher latency
*
* Default when `recall` is omitted: `'balanced'`.
*
* **Knob mapping** the JS HNSW preset values below match what Brainy 7.x
* shipped as defaults for `'balanced'`. Native acceleration providers (e.g.
* cortex's DiskANN wrapper) read the same `recall` value off the index
* config and translate it to their own internal knobs.
*/
import type { HNSWConfig } from '../coreTypes.js'
export type RecallPreset = 'fast' | 'balanced' | 'accurate'
/**
* The default preset when `config.vector.recall` is omitted. Matches Brainy
* 7.x's historical HNSW defaults exactly (`M=16`, `efConstruction=200`,
* `efSearch=50`).
*/
export const DEFAULT_RECALL: RecallPreset = 'balanced'
/**
refactor(8.0): simplify config.vector to 3 knobs + fold persistMode (scaffold step 8) Per user direction and BRAINY-8.0-RENAME-COORDINATION § G.2: the `config.vector.advanced.{hnsw, diskann}` escape-hatch surface was over-engineered. The 8.0 config surface collapses to **three knobs**: ``` config.vector = { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4 } persistMode?: 'immediate' | 'deferred' } ``` The algorithm-internal HNSW knobs (`M`, `efConstruction`, `efSearch`, `ml`) and DiskANN knobs (`pqM`, `searchListSize`, `alpha`, …) are no longer exposed at the public surface. The `recall` preset covers the legitimate quality/latency tradeoff; algorithm-internal tuning is intentionally hidden. If users need it later, we can add it back — shipping with too few knobs is easier to evolve than shipping with too many. CHANGES src/types/brainy.types.ts - BrainyConfig.hnswPersistMode (7.x top-level) → BrainyConfig.vector.persistMode. - BrainyConfig.vector — dropped `advanced.{hnsw, diskann}` sub-block. - BrainyConfig.vector.quantization — dropped `rerankMultiplier` (hardcoded to 3). - BrainyConfig.vector — dropped `vectorStorage: 'memory' | 'lazy'`. Brainy always keeps vectors resident in 8.0; the lazy-eviction path was a cloud-storage optimisation that's no longer needed (cloud adapters are gone). - JSDoc tightened to reflect the closed-form contract. src/utils/recallPreset.ts - resolveJsHnswConfig() signature narrowed: accepts `{ recall? }` only (no more `advanced` parameter). src/brainy.ts - normalizeConfig() — no longer carries `hnswPersistMode` on the resolved config. Dropped the deprecated 'gcs-native' warning, the gcsStorage HMAC-key warning, and the lenient storage-pairing block (dead code now that cloud adapters are gone). - resolveHNSWPersistMode() — collapsed to a one-liner: `return this.config.vector?.persistMode ?? 'immediate'`. The cloud-vs-local smart-default branching is no longer relevant (filesystem is the only persistent backend in 8.0). - setupIndex() — reads quantization fields from config.vector directly; rerankMultiplier hardcoded to 3. NO-OP scope The recall preset's 'balanced' = brainy 7.x DEFAULT_CONFIG byte-for-byte, so users who don't set recall get the same behavior. Users who set config.hnsw.* in 7.x will need to migrate to config.vector.* per the 8.0 release notes; pure 7.x defaults users see no change. VERIFICATION - npx tsc --noEmit: clean - npm test: 1408 / 1409 (same pre-existing test-isolation race condition from step 7, not related to step 8)
2026-06-09 14:20:57 -07:00
* The set of HNSW knobs that the `recall` preset controls. Brainy 8.0 does
* not expose these directly the preset is the only quality/latency knob
* surfaced on the public config. Algorithm-internal tuning is intentionally
* hidden behind the preset.
refactor(8.0): add config.vector path + 'vectors' cache category (scaffold step 2) Step 2 of the brainy 8.0 rename scaffolding. Adds the new algorithm-neutral config surface alongside the legacy config.hnsw path (which becomes a compat shim removed in the final cleanup commit). CHANGES src/utils/recallPreset.ts (NEW) - DEFAULT_RECALL = 'balanced' - HNSW_PRESETS table — fast/balanced/accurate → (M, efConstruction, efSearch). 'balanced' equals brainy 7.x DEFAULT_CONFIG byte-for-byte so a 7.x → 8.0 upgrade with no explicit recall value is a no-op. - resolveRecallHnswKnobs(preset) — preset → knobs. - resolveJsHnswConfig(vectorConfig) — preset first, then `advanced.hnsw` overrides win. Explicit knobs always beat the preset. src/types/brainy.types.ts - New optional `vector` field on BrainyConfig: vector?: { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4, rerankMultiplier? } vectorStorage?: 'memory' | 'lazy' advanced?: { hnsw?: { M?, efConstruction?, efSearch?, ml? } diskann?: { pqM?, pqKsub?, maxDegree?, searchListSize?, alpha?, ... } } } - `hnsw` field marked @deprecated with note that it stays as a compat shim during the 8.0 rename and is removed in the final cleanup commit. src/utils/unifiedCache.ts - Cache-category union widened from 'hnsw' | 'metadata' | 'embedding' | 'other' to 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other' - 5 union sites + 4 typed-counter maps (typeAccessCounts, checkFairness typeSizes/typeCounts/accessRatios/sizeRatios, getStats typeSizes/typeCounts, evictType loop) all gained the 'vectors' key with initial value 0. - The hard-coded fairness-check loop now iterates both keys. - Final 8.0 cleanup will narrow back to 'vectors' | 'metadata' | 'embedding' | 'other'. src/brainy.ts (normalizeConfig) - Adds the new `vector` field to the Required<BrainyConfig> return shape so TS compiles. Reads config?.vector ?? undefined as any (same pattern as the other optional config fields). NO-OP scope No behavioural change. The new config.vector path is silently ignored at runtime in this commit — wired up in step 4 (storage + index resolution). Pure surface plumbing. Tests + build green. VERIFICATION - npx tsc --noEmit: clean - npm test: 1468 / 1468 unit
2026-06-09 13:06:10 -07:00
*/
export interface RecallHnswKnobs {
M: number
efConstruction: number
efSearch: number
}
/**
* Preset JS HNSW knob tuples. The `'balanced'` entry equals Brainy 7.x's
* `DEFAULT_CONFIG` so a 7.x 8.0 upgrade with no explicit `recall` value is
* a no-op.
*/
const HNSW_PRESETS: Readonly<Record<RecallPreset, Readonly<RecallHnswKnobs>>> = {
fast: { M: 16, efConstruction: 100, efSearch: 30 },
balanced: { M: 16, efConstruction: 200, efSearch: 50 },
accurate: { M: 32, efConstruction: 400, efSearch: 100 },
}
/**
* Resolve a preset name (or `undefined` for the default) to its underlying
* HNSW knob tuple.
*
* @param preset - The preset name, or `undefined` to use {@link DEFAULT_RECALL}.
* @returns The HNSW knob values for the preset.
*/
export function resolveRecallHnswKnobs(preset: RecallPreset | undefined): RecallHnswKnobs {
return HNSW_PRESETS[preset ?? DEFAULT_RECALL]
}
/**
* Resolve a Brainy 8.0 `VectorIndexConfig` into a concrete `HNSWConfig` for
refactor(8.0): simplify config.vector to 3 knobs + fold persistMode (scaffold step 8) Per user direction and BRAINY-8.0-RENAME-COORDINATION § G.2: the `config.vector.advanced.{hnsw, diskann}` escape-hatch surface was over-engineered. The 8.0 config surface collapses to **three knobs**: ``` config.vector = { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4 } persistMode?: 'immediate' | 'deferred' } ``` The algorithm-internal HNSW knobs (`M`, `efConstruction`, `efSearch`, `ml`) and DiskANN knobs (`pqM`, `searchListSize`, `alpha`, …) are no longer exposed at the public surface. The `recall` preset covers the legitimate quality/latency tradeoff; algorithm-internal tuning is intentionally hidden. If users need it later, we can add it back — shipping with too few knobs is easier to evolve than shipping with too many. CHANGES src/types/brainy.types.ts - BrainyConfig.hnswPersistMode (7.x top-level) → BrainyConfig.vector.persistMode. - BrainyConfig.vector — dropped `advanced.{hnsw, diskann}` sub-block. - BrainyConfig.vector.quantization — dropped `rerankMultiplier` (hardcoded to 3). - BrainyConfig.vector — dropped `vectorStorage: 'memory' | 'lazy'`. Brainy always keeps vectors resident in 8.0; the lazy-eviction path was a cloud-storage optimisation that's no longer needed (cloud adapters are gone). - JSDoc tightened to reflect the closed-form contract. src/utils/recallPreset.ts - resolveJsHnswConfig() signature narrowed: accepts `{ recall? }` only (no more `advanced` parameter). src/brainy.ts - normalizeConfig() — no longer carries `hnswPersistMode` on the resolved config. Dropped the deprecated 'gcs-native' warning, the gcsStorage HMAC-key warning, and the lenient storage-pairing block (dead code now that cloud adapters are gone). - resolveHNSWPersistMode() — collapsed to a one-liner: `return this.config.vector?.persistMode ?? 'immediate'`. The cloud-vs-local smart-default branching is no longer relevant (filesystem is the only persistent backend in 8.0). - setupIndex() — reads quantization fields from config.vector directly; rerankMultiplier hardcoded to 3. NO-OP scope The recall preset's 'balanced' = brainy 7.x DEFAULT_CONFIG byte-for-byte, so users who don't set recall get the same behavior. Users who set config.hnsw.* in 7.x will need to migrate to config.vector.* per the 8.0 release notes; pure 7.x defaults users see no change. VERIFICATION - npx tsc --noEmit: clean - npm test: 1408 / 1409 (same pre-existing test-isolation race condition from step 7, not related to step 8)
2026-06-09 14:20:57 -07:00
* the open-core JS HNSW path. Applies the recall preset.
refactor(8.0): add config.vector path + 'vectors' cache category (scaffold step 2) Step 2 of the brainy 8.0 rename scaffolding. Adds the new algorithm-neutral config surface alongside the legacy config.hnsw path (which becomes a compat shim removed in the final cleanup commit). CHANGES src/utils/recallPreset.ts (NEW) - DEFAULT_RECALL = 'balanced' - HNSW_PRESETS table — fast/balanced/accurate → (M, efConstruction, efSearch). 'balanced' equals brainy 7.x DEFAULT_CONFIG byte-for-byte so a 7.x → 8.0 upgrade with no explicit recall value is a no-op. - resolveRecallHnswKnobs(preset) — preset → knobs. - resolveJsHnswConfig(vectorConfig) — preset first, then `advanced.hnsw` overrides win. Explicit knobs always beat the preset. src/types/brainy.types.ts - New optional `vector` field on BrainyConfig: vector?: { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4, rerankMultiplier? } vectorStorage?: 'memory' | 'lazy' advanced?: { hnsw?: { M?, efConstruction?, efSearch?, ml? } diskann?: { pqM?, pqKsub?, maxDegree?, searchListSize?, alpha?, ... } } } - `hnsw` field marked @deprecated with note that it stays as a compat shim during the 8.0 rename and is removed in the final cleanup commit. src/utils/unifiedCache.ts - Cache-category union widened from 'hnsw' | 'metadata' | 'embedding' | 'other' to 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other' - 5 union sites + 4 typed-counter maps (typeAccessCounts, checkFairness typeSizes/typeCounts/accessRatios/sizeRatios, getStats typeSizes/typeCounts, evictType loop) all gained the 'vectors' key with initial value 0. - The hard-coded fairness-check loop now iterates both keys. - Final 8.0 cleanup will narrow back to 'vectors' | 'metadata' | 'embedding' | 'other'. src/brainy.ts (normalizeConfig) - Adds the new `vector` field to the Required<BrainyConfig> return shape so TS compiles. Reads config?.vector ?? undefined as any (same pattern as the other optional config fields). NO-OP scope No behavioural change. The new config.vector path is silently ignored at runtime in this commit — wired up in step 4 (storage + index resolution). Pure surface plumbing. Tests + build green. VERIFICATION - npx tsc --noEmit: clean - npm test: 1468 / 1468 unit
2026-06-09 13:06:10 -07:00
*
* Native acceleration providers do NOT use this function they read
* `config.vector.recall` directly and apply their own preset table. This
* helper exists for Brainy's own JS HNSW implementation only.
*
* @param vectorConfig - The user-supplied `config.vector` block. May be omitted.
* @returns A complete `HNSWConfig` with `M`, `efConstruction`, `efSearch`, `ml`
refactor(8.0): simplify config.vector to 3 knobs + fold persistMode (scaffold step 8) Per user direction and BRAINY-8.0-RENAME-COORDINATION § G.2: the `config.vector.advanced.{hnsw, diskann}` escape-hatch surface was over-engineered. The 8.0 config surface collapses to **three knobs**: ``` config.vector = { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4 } persistMode?: 'immediate' | 'deferred' } ``` The algorithm-internal HNSW knobs (`M`, `efConstruction`, `efSearch`, `ml`) and DiskANN knobs (`pqM`, `searchListSize`, `alpha`, …) are no longer exposed at the public surface. The `recall` preset covers the legitimate quality/latency tradeoff; algorithm-internal tuning is intentionally hidden. If users need it later, we can add it back — shipping with too few knobs is easier to evolve than shipping with too many. CHANGES src/types/brainy.types.ts - BrainyConfig.hnswPersistMode (7.x top-level) → BrainyConfig.vector.persistMode. - BrainyConfig.vector — dropped `advanced.{hnsw, diskann}` sub-block. - BrainyConfig.vector.quantization — dropped `rerankMultiplier` (hardcoded to 3). - BrainyConfig.vector — dropped `vectorStorage: 'memory' | 'lazy'`. Brainy always keeps vectors resident in 8.0; the lazy-eviction path was a cloud-storage optimisation that's no longer needed (cloud adapters are gone). - JSDoc tightened to reflect the closed-form contract. src/utils/recallPreset.ts - resolveJsHnswConfig() signature narrowed: accepts `{ recall? }` only (no more `advanced` parameter). src/brainy.ts - normalizeConfig() — no longer carries `hnswPersistMode` on the resolved config. Dropped the deprecated 'gcs-native' warning, the gcsStorage HMAC-key warning, and the lenient storage-pairing block (dead code now that cloud adapters are gone). - resolveHNSWPersistMode() — collapsed to a one-liner: `return this.config.vector?.persistMode ?? 'immediate'`. The cloud-vs-local smart-default branching is no longer relevant (filesystem is the only persistent backend in 8.0). - setupIndex() — reads quantization fields from config.vector directly; rerankMultiplier hardcoded to 3. NO-OP scope The recall preset's 'balanced' = brainy 7.x DEFAULT_CONFIG byte-for-byte, so users who don't set recall get the same behavior. Users who set config.hnsw.* in 7.x will need to migrate to config.vector.* per the 8.0 release notes; pure 7.x defaults users see no change. VERIFICATION - npx tsc --noEmit: clean - npm test: 1408 / 1409 (same pre-existing test-isolation race condition from step 7, not related to step 8)
2026-06-09 14:20:57 -07:00
* populated from the preset.
refactor(8.0): add config.vector path + 'vectors' cache category (scaffold step 2) Step 2 of the brainy 8.0 rename scaffolding. Adds the new algorithm-neutral config surface alongside the legacy config.hnsw path (which becomes a compat shim removed in the final cleanup commit). CHANGES src/utils/recallPreset.ts (NEW) - DEFAULT_RECALL = 'balanced' - HNSW_PRESETS table — fast/balanced/accurate → (M, efConstruction, efSearch). 'balanced' equals brainy 7.x DEFAULT_CONFIG byte-for-byte so a 7.x → 8.0 upgrade with no explicit recall value is a no-op. - resolveRecallHnswKnobs(preset) — preset → knobs. - resolveJsHnswConfig(vectorConfig) — preset first, then `advanced.hnsw` overrides win. Explicit knobs always beat the preset. src/types/brainy.types.ts - New optional `vector` field on BrainyConfig: vector?: { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4, rerankMultiplier? } vectorStorage?: 'memory' | 'lazy' advanced?: { hnsw?: { M?, efConstruction?, efSearch?, ml? } diskann?: { pqM?, pqKsub?, maxDegree?, searchListSize?, alpha?, ... } } } - `hnsw` field marked @deprecated with note that it stays as a compat shim during the 8.0 rename and is removed in the final cleanup commit. src/utils/unifiedCache.ts - Cache-category union widened from 'hnsw' | 'metadata' | 'embedding' | 'other' to 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other' - 5 union sites + 4 typed-counter maps (typeAccessCounts, checkFairness typeSizes/typeCounts/accessRatios/sizeRatios, getStats typeSizes/typeCounts, evictType loop) all gained the 'vectors' key with initial value 0. - The hard-coded fairness-check loop now iterates both keys. - Final 8.0 cleanup will narrow back to 'vectors' | 'metadata' | 'embedding' | 'other'. src/brainy.ts (normalizeConfig) - Adds the new `vector` field to the Required<BrainyConfig> return shape so TS compiles. Reads config?.vector ?? undefined as any (same pattern as the other optional config fields). NO-OP scope No behavioural change. The new config.vector path is silently ignored at runtime in this commit — wired up in step 4 (storage + index resolution). Pure surface plumbing. Tests + build green. VERIFICATION - npx tsc --noEmit: clean - npm test: 1468 / 1468 unit
2026-06-09 13:06:10 -07:00
*/
export function resolveJsHnswConfig(
refactor(8.0): simplify config.vector to 3 knobs + fold persistMode (scaffold step 8) Per user direction and BRAINY-8.0-RENAME-COORDINATION § G.2: the `config.vector.advanced.{hnsw, diskann}` escape-hatch surface was over-engineered. The 8.0 config surface collapses to **three knobs**: ``` config.vector = { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4 } persistMode?: 'immediate' | 'deferred' } ``` The algorithm-internal HNSW knobs (`M`, `efConstruction`, `efSearch`, `ml`) and DiskANN knobs (`pqM`, `searchListSize`, `alpha`, …) are no longer exposed at the public surface. The `recall` preset covers the legitimate quality/latency tradeoff; algorithm-internal tuning is intentionally hidden. If users need it later, we can add it back — shipping with too few knobs is easier to evolve than shipping with too many. CHANGES src/types/brainy.types.ts - BrainyConfig.hnswPersistMode (7.x top-level) → BrainyConfig.vector.persistMode. - BrainyConfig.vector — dropped `advanced.{hnsw, diskann}` sub-block. - BrainyConfig.vector.quantization — dropped `rerankMultiplier` (hardcoded to 3). - BrainyConfig.vector — dropped `vectorStorage: 'memory' | 'lazy'`. Brainy always keeps vectors resident in 8.0; the lazy-eviction path was a cloud-storage optimisation that's no longer needed (cloud adapters are gone). - JSDoc tightened to reflect the closed-form contract. src/utils/recallPreset.ts - resolveJsHnswConfig() signature narrowed: accepts `{ recall? }` only (no more `advanced` parameter). src/brainy.ts - normalizeConfig() — no longer carries `hnswPersistMode` on the resolved config. Dropped the deprecated 'gcs-native' warning, the gcsStorage HMAC-key warning, and the lenient storage-pairing block (dead code now that cloud adapters are gone). - resolveHNSWPersistMode() — collapsed to a one-liner: `return this.config.vector?.persistMode ?? 'immediate'`. The cloud-vs-local smart-default branching is no longer relevant (filesystem is the only persistent backend in 8.0). - setupIndex() — reads quantization fields from config.vector directly; rerankMultiplier hardcoded to 3. NO-OP scope The recall preset's 'balanced' = brainy 7.x DEFAULT_CONFIG byte-for-byte, so users who don't set recall get the same behavior. Users who set config.hnsw.* in 7.x will need to migrate to config.vector.* per the 8.0 release notes; pure 7.x defaults users see no change. VERIFICATION - npx tsc --noEmit: clean - npm test: 1408 / 1409 (same pre-existing test-isolation race condition from step 7, not related to step 8)
2026-06-09 14:20:57 -07:00
vectorConfig?: { recall?: RecallPreset }
refactor(8.0): add config.vector path + 'vectors' cache category (scaffold step 2) Step 2 of the brainy 8.0 rename scaffolding. Adds the new algorithm-neutral config surface alongside the legacy config.hnsw path (which becomes a compat shim removed in the final cleanup commit). CHANGES src/utils/recallPreset.ts (NEW) - DEFAULT_RECALL = 'balanced' - HNSW_PRESETS table — fast/balanced/accurate → (M, efConstruction, efSearch). 'balanced' equals brainy 7.x DEFAULT_CONFIG byte-for-byte so a 7.x → 8.0 upgrade with no explicit recall value is a no-op. - resolveRecallHnswKnobs(preset) — preset → knobs. - resolveJsHnswConfig(vectorConfig) — preset first, then `advanced.hnsw` overrides win. Explicit knobs always beat the preset. src/types/brainy.types.ts - New optional `vector` field on BrainyConfig: vector?: { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4, rerankMultiplier? } vectorStorage?: 'memory' | 'lazy' advanced?: { hnsw?: { M?, efConstruction?, efSearch?, ml? } diskann?: { pqM?, pqKsub?, maxDegree?, searchListSize?, alpha?, ... } } } - `hnsw` field marked @deprecated with note that it stays as a compat shim during the 8.0 rename and is removed in the final cleanup commit. src/utils/unifiedCache.ts - Cache-category union widened from 'hnsw' | 'metadata' | 'embedding' | 'other' to 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other' - 5 union sites + 4 typed-counter maps (typeAccessCounts, checkFairness typeSizes/typeCounts/accessRatios/sizeRatios, getStats typeSizes/typeCounts, evictType loop) all gained the 'vectors' key with initial value 0. - The hard-coded fairness-check loop now iterates both keys. - Final 8.0 cleanup will narrow back to 'vectors' | 'metadata' | 'embedding' | 'other'. src/brainy.ts (normalizeConfig) - Adds the new `vector` field to the Required<BrainyConfig> return shape so TS compiles. Reads config?.vector ?? undefined as any (same pattern as the other optional config fields). NO-OP scope No behavioural change. The new config.vector path is silently ignored at runtime in this commit — wired up in step 4 (storage + index resolution). Pure surface plumbing. Tests + build green. VERIFICATION - npx tsc --noEmit: clean - npm test: 1468 / 1468 unit
2026-06-09 13:06:10 -07:00
): Pick<HNSWConfig, 'M' | 'efConstruction' | 'efSearch' | 'ml'> {
const preset = resolveRecallHnswKnobs(vectorConfig?.recall)
return {
refactor(8.0): simplify config.vector to 3 knobs + fold persistMode (scaffold step 8) Per user direction and BRAINY-8.0-RENAME-COORDINATION § G.2: the `config.vector.advanced.{hnsw, diskann}` escape-hatch surface was over-engineered. The 8.0 config surface collapses to **three knobs**: ``` config.vector = { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4 } persistMode?: 'immediate' | 'deferred' } ``` The algorithm-internal HNSW knobs (`M`, `efConstruction`, `efSearch`, `ml`) and DiskANN knobs (`pqM`, `searchListSize`, `alpha`, …) are no longer exposed at the public surface. The `recall` preset covers the legitimate quality/latency tradeoff; algorithm-internal tuning is intentionally hidden. If users need it later, we can add it back — shipping with too few knobs is easier to evolve than shipping with too many. CHANGES src/types/brainy.types.ts - BrainyConfig.hnswPersistMode (7.x top-level) → BrainyConfig.vector.persistMode. - BrainyConfig.vector — dropped `advanced.{hnsw, diskann}` sub-block. - BrainyConfig.vector.quantization — dropped `rerankMultiplier` (hardcoded to 3). - BrainyConfig.vector — dropped `vectorStorage: 'memory' | 'lazy'`. Brainy always keeps vectors resident in 8.0; the lazy-eviction path was a cloud-storage optimisation that's no longer needed (cloud adapters are gone). - JSDoc tightened to reflect the closed-form contract. src/utils/recallPreset.ts - resolveJsHnswConfig() signature narrowed: accepts `{ recall? }` only (no more `advanced` parameter). src/brainy.ts - normalizeConfig() — no longer carries `hnswPersistMode` on the resolved config. Dropped the deprecated 'gcs-native' warning, the gcsStorage HMAC-key warning, and the lenient storage-pairing block (dead code now that cloud adapters are gone). - resolveHNSWPersistMode() — collapsed to a one-liner: `return this.config.vector?.persistMode ?? 'immediate'`. The cloud-vs-local smart-default branching is no longer relevant (filesystem is the only persistent backend in 8.0). - setupIndex() — reads quantization fields from config.vector directly; rerankMultiplier hardcoded to 3. NO-OP scope The recall preset's 'balanced' = brainy 7.x DEFAULT_CONFIG byte-for-byte, so users who don't set recall get the same behavior. Users who set config.hnsw.* in 7.x will need to migrate to config.vector.* per the 8.0 release notes; pure 7.x defaults users see no change. VERIFICATION - npx tsc --noEmit: clean - npm test: 1408 / 1409 (same pre-existing test-isolation race condition from step 7, not related to step 8)
2026-06-09 14:20:57 -07:00
M: preset.M,
efConstruction: preset.efConstruction,
efSearch: preset.efSearch,
ml: 16,
refactor(8.0): add config.vector path + 'vectors' cache category (scaffold step 2) Step 2 of the brainy 8.0 rename scaffolding. Adds the new algorithm-neutral config surface alongside the legacy config.hnsw path (which becomes a compat shim removed in the final cleanup commit). CHANGES src/utils/recallPreset.ts (NEW) - DEFAULT_RECALL = 'balanced' - HNSW_PRESETS table — fast/balanced/accurate → (M, efConstruction, efSearch). 'balanced' equals brainy 7.x DEFAULT_CONFIG byte-for-byte so a 7.x → 8.0 upgrade with no explicit recall value is a no-op. - resolveRecallHnswKnobs(preset) — preset → knobs. - resolveJsHnswConfig(vectorConfig) — preset first, then `advanced.hnsw` overrides win. Explicit knobs always beat the preset. src/types/brainy.types.ts - New optional `vector` field on BrainyConfig: vector?: { recall?: 'fast' | 'balanced' | 'accurate' quantization?: { enabled?, bits?: 8 | 4, rerankMultiplier? } vectorStorage?: 'memory' | 'lazy' advanced?: { hnsw?: { M?, efConstruction?, efSearch?, ml? } diskann?: { pqM?, pqKsub?, maxDegree?, searchListSize?, alpha?, ... } } } - `hnsw` field marked @deprecated with note that it stays as a compat shim during the 8.0 rename and is removed in the final cleanup commit. src/utils/unifiedCache.ts - Cache-category union widened from 'hnsw' | 'metadata' | 'embedding' | 'other' to 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other' - 5 union sites + 4 typed-counter maps (typeAccessCounts, checkFairness typeSizes/typeCounts/accessRatios/sizeRatios, getStats typeSizes/typeCounts, evictType loop) all gained the 'vectors' key with initial value 0. - The hard-coded fairness-check loop now iterates both keys. - Final 8.0 cleanup will narrow back to 'vectors' | 'metadata' | 'embedding' | 'other'. src/brainy.ts (normalizeConfig) - Adds the new `vector` field to the Required<BrainyConfig> return shape so TS compiles. Reads config?.vector ?? undefined as any (same pattern as the other optional config fields). NO-OP scope No behavioural change. The new config.vector path is silently ignored at runtime in this commit — wired up in step 4 (storage + index resolution). Pure surface plumbing. Tests + build green. VERIFICATION - npx tsc --noEmit: clean - npm test: 1468 / 1468 unit
2026-06-09 13:06:10 -07:00
}
}