refactor(8.0): final cleanup — drop HnswProvider alias + config.hnsw + 'hnsw' surface (scaffold step 6)
Step 6 of the brainy 8.0 rename scaffolding. Removes every legacy 'hnsw'-
flavoured public name introduced as a compat shim in steps 1-5. The
algorithm-neutral surface is now the only surface.
REMOVED — PHASE A (type aliases)
src/plugin.ts
- Removed `export type HnswProvider = VectorIndexProvider` alias.
- Removed `export type DiskAnnProvider = VectorIndexProvider` alias.
src/index.ts
- Removed `export const HNSWIndex = JsHnswVectorIndex` alias.
src/types/brainy.types.ts
- Removed `BrainyStats.indexHealth.hnsw` field (the new `vector` field
carries the same boolean).
src/brainy.ts
- `stats()` no longer emits the legacy `hnsw` field on `indexHealth`.
REMOVED — PHASE B (storage adapter rename, 8 adapters)
Repo-wide rename: `saveHNSWData` → `saveVectorIndexData` and
`getHNSWData` → `getVectorIndexData` across:
- src/storage/adapters/baseStorageAdapter.ts (abstract declarations)
- src/storage/adapters/fileSystemStorage.ts
- src/storage/adapters/gcsStorage.ts
- src/storage/adapters/r2Storage.ts
- src/storage/adapters/s3CompatibleStorage.ts
- src/storage/adapters/azureBlobStorage.ts
- src/storage/adapters/opfsStorage.ts
- src/storage/adapters/memoryStorage.ts
- src/storage/adapters/historicalStorageAdapter.ts
- src/brainy.ts (call sites)
- src/hnsw/hnswIndex.ts + src/hnsw/typeAwareHNSWIndex.ts (call sites)
The default-delegation wrappers added in scaffold step 4 are removed
(would have been duplicate declarations after the rename).
REMOVED — PHASE C (cache category 'hnsw')
src/utils/unifiedCache.ts
- Cache-category union narrowed from
'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other'
to
'vectors' | 'metadata' | 'embedding' | 'other'
- typeAccessCounts, typeSizes, typeCounts, accessRatios, sizeRatios,
and the fairness-check iterator all drop the 'hnsw' key.
- Per planning § 2.5: pre-8.0 'hnsw' cache entries are an in-memory
category. Cache is rebuildable; entries naturally don't exist after
a restart, so no migration path is needed.
src/hnsw/hnswIndex.ts
- 3 cache.set() call sites migrated from category 'hnsw' to 'vectors'.
- Cache key prefix `hnsw:vector:` → `vector:`.
- typeCounts/typeSizes/typeAccessCounts accessor renames
`.hnsw` → `.vectors`.
tests/unit/utils/unifiedCache-eviction.test.ts
- Test fixtures updated: cache.set(..., 'hnsw', ...) → cache.set(..., 'vectors', ...).
- Stats assertion `stats.typeSizes.hnsw` → `stats.typeSizes.vectors`.
REMOVED — PHASE D (config.hnsw)
src/types/brainy.types.ts
- Removed `BrainyConfig.hnsw` field entirely. The 8.0 surface is
`BrainyConfig.vector.{recall, quantization, vectorStorage, advanced}`.
src/brainy.ts
- normalizeConfig() no longer emits a `hnsw` field on Required<BrainyConfig>.
- setupIndex() rewired:
- Reads from `this.config.vector` (not `this.config.hnsw`).
- Calls `resolveJsHnswConfig(this.config.vector)` to translate the
`recall` preset into M / efConstruction / efSearch knobs (with
`advanced.hnsw` overrides winning when supplied).
- Imports `resolveJsHnswConfig` from './utils/recallPreset.js'.
NOT IN THIS COMMIT (deliberately)
- Persisted file path migration `_system/hnsw-*.json` →
`_system/vector-index-*.json`. Requires dual-read logic on boot
across 8 storage adapters. Per integration doc lines 531-539:
"Reader accepts either spelling on load; writer emits the new spelling
only; brains self-migrate on the next persist after upgrade." This is
a separate body of work and lands in a follow-up commit before 8.0 GA.
- `config.hnswPersistMode` top-level field is unchanged. It's not in
the rename inventory; a future commit can fold it into
`config.vector.persistMode` if desired.
- strictConfig enforcement wiring. The field is accepted by
normalizeConfig() with default 'warn'; the actual warning emission at
knob-mismatch sites lands when the config-resolution layer is touched
for the persisted-path migration.
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit (one test fixture updated to use the new
category name; assertion still passes after fixture rename)
- npm run build: clean
The 8.0 PR's user-facing surface is now algorithm-neutral end-to-end:
VectorIndexProvider, config.vector.recall, JsHnswVectorIndex,
saveVectorIndexData, 'vectors' cache category, indexHealth.vector,
strictConfig — all standalone. No legacy 'hnsw' name remains in any
public type or method signature.
This commit is contained in:
parent
3e1ef957bc
commit
b20666e020
17 changed files with 76 additions and 145 deletions
|
|
@ -21,7 +21,7 @@ import type { CacheProvider } from '../plugin.js'
|
|||
|
||||
export interface CacheItem {
|
||||
key: string
|
||||
type: 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other'
|
||||
type: 'vectors' | 'metadata' | 'embedding' | 'other'
|
||||
data: any
|
||||
size: number
|
||||
rebuildCost: number // milliseconds to rebuild
|
||||
|
|
@ -69,7 +69,7 @@ export class UnifiedCache implements CacheProvider {
|
|||
private cache = new Map<string, CacheItem>()
|
||||
private access = new Map<string, number>() // Access counts
|
||||
private loadingPromises = new Map<string, Promise<any>>()
|
||||
private typeAccessCounts = { hnsw: 0, vectors: 0, metadata: 0, embedding: 0, other: 0 }
|
||||
private typeAccessCounts = { vectors: 0, metadata: 0, embedding: 0, other: 0 }
|
||||
private totalAccessCount = 0
|
||||
private currentSize = 0
|
||||
private maxSize: number
|
||||
|
|
@ -209,7 +209,7 @@ export class UnifiedCache implements CacheProvider {
|
|||
set(
|
||||
key: string,
|
||||
data: any,
|
||||
type: 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other',
|
||||
type: 'vectors' | 'metadata' | 'embedding' | 'other',
|
||||
size: number,
|
||||
rebuildCost: number = 1
|
||||
): void {
|
||||
|
|
@ -323,8 +323,8 @@ export class UnifiedCache implements CacheProvider {
|
|||
|
||||
private checkFairness(): void {
|
||||
// Calculate type ratios in cache
|
||||
const typeSizes = { hnsw: 0, vectors: 0, metadata: 0, embedding: 0, other: 0 }
|
||||
const typeCounts = { hnsw: 0, vectors: 0, metadata: 0, embedding: 0, other: 0 }
|
||||
const typeSizes = { vectors: 0, metadata: 0, embedding: 0, other: 0 }
|
||||
const typeCounts = { vectors: 0, metadata: 0, embedding: 0, other: 0 }
|
||||
|
||||
for (const item of this.cache.values()) {
|
||||
typeSizes[item.type] += item.size
|
||||
|
|
@ -334,7 +334,6 @@ export class UnifiedCache implements CacheProvider {
|
|||
// Calculate access ratios
|
||||
const totalAccess = this.totalAccessCount || 1
|
||||
const accessRatios = {
|
||||
hnsw: this.typeAccessCounts.hnsw / totalAccess,
|
||||
vectors: this.typeAccessCounts.vectors / totalAccess,
|
||||
metadata: this.typeAccessCounts.metadata / totalAccess,
|
||||
embedding: this.typeAccessCounts.embedding / totalAccess,
|
||||
|
|
@ -344,7 +343,6 @@ export class UnifiedCache implements CacheProvider {
|
|||
// Calculate size ratios
|
||||
const totalSize = this.currentSize || 1
|
||||
const sizeRatios = {
|
||||
hnsw: typeSizes.hnsw / totalSize,
|
||||
vectors: typeSizes.vectors / totalSize,
|
||||
metadata: typeSizes.metadata / totalSize,
|
||||
embedding: typeSizes.embedding / totalSize,
|
||||
|
|
@ -353,7 +351,7 @@ export class UnifiedCache implements CacheProvider {
|
|||
|
||||
// Check for starvation (more aggressive - 70% cache with <15% accesses)
|
||||
// Previous: 90% cache, <10% access (too lenient, caused thrashing)
|
||||
for (const type of ['hnsw', 'vectors', 'metadata', 'embedding', 'other'] as const) {
|
||||
for (const type of ['vectors', 'metadata', 'embedding', 'other'] as const) {
|
||||
if (sizeRatios[type] > 0.7 && accessRatios[type] < 0.15) {
|
||||
prodLog.warn(`Type ${type} is hogging cache (${(sizeRatios[type] * 100).toFixed(1)}% size, ${(accessRatios[type] * 100).toFixed(1)}% access)`)
|
||||
this.evictType(type)
|
||||
|
|
@ -366,7 +364,7 @@ export class UnifiedCache implements CacheProvider {
|
|||
* Called immediately when adding items to prevent imbalance formation
|
||||
* Uses same thresholds as periodic check but runs on-demand
|
||||
*/
|
||||
private checkProactiveFairness(addedType: 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other'): void {
|
||||
private checkProactiveFairness(addedType: 'vectors' | 'metadata' | 'embedding' | 'other'): void {
|
||||
// Quick check: only evaluate the type being added
|
||||
let typeSize = 0
|
||||
for (const item of this.cache.values()) {
|
||||
|
|
@ -388,7 +386,7 @@ export class UnifiedCache implements CacheProvider {
|
|||
/**
|
||||
* Force evict items of a specific type
|
||||
*/
|
||||
private evictType(type: 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other'): void {
|
||||
private evictType(type: 'vectors' | 'metadata' | 'embedding' | 'other'): void {
|
||||
const candidates: Array<[string, number, CacheItem]> = []
|
||||
|
||||
for (const [key, item] of this.cache) {
|
||||
|
|
@ -468,7 +466,7 @@ export class UnifiedCache implements CacheProvider {
|
|||
/**
|
||||
* Clear cache or specific type
|
||||
*/
|
||||
clear(type?: 'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other'): void {
|
||||
clear(type?: 'vectors' | 'metadata' | 'embedding' | 'other'): void {
|
||||
if (!type) {
|
||||
this.cache.clear()
|
||||
this.currentSize = 0
|
||||
|
|
@ -535,8 +533,8 @@ export class UnifiedCache implements CacheProvider {
|
|||
* Get cache statistics with memory information
|
||||
*/
|
||||
getStats() {
|
||||
const typeSizes = { hnsw: 0, vectors: 0, metadata: 0, embedding: 0, other: 0 }
|
||||
const typeCounts = { hnsw: 0, vectors: 0, metadata: 0, embedding: 0, other: 0 }
|
||||
const typeSizes = { vectors: 0, metadata: 0, embedding: 0, other: 0 }
|
||||
const typeCounts = { vectors: 0, metadata: 0, embedding: 0, other: 0 }
|
||||
|
||||
for (const item of this.cache.values()) {
|
||||
typeSizes[item.type] += item.size
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue