refactor(8.0): final cleanup — drop HnswProvider alias + config.hnsw + 'hnsw' surface (scaffold step 6)
Step 6 of the brainy 8.0 rename scaffolding. Removes every legacy 'hnsw'-
flavoured public name introduced as a compat shim in steps 1-5. The
algorithm-neutral surface is now the only surface.
REMOVED — PHASE A (type aliases)
src/plugin.ts
- Removed `export type HnswProvider = VectorIndexProvider` alias.
- Removed `export type DiskAnnProvider = VectorIndexProvider` alias.
src/index.ts
- Removed `export const HNSWIndex = JsHnswVectorIndex` alias.
src/types/brainy.types.ts
- Removed `BrainyStats.indexHealth.hnsw` field (the new `vector` field
carries the same boolean).
src/brainy.ts
- `stats()` no longer emits the legacy `hnsw` field on `indexHealth`.
REMOVED — PHASE B (storage adapter rename, 8 adapters)
Repo-wide rename: `saveHNSWData` → `saveVectorIndexData` and
`getHNSWData` → `getVectorIndexData` across:
- src/storage/adapters/baseStorageAdapter.ts (abstract declarations)
- src/storage/adapters/fileSystemStorage.ts
- src/storage/adapters/gcsStorage.ts
- src/storage/adapters/r2Storage.ts
- src/storage/adapters/s3CompatibleStorage.ts
- src/storage/adapters/azureBlobStorage.ts
- src/storage/adapters/opfsStorage.ts
- src/storage/adapters/memoryStorage.ts
- src/storage/adapters/historicalStorageAdapter.ts
- src/brainy.ts (call sites)
- src/hnsw/hnswIndex.ts + src/hnsw/typeAwareHNSWIndex.ts (call sites)
The default-delegation wrappers added in scaffold step 4 are removed
(would have been duplicate declarations after the rename).
REMOVED — PHASE C (cache category 'hnsw')
src/utils/unifiedCache.ts
- Cache-category union narrowed from
'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other'
to
'vectors' | 'metadata' | 'embedding' | 'other'
- typeAccessCounts, typeSizes, typeCounts, accessRatios, sizeRatios,
and the fairness-check iterator all drop the 'hnsw' key.
- Per planning § 2.5: pre-8.0 'hnsw' cache entries are an in-memory
category. Cache is rebuildable; entries naturally don't exist after
a restart, so no migration path is needed.
src/hnsw/hnswIndex.ts
- 3 cache.set() call sites migrated from category 'hnsw' to 'vectors'.
- Cache key prefix `hnsw:vector:` → `vector:`.
- typeCounts/typeSizes/typeAccessCounts accessor renames
`.hnsw` → `.vectors`.
tests/unit/utils/unifiedCache-eviction.test.ts
- Test fixtures updated: cache.set(..., 'hnsw', ...) → cache.set(..., 'vectors', ...).
- Stats assertion `stats.typeSizes.hnsw` → `stats.typeSizes.vectors`.
REMOVED — PHASE D (config.hnsw)
src/types/brainy.types.ts
- Removed `BrainyConfig.hnsw` field entirely. The 8.0 surface is
`BrainyConfig.vector.{recall, quantization, vectorStorage, advanced}`.
src/brainy.ts
- normalizeConfig() no longer emits a `hnsw` field on Required<BrainyConfig>.
- setupIndex() rewired:
- Reads from `this.config.vector` (not `this.config.hnsw`).
- Calls `resolveJsHnswConfig(this.config.vector)` to translate the
`recall` preset into M / efConstruction / efSearch knobs (with
`advanced.hnsw` overrides winning when supplied).
- Imports `resolveJsHnswConfig` from './utils/recallPreset.js'.
NOT IN THIS COMMIT (deliberately)
- Persisted file path migration `_system/hnsw-*.json` →
`_system/vector-index-*.json`. Requires dual-read logic on boot
across 8 storage adapters. Per integration doc lines 531-539:
"Reader accepts either spelling on load; writer emits the new spelling
only; brains self-migrate on the next persist after upgrade." This is
a separate body of work and lands in a follow-up commit before 8.0 GA.
- `config.hnswPersistMode` top-level field is unchanged. It's not in
the rename inventory; a future commit can fold it into
`config.vector.persistMode` if desired.
- strictConfig enforcement wiring. The field is accepted by
normalizeConfig() with default 'warn'; the actual warning emission at
knob-mismatch sites lands when the config-resolution layer is touched
for the persisted-path migration.
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit (one test fixture updated to use the new
category name; assertion still passes after fixture rename)
- npm run build: clean
The 8.0 PR's user-facing surface is now algorithm-neutral end-to-end:
VectorIndexProvider, config.vector.recall, JsHnswVectorIndex,
saveVectorIndexData, 'vectors' cache category, indexHealth.vector,
strictConfig — all standalone. No legacy 'hnsw' name remains in any
public type or method signature.
This commit is contained in:
parent
3e1ef957bc
commit
b20666e020
17 changed files with 76 additions and 145 deletions
|
|
@ -61,7 +61,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
// Optional connections codec (2.4.0 #3). When set, node-persist writes the
|
||||
// node's per-level connection sets as a single delta-varint-compressed
|
||||
// binary blob (typically ~4× smaller than the legacy JSON-UUID-array
|
||||
// shape), plus an empty `connections: {}` in saveHNSWData as the marker.
|
||||
// shape), plus an empty `connections: {}` in saveVectorIndexData as the marker.
|
||||
// The read path tries to load the blob first; missing blob → legacy
|
||||
// connections field. Format convergence is lazy: pre-2.4.0 nodes still
|
||||
// load via the legacy path, then write the compressed form on next dirty
|
||||
|
|
@ -139,7 +139,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
/**
|
||||
* @description Inject (or detach) the connections codec. When set, node
|
||||
* persistence writes a delta-varint-compressed binary blob alongside an
|
||||
* empty `connections: {}` marker in saveHNSWData; the load path reads the
|
||||
* empty `connections: {}` marker in saveVectorIndexData; the load path reads the
|
||||
* blob and decodes. Null reverts to the legacy JSON-array path. Wiring is
|
||||
* done by brainy.ts when the `graph:compression` provider is registered
|
||||
* AND the storage adapter exposes the binary-blob primitive AND the
|
||||
|
|
@ -227,7 +227,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
* @description Persist one node's connections. When the connections codec is
|
||||
* wired AND the storage adapter exposes `saveBinaryBlob`, the per-level
|
||||
* connection sets are encoded into a single compact buffer and stored as a
|
||||
* binary blob; `saveHNSWData` then records the node's level and an empty
|
||||
* binary blob; `saveVectorIndexData` then records the node's level and an empty
|
||||
* `connections: {}` as the marker. Otherwise the legacy JSON-array path is
|
||||
* taken — the format every brainy release before 2.4.0 wrote. The two are
|
||||
* intentionally NOT dual-written: one or the other, never both, so format
|
||||
|
|
@ -250,7 +250,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
if (canCompress) {
|
||||
const encoded = this.connectionsCodec!.encode(noun.connections)
|
||||
await storageWithBlob.saveBinaryBlob!(compressedConnectionsKey(nodeId), encoded)
|
||||
await this.storage.saveHNSWData(nodeId, {
|
||||
await this.storage.saveVectorIndexData(nodeId, {
|
||||
level: noun.level,
|
||||
connections: {}
|
||||
})
|
||||
|
|
@ -261,7 +261,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
for (const [level, nounIds] of noun.connections.entries()) {
|
||||
connectionsObj[level.toString()] = Array.from(nounIds)
|
||||
}
|
||||
await this.storage.saveHNSWData(nodeId, {
|
||||
await this.storage.saveVectorIndexData(nodeId, {
|
||||
level: noun.level,
|
||||
connections: connectionsObj
|
||||
})
|
||||
|
|
@ -1197,7 +1197,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
this.unifiedCache.set(
|
||||
cacheKey,
|
||||
fromMmap,
|
||||
'hnsw',
|
||||
'vectors',
|
||||
fromMmap.length * 4,
|
||||
50
|
||||
)
|
||||
|
|
@ -1232,7 +1232,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
this.unifiedCache.set(
|
||||
cacheKey,
|
||||
loaded,
|
||||
'hnsw', // Type for fairness monitoring
|
||||
'vectors', // Type for fairness monitoring
|
||||
loaded.length * 4, // Size in bytes (float32)
|
||||
50 // Rebuild cost in ms (moderate priority)
|
||||
)
|
||||
|
|
@ -1303,7 +1303,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
const id = uncachedIds[i]
|
||||
const v = fromMmap[i]
|
||||
if (v) {
|
||||
this.unifiedCache.set(`hnsw:vector:${id}`, v, 'hnsw', v.length * 4, 50)
|
||||
this.unifiedCache.set(`vector:${id}`, v, 'vectors', v.length * 4, 50)
|
||||
} else {
|
||||
misses.push(id)
|
||||
}
|
||||
|
|
@ -1322,7 +1322,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
prodLog.debug(`MmapVectorBackend write-back failed for ${id}`, error)
|
||||
}
|
||||
}
|
||||
this.unifiedCache.set(`hnsw:vector:${id}`, vector, 'hnsw', vector.length * 4, 50)
|
||||
this.unifiedCache.set(`vector:${id}`, vector, 'vectors', vector.length * 4, 50)
|
||||
}))
|
||||
return
|
||||
}
|
||||
|
|
@ -1335,7 +1335,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
|
||||
const vector = await this.storage.getNounVector(id)
|
||||
if (vector) {
|
||||
this.unifiedCache.set(cacheKey, vector, 'hnsw', vector.length * 4, 50)
|
||||
this.unifiedCache.set(cacheKey, vector, 'vectors', vector.length * 4, 50)
|
||||
}
|
||||
return vector
|
||||
})
|
||||
|
|
@ -1500,7 +1500,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
for (const nounData of result.items) {
|
||||
try {
|
||||
// Load HNSW graph data for this entity
|
||||
const hnswData = await (this.storage as any).getHNSWData(nounData.id)
|
||||
const hnswData = await (this.storage as any).getVectorIndexData(nounData.id)
|
||||
|
||||
if (!hnswData) {
|
||||
// No HNSW data - skip (might be entity added before persistence)
|
||||
|
|
@ -1584,7 +1584,7 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
for (const nounData of result.items) {
|
||||
try {
|
||||
// Load HNSW graph data for this entity
|
||||
const hnswData = await (this.storage as any).getHNSWData(nounData.id)
|
||||
const hnswData = await (this.storage as any).getVectorIndexData(nounData.id)
|
||||
|
||||
if (!hnswData) {
|
||||
// No HNSW data - skip (might be entity added before persistence)
|
||||
|
|
@ -1816,12 +1816,12 @@ export class JsHnswVectorIndex implements VectorIndexProvider {
|
|||
const estimatedVectorMemoryMB = (entityCount * bytesPerVector) / (1024 * 1024)
|
||||
const availableCacheMB = (cacheStats.maxSize * 0.8) / (1024 * 1024) // 80% threshold
|
||||
|
||||
// Calculate HNSW-specific cache stats
|
||||
const vectorsInCache = cacheStats.typeCounts.hnsw || 0
|
||||
const hnswMemoryBytes = cacheStats.typeSizes.hnsw || 0
|
||||
// Calculate vector-index-specific cache stats
|
||||
const vectorsInCache = cacheStats.typeCounts.vectors || 0
|
||||
const hnswMemoryBytes = cacheStats.typeSizes.vectors || 0
|
||||
|
||||
// Calculate fairness metrics
|
||||
const hnswAccessCount = cacheStats.typeAccessCounts.hnsw || 0
|
||||
const hnswAccessCount = cacheStats.typeAccessCounts.vectors || 0
|
||||
const totalAccessCount = cacheStats.totalAccessCount
|
||||
const hnswAccessPercent = totalAccessCount > 0 ? (hnswAccessCount / totalAccessCount) * 100 : 0
|
||||
|
||||
|
|
|
|||
|
|
@ -551,7 +551,7 @@ export class TypeAwareHNSWIndex {
|
|||
const index = this.getIndexForType(nounType as NounType)
|
||||
|
||||
// Load HNSW graph data
|
||||
const hnswData = await (this.storage as any).getHNSWData(nounData.id)
|
||||
const hnswData = await (this.storage as any).getVectorIndexData(nounData.id)
|
||||
if (!hnswData) {
|
||||
continue // No HNSW data
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue