chore(8.0): delete vectorStore:mmap wiring — dead in the 8.0 provider world
The mmap-vector backend was a 2.x-era accelerator for the JS HNSW read path: a native provider registered under 'vectorStore:mmap' supplied a single-file mmap'd vector store, and JsHnswVectorIndex tried it before per-entity storage reads with lazy write-back migration. In the 8.0 provider world nothing registers that key — the native plugin's 3.0 line replaces the entire vector index under the 'vector' provider key (vectors live inside its own index file), and the open-core path never had an mmap provider. The wiring is unreachable; per the no-unwired-code rule it goes away: - src/hnsw/mmapVectorBackend.ts deleted (175 LOC) - wireMmapVectorBackend() + its init call site removed from brainy.ts - VectorStoreMmapProvider + VectorStoreMmapInstance contracts removed from plugin.ts - JsHnswVectorIndex loses the vectorBackend field, setVectorBackend(), and the mmap-first branches in getVectorSafe/preloadVectors — the storage + UnifiedCache path is now the single read path - tests/unit/hnsw/mmap-vector-backend.test.ts deleted The 7.x line keeps the wiring and got the capacity-NaN bugfix as 7.31.3 (see the platform handoff mmap thread). EntityIdMapperProvider stays — the connections codec and the id mapper still implement it. 1403/1403 tests, build clean.
This commit is contained in:
parent
42159f2bd7
commit
62f6472fa0
5 changed files with 2 additions and 640 deletions
|
|
@ -40,10 +40,8 @@ import { PluginRegistry } from './plugin.js'
|
|||
import type {
|
||||
BrainyPlugin,
|
||||
BrainyPluginContext,
|
||||
GraphCompressionProvider,
|
||||
VectorStoreMmapProvider
|
||||
GraphCompressionProvider
|
||||
} from './plugin.js'
|
||||
import { MmapVectorBackend } from './hnsw/mmapVectorBackend.js'
|
||||
import { ConnectionsCodec } from './hnsw/connectionsCodec.js'
|
||||
import { TransactionManager } from './transaction/TransactionManager.js'
|
||||
import { RevisionConflictError } from './transaction/RevisionConflictError.js'
|
||||
|
|
@ -604,15 +602,6 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
this.graphIndex = graphIndex
|
||||
}
|
||||
|
||||
// Wire the mmap-vector backend (2.4.0 #2). When the vectorStore:mmap
|
||||
// provider is registered AND the storage adapter resolves a real local
|
||||
// path via getBinaryBlobPath(), open the mmap file there and inject the
|
||||
// backend into JsHnswVectorIndex. Cloud adapters return null and skip silently;
|
||||
// JsHnswVectorIndex's behaviour is then identical to pre-2.4.0 (per-entity reads
|
||||
// from canonical storage). Failures are non-fatal — the index still
|
||||
// works, just without the zero-copy fast path.
|
||||
await this.wireMmapVectorBackend()
|
||||
|
||||
// Wire the connections codec (2.4.0 #3). When the graph:compression
|
||||
// provider is registered AND the metadata index exposes a stable
|
||||
// idMapper, inject a codec that encodes HNSW connections as
|
||||
|
|
@ -9379,72 +9368,6 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
await this.lazyRebuildPromise
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Open the mmap-vector backend and inject it into JsHnswVectorIndex.
|
||||
* Called once during init after plugin activation, metadataIndex init, and
|
||||
* graphIndex setup — so the idMapper is available and providers are wired.
|
||||
*
|
||||
* Activation conditions (ALL must hold; otherwise this is a no-op and the
|
||||
* index falls back to per-entity storage reads, the pre-2.4.0 behaviour):
|
||||
* 1. The `vectorStore:mmap` provider is registered (cortex registers this).
|
||||
* 2. The storage adapter resolves a real local path via `getBinaryBlobPath()`
|
||||
* — cloud adapters return null and the mmap layer is skipped silently.
|
||||
* 3. The metadata index exposes its idMapper (a stable UUID↔int map). The
|
||||
* mmap layout is keyed by these ints, so 2.4.0 #1 is the prerequisite.
|
||||
*
|
||||
* Vector dimensionality: read from `this.dimensions` if set; otherwise the
|
||||
* default for the standard embedding model (384). The dim is fixed at
|
||||
* file-creation time and cannot change later, so a wrong default would only
|
||||
* matter on the very first init of a brand-new instance. After the first
|
||||
* `add()` the in-memory `this.dimensions` is set, and any subsequent init
|
||||
* (or a re-open) sees the correct value.
|
||||
*
|
||||
* Failures here are non-fatal: a debug-level log records the reason, and
|
||||
* JsHnswVectorIndex continues without the mmap fast path.
|
||||
*/
|
||||
private async wireMmapVectorBackend(): Promise<void> {
|
||||
const provider = this.pluginRegistry.getProvider<VectorStoreMmapProvider>('vectorStore:mmap')
|
||||
if (!provider) return
|
||||
|
||||
const storageWithBlob = this.storage as unknown as {
|
||||
getBinaryBlobPath?: (key: string) => string | null
|
||||
}
|
||||
const vectorPath = storageWithBlob.getBinaryBlobPath?.('_vectors/main') ?? null
|
||||
if (!vectorPath) return
|
||||
|
||||
const idMapper = this.metadataIndex.getIdMapper?.()
|
||||
if (!idMapper) return
|
||||
|
||||
const dim = this.dimensions ?? 384
|
||||
const initialCapacity = Math.max(idMapper.size * 2, 1024)
|
||||
|
||||
try {
|
||||
const backend = await MmapVectorBackend.open(
|
||||
provider,
|
||||
vectorPath,
|
||||
dim,
|
||||
initialCapacity,
|
||||
idMapper
|
||||
)
|
||||
this.index.setVectorBackend(backend)
|
||||
if (!this.config.silent) {
|
||||
console.log(
|
||||
`[brainy] vector mmap backend wired (path=${vectorPath}, dim=${dim}, capacity=${initialCapacity})`
|
||||
)
|
||||
}
|
||||
} catch (error) {
|
||||
// Common cases: dim mismatch from a prior init at a different dimension,
|
||||
// permission errors, disk full, file corruption. Index keeps working via
|
||||
// the per-entity storage path; flag the cause for diagnosis.
|
||||
if (!this.config.silent) {
|
||||
console.warn(
|
||||
`[brainy] mmap-vector backend not wired (${(error as Error).message}); ` +
|
||||
`falling back to per-entity vector reads`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Wire the HNSW connections codec (2.4.0 #3). Activates when
|
||||
* BOTH (a) the `graph:compression` provider is registered (cortex registers
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue