fix: mmap-vector backend capacity NaN at the provider FFI boundary
A production deployment reported "[brainy] mmap-vector backend not wired (Capacity must be > 0); falling back to per-entity vector reads" on every cold start of populated brains under the native plugin — degrading vector recall to per-entity disk reads (8.3s cold starts at 685 entities, scaling linearly). Root cause: wireMmapVectorBackend computes its initial slot capacity as idMapper.size * 2. When the metadata index is provider-backed, getIdMapper() returns a façade exposing getInt/getOrAssign/getUuid but NO `size` property. `undefined * 2` is NaN, Math.max(NaN, 1024) stays NaN, and NaN coerces to 0 via ToUint32 at the provider's u32 FFI boundary — tripping the provider's "Capacity must be > 0" guard. The JS-only path never hit this because brainy's own EntityIdMapper has a real `size` getter. Fix, two independent layers: - wireMmapVectorBackend treats a missing/non-finite mapper size as 0, so the 1024-slot floor always holds (the file grows on demand past it). - MmapVectorBackend.open sanitizes the capacity (NaN/Infinity/non-positive → 16-slot floor) so no caller can ever hand the provider an invalid allocation size. Regression tests mimic the exact failure: a U32-coercing provider that rejects capacity 0 plus an idMapper façade without `size`. Pre-fix the NaN reached create() and threw; post-fix the backend wires with the floor capacity and round-trips vectors. 1464/1464 unit suite passing.
This commit is contained in:
parent
d6daafb426
commit
eade6ff1be
3 changed files with 68 additions and 2 deletions
|
|
@ -9410,7 +9410,15 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
if (!idMapper) return
|
||||
|
||||
const dim = this.dimensions ?? 384
|
||||
const initialCapacity = Math.max(idMapper.size * 2, 1024)
|
||||
// Provider-supplied id mappers (e.g. a native metadata index's façade) may
|
||||
// not expose a `size` property. `undefined * 2` is NaN, and NaN coerces to
|
||||
// 0 at the napi u32 boundary — the provider then rejects the create() with
|
||||
// "Capacity must be > 0". Treat a missing/non-finite size as 0 so the
|
||||
// 1024 floor always holds; the file grows on demand past it.
|
||||
const mapperSize = Number.isFinite((idMapper as { size?: number }).size)
|
||||
? (idMapper as { size: number }).size
|
||||
: 0
|
||||
const initialCapacity = Math.max(mapperSize * 2, 1024)
|
||||
|
||||
try {
|
||||
const backend = await MmapVectorBackend.open(
|
||||
|
|
|
|||
|
|
@ -77,11 +77,17 @@ export class MmapVectorBackend {
|
|||
idMapper: EntityIdMapperProvider
|
||||
): Promise<MmapVectorBackend> {
|
||||
await mkdir(dirname(path), { recursive: true })
|
||||
// NaN/Infinity/non-positive capacities coerce to 0 at the provider's u32
|
||||
// FFI boundary ("Capacity must be > 0"). Sanitize here so no caller can
|
||||
// ever hand the provider an invalid allocation size.
|
||||
const capacity = Number.isFinite(initialCapacity) && initialCapacity >= 16
|
||||
? Math.ceil(initialCapacity)
|
||||
: 16
|
||||
let store: VectorStoreMmapInstance
|
||||
try {
|
||||
store = provider.open(path)
|
||||
} catch {
|
||||
store = provider.create(path, dim, Math.max(initialCapacity, 16))
|
||||
store = provider.create(path, dim, capacity)
|
||||
}
|
||||
return new MmapVectorBackend(store, idMapper)
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue