feat: SQ4 (4-bit) scalar quantization + native distance hook (2.5.0 #30)

Brainy now has a complete reference SQ4 (4-bit per dimension, 8× compression
vs float32) quantization layer in src/utils/vectorQuantization.ts: quantizeSQ4,
dequantizeSQ4, distanceSQ4Js, serializeSQ4, deserializeSQ4 — all byte-for-byte
identical to cortex's Rust quantize_sq4 / dequantize_sq4 / cosine_distance_sq4.

The pack format mirrors cortex exactly:
- Two nibbles per byte. High nibble = vector[2i], low nibble = vector[2i+1].
- Odd-dim vectors place the final value in the high nibble and pad the low
  nibble with 0. Packed length = ceil(dim / 2).
- Zero-range vectors (all values identical) map every nibble to 8 (midpoint
  of [0, 15]) — both reference impls produce 0x88 bytes.

The active-fn dispatch pattern matches SQ8: distanceSQ4Js is the pure-JS
reference; distanceSQ4 routes through a swappable activeSQ4Distance slot;
setSQ4DistanceImplementation(fn) swaps in cortex's SIMD Rust distance when the
distance:sq4 provider is registered. brainy.ts wires the provider in init,
same shape as the existing SQ8 wiring (~14 LOC). The dispatch path is the
single point of integration so HNSW search code never needs to know whether
it's running on JS or native.

Serialization adds a uint32 dim field after min/max (12-byte header total) —
SQ8 doesn't need it because the packed length equals dim, but SQ4's packed
length rounds up so dim must be recorded explicitly to disambiguate the
trailing pad nibble.

Tests (1462 total, +15):
- Pack format: high-nibble-first, odd-dim trailing pad = 0, ceil(dim/2) length
- Round-trip error envelope: every reconstructed value within half a quantum
  step of the original (verified across dims 1, 2, 3, 4, 16, 17, 384, 385, 1000)
- Edge cases: empty input throws, zero-range maps to 0x88, out-of-range
  clamping
- distanceSQ4Js agreement with dequantize-then-cosine baseline within 1e-9
- dispatch swap (setSQ4DistanceImplementation): swap in a sentinel fn, observe
  the active fn changes, restore the JS default, observe the revert
- serialize/deserialize round-trip preserves all four fields byte-for-byte for
  both even and odd dim

The HNSW SQ4 reranking path (bits === 4 routing in HNSWIndex's quantization
config) is wired to use distanceSQ4 — when cortex's distance:sq4 provider
is registered, that hot path immediately becomes native SIMD with zero brainy
change required. Cross-language byte-format parity tests run in the cortex
test suite (paired release brainy 7.28.0 + cortex 2.5.0).
This commit is contained in:
David Snelling 2026-05-28 12:27:10 -07:00
parent 338946cec2
commit 73e7e39b52
3 changed files with 614 additions and 0 deletions

View file

@ -518,6 +518,21 @@ export class Brainy<T = any> implements BrainyInterface<T> {
setSQ8DistanceImplementation(sq8DistanceProvider)
}
// Provider: native SQ4 approximate distance (cortex's Rust). Same swap
// pattern as SQ8; signature is byte-compatible with the JS distanceSQ4
// (4-bit quantization range, packed nibbles). Used in HNSW SQ4 reranking
// when config.hnsw.quantization.bits === 4. Falls back to JS when absent.
const sq4DistanceProvider = this.pluginRegistry.getProvider<
(
a: Uint8Array, aMin: number, aMax: number, aDim: number,
b: Uint8Array, bMin: number, bMax: number, bDim: number
) => number
>('distance:sq4')
if (sq4DistanceProvider) {
const { setSQ4DistanceImplementation } = await import('./utils/vectorQuantization.js')
setSQ4DistanceImplementation(sq4DistanceProvider)
}
// Provider: sort:topK (e.g. cortex's native partial-sort / heap-select) — swaps the
// JS result-ranking used by find() to pick the top `offset + limit` rows. The provider
// returns indices into a scores array, ordered descending with stable ties, identical