This repository has been archived on 2026-09-03. You can view files and clone it, but you cannot make any changes to it's state, such as pushing and creating new issues, pull requests or comments.
open-brainy/src/hnsw
David Snelling 4d1d567236 perf(hnsw): deferred persistence mode for 30-50× faster cloud storage adds
Problem:
Each add() triggered 70 GCS operations (34 reads + 36 writes) because HNSW
updates 16+ neighbors per add, and each neighbor did a read-modify-write cycle.
Result: 7-11 seconds per add() on GCS.

Solution:
- Add `hnswPersistMode: 'immediate' | 'deferred'` config option
- In deferred mode, track dirty nodes instead of persisting immediately
- Flush dirty nodes on close() or explicit flush()
- Smart defaults: cloud storage (GCS/S3/R2/Azure) = deferred, local = immediate

Performance impact:
- Single add(): 7-11 seconds → 200-400ms (30-50× faster)
- GCS operations per add: 70 → 2-3

Zero configuration - cloud storage automatically uses deferred mode.

Files changed:
- src/types/brainy.types.ts: Add hnswPersistMode config option
- src/hnsw/hnswIndex.ts: Deferred mode, dirty tracking, flush()
- src/hnsw/typeAwareHNSWIndex.ts: Propagate to child indexes
- src/brainy.ts: Smart defaults, flush on close()

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 14:20:04 -08:00
..
hnswIndex.ts perf(hnsw): deferred persistence mode for 30-50× faster cloud storage adds 2025-12-02 14:20:04 -08:00
typeAwareHNSWIndex.ts perf(hnsw): deferred persistence mode for 30-50× faster cloud storage adds 2025-12-02 14:20:04 -08:00