perf: defer HNSW persistence during addMany() batch operations
addMany() now temporarily switches the HNSW index to deferred persist mode during the batch loop. Previously, each add() triggered ~16-20 saveHNSWData calls for modified neighbors (each a read→gzip→atomic-write cycle). For 450 items this was ~8,100 individual storage writes. With deferred mode, dirty node IDs are collected in a Set during the batch and flushed once at the end — deduplicating repeated neighbor updates. A node modified by insert #3 and again by insert #47 is written only once. Also adds setPersistMode() to TypeAwareHNSWIndex, propagating mode changes to all existing type-specific sub-indexes.
This commit is contained in:
parent
c054c41943
commit
973b6aaa39
2 changed files with 80 additions and 42 deletions
|
|
@ -163,6 +163,19 @@ export class TypeAwareHNSWIndex {
|
|||
return this.persistMode
|
||||
}
|
||||
|
||||
/**
|
||||
* Switch persist mode at runtime. Propagates to all existing type-specific indexes.
|
||||
* Used by addMany() to defer persistence during batch operations.
|
||||
*/
|
||||
public setPersistMode(mode: 'immediate' | 'deferred'): void {
|
||||
this.persistMode = mode
|
||||
for (const index of this.indexes.values()) {
|
||||
if (typeof (index as any).setPersistMode === 'function') {
|
||||
(index as any).setPersistMode(mode)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get or create HNSW index for a specific type (lazy initialization)
|
||||
*
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue