fix: populate totalNodes/totalEdges in ALL storage adapters for HNSW rebuild
Critical fix for v3.37.1 GCS persistence bug where HNSW rebuild reported "Preloading 0 vectors" and hung indefinitely during container restarts. Root Cause: HNSW rebuild (hnswIndex.ts:835) calls storage.getStatistics() and expects stats.totalNodes to determine entity count for adaptive caching. When totalNodes was undefined, entityCount evaluated to 0, causing HNSW to report "0 vectors" and hang waiting for entities that it couldn't find. Fixes Applied: - GcsStorage: Populate totalNodes/totalEdges from this.totalNounCount/totalVerbCount - MemoryStorage: Populate totalNodes/totalEdges from in-memory counts - OPFSStorage: Populate totalNodes/totalEdges in ALL return paths (cache, current day, yesterday, legacy) - S3CompatibleStorage: Populate totalNodes/totalEdges in ALL return paths (cache, fresh stats, error fallback) Impact: - ✅ GCS container restarts now work correctly - ✅ HNSW rebuild correctly detects entity count - ✅ Adaptive caching strategy works as designed - ✅ All storage adapters now consistent Files exist in GCS, counts load correctly, but HNSW couldn't see them because getStatisticsData() didn't populate the totalNodes field that HNSW depends on. Closes: BRAINY_V3_37_1_INDEX_REBUILD_BUG.md Related: v3.37.0 (metadata reconstruction), v3.37.1 (getNoun_internal fix)
This commit is contained in:
parent
9fe790e0a7
commit
a21a84545a
4 changed files with 58 additions and 10 deletions
|
|
@ -691,6 +691,10 @@ export class MemoryStorage extends BaseStorage {
|
|||
verbCount: {...this.statistics.verbCount},
|
||||
metadataCount: {...this.statistics.metadataCount},
|
||||
hnswIndexSize: this.statistics.hnswIndexSize,
|
||||
// CRITICAL FIX: Populate totalNodes and totalEdges from in-memory counts
|
||||
// HNSW rebuild depends on these fields to determine entity count
|
||||
totalNodes: this.totalNounCount,
|
||||
totalEdges: this.totalVerbCount,
|
||||
lastUpdated: this.statistics.lastUpdated,
|
||||
// Include serviceActivity if present
|
||||
...(this.statistics.serviceActivity && {
|
||||
|
|
@ -703,11 +707,11 @@ export class MemoryStorage extends BaseStorage {
|
|||
services: this.statistics.services.map(s => ({...s}))
|
||||
}),
|
||||
// Include distributedConfig if present
|
||||
...(this.statistics.distributedConfig && {
|
||||
distributedConfig: JSON.parse(JSON.stringify(this.statistics.distributedConfig))
|
||||
...(this.statistics.distributedConfig && {
|
||||
distributedConfig: JSON.parse(JSON.stringify(this.statistics.distributedConfig))
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
// Since this is in-memory, there's no need for fallback mechanisms
|
||||
// to check multiple storage locations
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue