soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

fix: correct cache eviction formula to prioritize high-value items

Browse source 
Fixed inverted eviction scoring formula in UnifiedCache that was causing
metadata (cheap to rebuild) to be retained while HNSW vectors (expensive,
frequently accessed) were evicted. This was causing OOM crashes during
large Excel imports with relationship extraction.

Changes:
- evictLowestValue(): Changed accessScore / rebuildCost to accessScore * rebuildCost
- evictForSize(): Changed accessScore / rebuildCost to accessScore * rebuildCost
- evictType(): Changed accessScore / rebuildCost to accessScore * rebuildCost

With the corrected formula, items with higher access counts AND higher
rebuild costs get higher scores and are protected from eviction.

Test coverage: Added comprehensive eviction scoring tests

Fixes: Type metadata hogging 99.7% of cache with only 3.7% access rate
This commit is contained in:
David Snelling 2025-10-13 11:21:19 -07:00
parent d62875dc6c
commit 8e7b52bda9
2 changed files with 238 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

12
src/utils/unifiedCache.ts
View file

@ -241,9 +241,9 @@ export class UnifiedCache {
let lowestScore = Infinity
for (const [key, item] of this.cache) {
// Calculate value score: access frequency / rebuild cost
// Calculate value score: access frequency * rebuild cost (higher is better)
const accessScore = (this.access.get(key) || 1)
const score = accessScore / Math.max(item.rebuildCost, 1)
const score = accessScore * item.rebuildCost
if (score < lowestScore) {
lowestScore = score
@ -267,9 +267,9 @@ export class UnifiedCache {
*/
evictForSize(bytesNeeded: number): boolean {
const candidates: Array<[string, number, CacheItem]> = []
for (const [key, item] of this.cache) {
const score = (this.access.get(key) || 1) / item.rebuildCost
const score = (this.access.get(key) || 1) * item.rebuildCost
candidates.push([key, score, item])
}
@ -349,10 +349,10 @@ export class UnifiedCache {
*/
private evictType(type: 'hnsw' | 'metadata' | 'embedding' | 'other'): void {
const candidates: Array<[string, number, CacheItem]> = []
for (const [key, item] of this.cache) {
if (item.type === type) {
const score = (this.access.get(key) || 1) / item.rebuildCost
const score = (this.access.get(key) || 1) * item.rebuildCost
candidates.push([key, score, item])
}
}