soulcraftlabs/open-brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 341 Packages Wiki Activity Actions 5

fix: exclude __words__ keyword index from corruption detection and getStats()

Browse source 
The __words__ keyword index stores 50-5000 entries per entity (one per
word), which inflated avg entries/entity well above the corruption
threshold of 100. This caused:

1. validateConsistency() to falsely detect corruption on every startup,
   triggering unnecessary clearAllIndexData() + rebuild() cycles
2. getStats() to log false "Metadata index may be corrupted" warnings
   and report inflated totalEntries/totalIds stats

Both methods now skip __words__ when counting, so stats and health
checks reflect metadata fields only (noun, type, createdAt, etc.).
Keyword search is unaffected since the __words__ field index itself
is not modified.
This commit is contained in:
David Snelling 2026-01-27 15:38:21 -08:00
parent 32dbdcec61
commit 364360d447
128 changed files with 5637 additions and 5682 deletions
Download patch file Download diff file Expand all files Collapse all files

4
src/storage/cow/binaryDataCodec.ts
View file

@ -86,7 +86,7 @@ export function unwrapBinaryData(data: any): Buffer {
/**
* Wrap binary data for JSON storage
*
* ⚠️ WARNING: DO NOT USE THIS ON WRITE PATH! (v6.2.0)
* ⚠️ WARNING: DO NOT USE THIS ON WRITE PATH!
* ⚠️ Use key-based dispatch in baseStorage.ts COW adapter instead.
* ⚠️ This function exists for legacy/compatibility only.
*
@ -94,7 +94,7 @@ export function unwrapBinaryData(data: any): Buffer {
* This is FRAGILE because compressed binary can accidentally parse as valid JSON,
* causing blob integrity failures.
*
* v6.2.0 SOLUTION: baseStorage.ts COW adapter now uses key naming convention:
* SOLUTION: baseStorage.ts COW adapter now uses key naming convention:
* - Keys with '-meta:' or 'ref:' prefix → Always JSON
* - Keys with 'blob:', 'commit:', 'tree:' prefix → Always binary
* No guessing needed!