soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

fix: exclude __words__ keyword index from corruption detection and getStats()

Browse source 
The __words__ keyword index stores 50-5000 entries per entity (one per
word), which inflated avg entries/entity well above the corruption
threshold of 100. This caused:

1. validateConsistency() to falsely detect corruption on every startup,
   triggering unnecessary clearAllIndexData() + rebuild() cycles
2. getStats() to log false "Metadata index may be corrupted" warnings
   and report inflated totalEntries/totalIds stats

Both methods now skip __words__ when counting, so stats and health
checks reflect metadata fields only (noun, type, createdAt, etc.).
Keyword search is unaffected since the __words__ field index itself
is not modified.
This commit is contained in:
David Snelling 2026-01-27 15:38:21 -08:00
parent 32dbdcec61
commit 364360d447
128 changed files with 5637 additions and 5682 deletions
Download patch file Download diff file Expand all files Collapse all files

4
docs/guides/streaming-imports.md
View file

@ -300,7 +300,7 @@ interface ImportProgress {
relationships?: number // Relationships inferred so far
/**
* Whether data is queryable (v4.2.0+)
* Whether data is queryable
*
* true = Indexes flushed, queries will be fast and complete
* false/undefined = Data in storage but indexes not flushed yet
@ -376,7 +376,7 @@ No changes required! Streaming is now always enabled with optimal defaults:
// Before (v3.x, v4.0, v4.1): Works the same
await brain.import(file)
// After (v4.2.0+): Streaming always on, zero config
// After: Streaming always on, zero config
await brain.import(file)
```