soulcraftlabs/open-brainy
Archived
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 345 Packages Wiki Activity

fix: exclude __words__ keyword index from corruption detection and getStats()

Browse source 
The __words__ keyword index stores 50-5000 entries per entity (one per
word), which inflated avg entries/entity well above the corruption
threshold of 100. This caused:

1. validateConsistency() to falsely detect corruption on every startup,
   triggering unnecessary clearAllIndexData() + rebuild() cycles
2. getStats() to log false "Metadata index may be corrupted" warnings
   and report inflated totalEntries/totalIds stats

Both methods now skip __words__ when counting, so stats and health
checks reflect metadata fields only (noun, type, createdAt, etc.).
Keyword search is unaffected since the __words__ field index itself
is not modified.
This commit is contained in:
David Snelling 2026-01-27 15:38:21 -08:00
parent 32dbdcec61
commit 364360d447
128 changed files with 5637 additions and 5682 deletions
Download patch file Download diff file Expand all files Collapse all files

10
src/storage/adapters/historicalStorageAdapter.ts
View file

@ -25,7 +25,7 @@
* - Lazy loading: only loads entities when accessed
* - No eager-loading of entire commit state
*
* v5.4.0: Production-ready, billion-scale historical queries
* Production-ready, billion-scale historical queries
*/
import { BaseStorage } from '../baseStorage.js'
@ -144,7 +144,7 @@ export class HistoricalStorageAdapter extends BaseStorage {
*/
public async init(): Promise<void> {
// Get COW components from underlying storage
// v6.2.4: Fixed property names - use public properties without underscore prefix
// Fixed property names - use public properties without underscore prefix
this.commitLog = this.underlyingStorage.commitLog
this.blobStorage = this.underlyingStorage.blobStorage
@ -161,7 +161,7 @@ export class HistoricalStorageAdapter extends BaseStorage {
throw new Error(`Commit not found: ${this.commitId}`)
}
// v6.0.0: Initialize GraphAdjacencyIndex and type statistics
// Initialize GraphAdjacencyIndex and type statistics
await super.init()
}
@ -310,12 +310,12 @@ export class HistoricalStorageAdapter extends BaseStorage {
/**
* Check if COW has been explicitly disabled via clear()
* v5.10.4: No-op for HistoricalStorageAdapter (read-only, doesn't manage COW)
* No-op for HistoricalStorageAdapter (read-only, doesn't manage COW)
* @returns Always false (read-only adapter doesn't manage COW state)
* @protected
*/
/**
* v5.11.0: Removed checkClearMarker() and createClearMarker() methods
* Removed checkClearMarker() and createClearMarker() methods
* COW is now always enabled - marker files are no longer used
*/