fix: exclude __words__ keyword index from corruption detection and getStats()

The __words__ keyword index stores 50-5000 entries per entity (one per
word), which inflated avg entries/entity well above the corruption
threshold of 100. This caused:

1. validateConsistency() to falsely detect corruption on every startup,
   triggering unnecessary clearAllIndexData() + rebuild() cycles
2. getStats() to log false "Metadata index may be corrupted" warnings
   and report inflated totalEntries/totalIds stats

Both methods now skip __words__ when counting, so stats and health
checks reflect metadata fields only (noun, type, createdAt, etc.).
Keyword search is unaffected since the __words__ field index itself
is not modified.
This commit is contained in:
David Snelling 2026-01-27 15:38:21 -08:00
parent 32dbdcec61
commit 364360d447
128 changed files with 5637 additions and 5682 deletions

View file

@ -117,7 +117,7 @@ export const neuralCommand = {
await handleNeighborsCommand(neural, argv)
break
case 'path':
console.log(chalk.yellow('\n⚠ Semantic path finding coming in v3.21.0'))
console.log(chalk.yellow('\n⚠ Semantic path finding coming soon'))
console.log(chalk.dim('This feature requires implementing graph traversal algorithms'))
console.log(chalk.dim('Use "neighbors" and "hierarchy" commands to explore connections'))
break
@ -148,7 +148,7 @@ async function promptForAction(): Promise<string> {
{ name: '🎯 Find semantic clusters', value: 'clusters' },
{ name: '🌳 Show item hierarchy', value: 'hierarchy' },
{ name: '🕸️ Find semantic neighbors', value: 'neighbors' },
{ name: '🛣️ Find semantic path between items (v3.21.0)', value: 'path', disabled: true },
{ name: '🛣️ Find semantic path between items (coming soon)', value: 'path', disabled: true },
{ name: '🚨 Detect outliers', value: 'outliers' },
{ name: '📊 Generate visualization data', value: 'visualize' }
]