feat: implement unified metadata intelligence system

- Add cardinality tracking for all metadata fields with distribution analysis
- Implement smart normalization for high-cardinality fields (timestamps, floats)
- Add field statistics tracking (query counts, patterns, performance)
- Integrate field discovery methods for query optimization
- Fix UPDATE bug by passing old metadata to removeFromIndex
- Track query patterns to optimize index strategies dynamically
- Add getFieldStatistics, getFieldCardinality, getOptimalQueryPlan methods
- Implement time bucketing for timestamp fields (1-minute precision)
- Add float precision reduction for numeric fields (2 decimal places)

This unifies metadata performance optimization with field discovery,
providing a complete metadata intelligence system that self-optimizes
based on usage patterns and data characteristics.
This commit is contained in:
David Snelling 2025-09-12 12:45:32 -07:00
parent 33c6b06649
commit d1be41fa91
2 changed files with 334 additions and 4 deletions

View file

@ -316,7 +316,7 @@ export class Brainy<T = any> {
})
// Update metadata index - remove old entry and add new one
await this.metadataIndex.removeFromIndex(params.id)
await this.metadataIndex.removeFromIndex(params.id, existing.metadata)
await this.metadataIndex.addToIndex(params.id, updatedMetadata)
})
}