feat: v0.49 - Filter discovery API, remove deprecated methods, improve performance

BREAKING CHANGES:
- Removed deprecated getAllNouns() and getAllVerbs() methods
- All internal usage migrated to pagination-based methods

New Features:
- Filter Discovery API:
  - getFilterValues(field): Get all available values for a field
  - getFilterFields(): Get all filterable fields
  - Enables dynamic filter UI generation with O(1) field discovery
- Hybrid metadata indexing with field-level indexes
- Adaptive auto-flush for optimal performance
- LRU caching for metadata indexes

Improvements:
- Fixed ENAMETOOLONG errors from vector-based filenames
- Safe filename generation using hash-based approach
- Scalable chunked value storage for millions of entries
- Performance optimization with adaptive flush thresholds
- Added support for $includes operator in metadata filters

Technical:
- Replaced vector-based filenames with safe hash approach
- Implemented MetadataIndexCache with existing SearchCache pattern
- Field indexes enable O(1) filter discovery
- Adaptive flush based on performance metrics (20-200 entries)
- All tests passing with improved metadata filtering
This commit is contained in:
David Snelling 2025-08-06 14:39:33 -07:00
parent cc1c6fb718
commit 2231f90913
18 changed files with 958 additions and 486 deletions

16
simple-init-test.js Normal file
View file

@ -0,0 +1,16 @@
import { BrainyData } from './dist/index.js'
console.log('Starting simple init test...')
const brainy = new BrainyData({
storage: { type: 'memory' }
})
console.log('Calling init...')
await brainy.init()
console.log('Init complete!')
await brainy.add('test', { name: 'test' })
console.log('Add complete!')
const results = await brainy.searchText('test', 1)
console.log('Search complete! Found:', results.length)