soulcraft/brainy
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

feat: v0.49 - Filter discovery API, remove deprecated methods, improve performance

Browse source 
BREAKING CHANGES:
- Removed deprecated getAllNouns() and getAllVerbs() methods
- All internal usage migrated to pagination-based methods

New Features:
- Filter Discovery API:
  - getFilterValues(field): Get all available values for a field
  - getFilterFields(): Get all filterable fields
  - Enables dynamic filter UI generation with O(1) field discovery
- Hybrid metadata indexing with field-level indexes
- Adaptive auto-flush for optimal performance
- LRU caching for metadata indexes

Improvements:
- Fixed ENAMETOOLONG errors from vector-based filenames
- Safe filename generation using hash-based approach
- Scalable chunked value storage for millions of entries
- Performance optimization with adaptive flush thresholds
- Added support for $includes operator in metadata filters

Technical:
- Replaced vector-based filenames with safe hash approach
- Implemented MetadataIndexCache with existing SearchCache pattern
- Field indexes enable O(1) filter discovery
- Adaptive flush based on performance metrics (20-200 entries)
- All tests passing with improved metadata filtering
This commit is contained in:
David Snelling 2025-08-06 14:43:46 -07:00
parent 2dc909909a
commit 2139f69754
2 changed files with 9 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

8
README.md
View file

@ -18,17 +18,17 @@
**Discover available filters and scale to millions of items!**
```javascript
// Discover what filters are available
// Discover what filters are available - O(1) field lookup
const categories = await brainy.getFilterValues('category')
// Returns: ['electronics', 'books', 'clothing', ...]
const fields = await brainy.getFilterFields()
const fields = await brainy.getFilterFields() // O(1) operation
// Returns: ['category', 'price', 'brand', 'rating', ...]
```
- ✅ **Filter Discovery API**: See what values are available for filtering
- ✅ **Filter Discovery API**: O(1) field discovery for instant filter UI generation
- ✅ **Improved Performance**: Removed deprecated methods, now uses pagination everywhere
- ✅ **Better Scalability**: Hybrid indexing approach handles millions of items
- ✅ **Better Scalability**: Hybrid indexing with O(1) field access scales to millions
- ✅ **Smart Caching**: LRU cache for frequently accessed filters
- ✅ **Zero Configuration**: Everything auto-optimizes based on usage patterns

10
docs/api-reference/search-methods.md
View file

@ -431,14 +431,14 @@ Brainy validates metadata queries and provides helpful error messages:
### getFilterValues(field)
Get all available values for a specific field that can be used in filters:
Get all available values for a specific field with O(1) field lookup:
```javascript
// Get all categories in the database
// Get all categories in the database - O(1) field access
const categories = await brainy.getFilterValues('category')
// Returns: ['electronics', 'books', 'clothing', 'home', ...]
// Get all brands
// Get all brands - O(1) field lookup + O(n) value retrieval
const brands = await brainy.getFilterValues('brand')
// Returns: ['apple', 'samsung', 'sony', ...]
@ -453,10 +453,10 @@ const results = await brainy.search("products", 10, {
### getFilterFields()
Get all fields that have been indexed and can be used for filtering:
Get all fields that have been indexed - O(1) operation:
```javascript
// Discover what fields are available
// Discover what fields are available - O(1) direct access
const fields = await brainy.getFilterFields()
// Returns: ['category', 'price', 'brand', 'rating', 'tags', ...]