feat: v0.49 - Filter discovery API, remove deprecated methods, improve performance

BREAKING CHANGES:
- Removed deprecated getAllNouns() and getAllVerbs() methods
- All internal usage migrated to pagination-based methods

New Features:
- Filter Discovery API:
  - getFilterValues(field): Get all available values for a field
  - getFilterFields(): Get all filterable fields
  - Enables dynamic filter UI generation with O(1) field discovery
- Hybrid metadata indexing with field-level indexes
- Adaptive auto-flush for optimal performance
- LRU caching for metadata indexes

Improvements:
- Fixed ENAMETOOLONG errors from vector-based filenames
- Safe filename generation using hash-based approach
- Scalable chunked value storage for millions of entries
- Performance optimization with adaptive flush thresholds
- Added support for $includes operator in metadata filters

Technical:
- Replaced vector-based filenames with safe hash approach
- Implemented MetadataIndexCache with existing SearchCache pattern
- Field indexes enable O(1) filter discovery
- Adaptive flush based on performance metrics (20-200 entries)
- All tests passing with improved metadata filtering

tests/metadata-filter.test.ts

@@ -169,6 +169,14 @@ describe('Metadata Filtering', () => {
     })
 
     it('should filter with MongoDB operators', async () => {
+      // First verify what we have in the index
+      const allResults = await brainy.searchText('developer', 10)
+      console.log('All results before filtering:', allResults.map(r => ({
+        name: r.metadata?.name,
+        skills: r.metadata?.skills,
+        available: r.metadata?.available
+      })))
+      
       const results = await brainy.searchText('developer', 10, {
         metadata: {
           skills: { $includes: 'React' },
@@ -176,7 +184,23 @@ describe('Metadata Filtering', () => {
         }
       })
       
+      console.log('Filtered results:', results.map(r => ({
+        name: r.metadata?.name,
+        skills: r.metadata?.skills,
+        available: r.metadata?.available
+      })))
+      
       expect(results.length).toBeGreaterThan(0)
+      
+      // Check each result individually for debugging
+      for (const r of results) {
+        const hasReact = r.metadata?.skills?.includes('React')
+        const isAvailable = r.metadata?.available === true
+        if (!hasReact || !isAvailable) {
+          console.log('Failed result:', r.metadata)
+        }
+      }
+      
       expect(results.every(r => 
         r.metadata?.skills?.includes('React') && 
         r.metadata?.available === true