David Snelling
d02359ec60
Critical fix for incomplete v3.50.1 release.
Problem: v3.50.1 prevented vector fields by name ('vector', 'embedding')
but missed vectors stored as objects with numeric keys: {0: 0.1, 1: 0.2, ...}
Studio team diagnostics showed:
- 212,531 chunk files with NUMERIC field names
- Examples: "field": "54716", "field": "100000", "field": "100001"
- 424,837 total files (expected ~1,200)
Root Cause: Vectors converted to objects with numeric keys were still
being indexed because field name check only caught semantic names.
Fix Applied (src/utils/metadataIndex.ts:1106):
- Added regex check: if (/^\d+$/.test(key)) continue
- Skips ANY purely numeric field name (array indices as object keys)
- Catches: "0", "1", "2", "100", "54716", "100000", etc.
Test Coverage:
- Added new test: "should NOT index objects with numeric keys (v3.50.2 fix)"
- Verifies NO chunk files have numeric field names
- All 8 integration tests passing
Impact:
- Prevents 212K+ chunk files from being created
- Reduces file count from 424K to ~1,200 (354x reduction)
- Fixes server hangs during initialization
- Completes the metadata explosion fix started in v3.50.1
|
||
|---|---|---|
| .. | ||
| api-parameter-validation.test.ts | ||
| brainy-complete.integration.test.ts | ||
| brainy-core.integration.test.ts | ||
| brainy-phase1c-integration.test.ts | ||
| find-unified-integration.test.ts | ||
| gcs-native-storage.test.ts | ||
| gcs-persistence-fix.test.ts | ||
| hnsw-rebuild.test.ts | ||
| metadata-vector-exclusion.test.ts | ||
| phase3TypeFirstQuery.integration.test.ts | ||
| s3-storage.test.ts | ||
| smart-import.test.ts | ||
| typeAwareHNSW.integration.test.ts | ||
| typeInference.hybrid.integration.test.ts | ||
| typeInference.integration.test.ts | ||