fix: v4.8.1 - Fix 11-version VFS bug (metadata skip + TypeAware performance)

CRITICAL BUG FIX: FileSystemStorage was skipping verbs without metadata files,
causing getVerbsBySource() to return 0 results despite 601 relationships existing.

Root Cause:
- Workshop data had 601 verb vector files with 0 metadata files
- FileSystemStorage.getVerbsWithPagination() skipped verbs if metadata === null
- This bug survived 11 versions (v4.5.1 through v4.8.0)

Fixes Applied:

1. FileSystemStorage (2 locations):
   - Line 1391-1406: Remove metadata skip in getVerbsWithPagination()
   - Line 2427-2442: Remove metadata skip in streaming method
   - Use (metadata || {}) pattern like other adapters

2. TypeAwareStorage (2 methods):
   - getVerbsBySource_internal: Delegate to underlying storage (was O(total_verbs))
   - getVerbsByTarget_internal: Delegate to underlying storage
   - Fixes catastrophic performance bug (scanned all 40 types + all files)

Verification:
- Built successfully with TypeScript
- All 25 augmentation tests passing
- Tested against Workshop's 601 verb dataset
- Result: 0 verbs → 2 verbs returned ✓

Impact:
- VFS relationships now queryable without metadata files
- TypeAware performance: O(total_verbs) → O(matching_verbs)
- Compatible with existing data (backward compatible)
This commit is contained in:
David Snelling 2025-10-28 09:12:09 -07:00
parent 26204a7d67
commit dbb827a560
2 changed files with 24 additions and 137 deletions

View file

@ -1388,11 +1388,9 @@ export class FileSystemStorage extends BaseStorage {
// Get metadata which contains the actual verb information
const metadata = await this.getVerbMetadata(id)
// v4.0.0: No fallbacks - skip verbs without metadata
if (!metadata) {
console.warn(`Verb ${id} has no metadata, skipping`)
continue
}
// v4.8.1: Don't skip verbs without metadata - metadata is optional
// FIX: This was the root cause of the VFS bug (11 versions)
// Verbs can exist without metadata files (e.g., from imports/migrations)
// Convert connections Map to proper format if needed
let connections = edge.connections
@ -1405,7 +1403,7 @@ export class FileSystemStorage extends BaseStorage {
}
// v4.8.0: Extract standard fields from metadata to top-level
const metadataObj = metadata as VerbMetadata
const metadataObj = (metadata || {}) as VerbMetadata
const { createdAt, updatedAt, confidence, weight, service, data: dataField, createdBy, ...customMetadata } = metadataObj
const verbWithMetadata: HNSWVerbWithMetadata = {
@ -2426,11 +2424,9 @@ export class FileSystemStorage extends BaseStorage {
const edge = JSON.parse(data)
const metadata = await this.getVerbMetadata(id)
// v4.0.0: No fallbacks - skip verbs without metadata
if (!metadata) {
processedCount++
return true // continue, skip this verb
}
// v4.8.1: Don't skip verbs without metadata - metadata is optional
// FIX: This was the root cause of the VFS bug (11 versions)
// Verbs can exist without metadata files (e.g., from imports/migrations)
// Convert connections if needed
let connections = edge.connections
@ -2443,7 +2439,7 @@ export class FileSystemStorage extends BaseStorage {
}
// v4.8.0: Extract standard fields from metadata to top-level
const metadataObj = metadata as VerbMetadata
const metadataObj = (metadata || {}) as VerbMetadata
const { createdAt, updatedAt, confidence, weight, service, data: dataField, createdBy, ...customMetadata } = metadataObj
const verbWithMetadata: HNSWVerbWithMetadata = {