brainy/docs/guides/MIGRATING_TO_V5.11.md

Migrating to v5.11.1

Overview

v5.11.1 introduces a breaking change with massive performance benefits:

• brain.get() now loads metadata-only by default (76-81% faster!)
• Vector embeddings require explicit opt-in: { includeVectors: true }

Impact: Only ~6% of codebases need changes (code that computes similarity on retrieved entities).

What Changed

Before (v5.11.0 and earlier)

const entity = await brain.get(id)
// entity.vector was ALWAYS loaded (384 dimensions, 6KB)
console.log(entity.vector.length)  // 384

After (v5.11.1)

// DEFAULT: Metadata-only (76-81% faster)
const entity = await brain.get(id)
console.log(entity.vector)  // [] (empty array - not loaded)

// EXPLICIT: Full entity with vectors
const entity = await brain.get(id, { includeVectors: true })
console.log(entity.vector.length)  // 384

Who Needs to Update?

✅ NO CHANGES NEEDED (94% of code)

If you use brain.get() for:

• VFS operations (readFile, stat, readdir)
• Existence checks: if (await brain.get(id))
• Metadata access: entity.data, entity.type, entity.metadata
• Relationship traversal
• Admin tools, import utilities, data APIs

→ Zero changes needed, automatic 76-81% speedup!

⚠️ REQUIRES UPDATE (~6% of code)

If you use brain.get() AND then compute similarity on the returned entity:

// ❌ BEFORE (v5.11.0) - will break in v5.11.1
const entity = await brain.get(id)
const similar = await brain.similar({ to: entity.vector })  // entity.vector is [] !

// ✅ AFTER (v5.11.1) - add includeVectors
const entity = await brain.get(id, { includeVectors: true })
const similar = await brain.similar({ to: entity.vector })  // Works!

Note: brain.similar({ to: entityId }) (using ID) still works - no changes needed!

Migration Steps

Step 1: Find Affected Code

Search your codebase for patterns that use vectors from brain.get():

# Find brain.get() calls that access .vector
grep -r "await brain.get(" --include="*.ts" --include="*.js" | \
  grep -E "(\.vector|entity\.vector)"

Step 2: Update Pattern-by-Pattern

Pattern 1: Similarity Using Retrieved Entity Vector

// ❌ BEFORE
const entity = await brain.get(id)
const similar = await brain.similar({ to: entity.vector })

// ✅ AFTER - Option A: Add includeVectors
const entity = await brain.get(id, { includeVectors: true })
const similar = await brain.similar({ to: entity.vector })

// ✅ AFTER - Option B: Use ID directly (recommended)
const similar = await brain.similar({ to: id })

Pattern 2: Manual Vector Operations

// ❌ BEFORE
const entity = await brain.get(id)
const magnitude = Math.sqrt(entity.vector.reduce((sum, v) => sum + v*v, 0))

// ✅ AFTER
const entity = await brain.get(id, { includeVectors: true })
const magnitude = Math.sqrt(entity.vector.reduce((sum, v) => sum + v*v, 0))

Pattern 3: Vector Assertions in Tests

// ❌ BEFORE
const entity = await brain.get(id)
expect(entity.vector).toBeDefined()
expect(entity.vector.length).toBe(384)

// ✅ AFTER
const entity = await brain.get(id, { includeVectors: true })
expect(entity.vector).toBeDefined()
expect(entity.vector.length).toBe(384)

Step 3: Verify Migration

Run your test suite to catch any remaining issues:

npm test

Look for errors like:

• entity.vector is empty or entity.vector.length is 0
• Cannot compute similarity on empty vector

Add { includeVectors: true } wherever these errors occur.

Performance Impact

Before Migration

brain.get(): 43ms, 6KB per call
VFS readFile(): 53ms per file
VFS readdir(100 files): 5.3s

After Migration

brain.get(): 10ms, 300 bytes per call (76-81% faster) ✨
brain.get({ includeVectors: true }): 43ms, 6KB (unchanged)
VFS readFile(): ~13ms per file (75% faster) ✨
VFS readdir(100 files): ~1.3s (75% faster) ✨

Result:

• VFS operations: 75% faster
• Metadata access: 76-81% faster
• Vector similarity: Unchanged (still fast when needed)

TypeScript Support

The new GetOptions interface is fully typed:

interface GetOptions {
  /**
   * Include 384-dimensional vector embeddings in the response
   *
   * Default: false (metadata-only for 76-81% speedup)
   */
  includeVectors?: boolean
}

// TypeScript will autocomplete and validate
const entity = await brain.get(id, { includeVectors: true })

Rollback Plan

If you encounter issues, you can temporarily force full entity loading everywhere:

// Temporary wrapper (NOT RECOMMENDED - defeats optimization)
async function getLegacy(id: string) {
  return brain.get(id, { includeVectors: true })
}

// Use throughout codebase while migrating
const entity = await getLegacy(id)

Important: This defeats the 76-81% performance improvement. Only use temporarily while fixing affected code.

FAQ

Q: Why did you make this a breaking change?

A: The performance gains are massive (76-81% speedup, 95% less bandwidth) and affect 94% of code positively. Only ~6% of code needs updates. The net benefit is enormous.

Q: Do I need to update my VFS code?

A: No! VFS automatically benefits from the optimization with zero code changes. Your VFS operations are now 75% faster automatically.

Q: Will brain.similar() still work?

A: Yes! brain.similar({ to: entityId }) works exactly as before. Only brain.similar({ to: entity.vector }) requires the entity to be loaded with { includeVectors: true }.

Q: What about backward compatibility?

A: Entities returned without vectors have vector: [] (empty array), which is type-safe. Code that doesn't use vectors continues to work. Only code that explicitly uses entity.vector needs updating.

Q: Can I check if vectors are loaded?

A: Yes! Check entity.vector.length > 0 to detect if vectors were loaded.

const entity = await brain.get(id)
if (entity.vector.length > 0) {
  // Vectors are loaded
} else {
  // Metadata-only
}

Support

If you encounter migration issues:

1. Check the VFS Performance Guide
2. Review API Reference
3. See Performance Documentation
4. File an issue: https://github.com/soulcraft/brainy/issues

Changelog

See CHANGELOG.md for complete v5.11.1 release notes.