brainy/MIGRATION_PLAN_DEPRECATED_METHODS.md

Migration Plan: Deprecated Methods

Overview

This document outlines the migration plan for deprecated methods in the Brainy codebase. These methods are marked as deprecated due to potential memory issues with large datasets.

Deprecated Methods

1. Storage Adapter Methods

• getAllNouns() → Use getNouns() with pagination
• getAllVerbs() → Use getVerbs() with pagination
• getNounsByNounType(nounType) → Use getNouns({ nounType })
• getVerbsBySource(sourceId) → Use getVerbs({ sourceId })
• getVerbsByTarget(targetId) → Use getVerbs({ targetId })
• getVerbsByType(type) → Use getVerbs({ verbType: type })

2. HNSW Index Methods

• getNouns() → Use getNounsPaginated()

Migration Strategy

Phase 1: Update Internal BrainyData Usage (High Priority)

Files to update:

• src/brainyData.ts - 8 usages of deprecated methods
• src/augmentations/memoryAugmentations.ts - 1 usage
• src/mcp/brainyMCPAdapter.ts - 2 usages

Specific Changes:

src/brainyData.ts

// OLD: 
const nouns = await this.storage!.getAllNouns()

// NEW:
const nouns: HNSWNoun[] = []
let cursor: SearchCursor | undefined
do {
  const result = await this.storage!.getNouns({}, { limit: 1000, cursor })
  nouns.push(...result.results)
  cursor = result.cursor
} while (cursor)

src/augmentations/memoryAugmentations.ts

// OLD:
const nodes = await this.storage.getAllNouns()

// NEW: 
const nodes = await this.storage.getNouns({}, { limit: Number.MAX_SAFE_INTEGER })

src/mcp/brainyMCPAdapter.ts

// OLD:
const outgoing = await (this.brainyData as any).getVerbsBySource?.(id) || []

// NEW:
const outgoing = await this.brainyData.getVerbs({ sourceId: id }, { limit: Number.MAX_SAFE_INTEGER }) || []

Phase 2: Update Storage Adapter Implementations (Medium Priority)

Files to update:

• src/storage/adapters/memoryStorage.ts
• src/storage/adapters/fileSystemStorage.ts
• src/storage/adapters/opfsStorage.ts
• src/storage/adapters/s3CompatibleStorage.ts

Strategy: Keep deprecated methods but mark them as internal-only and implement them using the new filtered methods.

Phase 3: Update Type Definitions (Low Priority)

Files to update:

• src/coreTypes.ts - Remove deprecated method signatures
• src/storage/baseStorage.ts - Remove deprecated implementations

Phase 4: Update Tests (Low Priority)

Files to update:

• tests/*.test.ts - Update test files that use deprecated methods

Implementation Order

1. Immediate (Safe Changes)

• ✅ Remove unused files (tensorflowUtils.ts, etc.)
• ✅ Fix TODO items with version handling
• Update internal BrainyData usage with pagination

2. Short-term (1-2 weeks)

• Update augmentation and MCP adapter usage
• Add deprecation warnings to method implementations
• Update storage adapter internal implementations

3. Long-term (Next major version)

• Remove deprecated method signatures from interfaces
• Remove deprecated method implementations
• Update all test files

Backward Compatibility

Option 1: Gradual Deprecation (Recommended)

• Keep deprecated methods but add console warnings
• Implement them using new filtered methods internally
• Remove in next major version (0.42.0 → 1.0.0)

Option 2: Immediate Removal

• Remove deprecated methods now
• Update all usage immediately
• Risk: Breaking changes for external users

Testing Strategy

1. Unit Tests: Update existing tests to use new methods
2. Integration Tests: Verify pagination works correctly
3. Performance Tests: Ensure new implementations don't regress performance
4. Memory Tests: Verify large dataset handling improves

Rollback Plan

If issues arise:

1. Revert to previous implementations
2. Add memory limits as temporary solution
3. Implement pagination more gradually

Success Criteria

• ✅ All deprecated method usage removed from core files
• ✅ No memory issues with large datasets
• ✅ Performance maintained or improved
• ✅ All tests passing
• ✅ Backward compatibility preserved (if Option 1)

Timeline

• Week 1: Complete Phase 1 (internal usage)
• Week 2: Complete Phase 2 (storage adapters)
• Week 3: Complete Phase 3 (type definitions)
• Week 4: Complete Phase 4 (tests) and validation

Notes

• The deprecated methods are still widely used, so this migration requires careful planning
• Consider adding configuration option to control pagination limits
• Document the breaking changes clearly in CHANGELOG.md
• Consider providing migration utilities for external users