David Snelling
bbc77f292b
Remove unused files and implement proper version handling: - Remove unused files: tensorflowUtils.ts, patched-platform-node.ts, test reporters - Fix 5 TODO items with centralized version management in utils/version.ts - Clean up duplicate metadata definitions in examples/basicUsage.ts - Fix rollup config to use @rollup/plugin-terser instead of deprecated package - Add comprehensive migration plan for deprecated methods (12 methods identified) This cleanup removes 370 lines of dead code while maintaining full API compatibility. All tests pass and build system works correctly.
143 lines
No EOL
4.5 KiB
Markdown
143 lines
No EOL
4.5 KiB
Markdown
# Migration Plan: Deprecated Methods
|
|
|
|
## Overview
|
|
This document outlines the migration plan for deprecated methods in the Brainy codebase. These methods are marked as deprecated due to potential memory issues with large datasets.
|
|
|
|
## Deprecated Methods
|
|
|
|
### 1. Storage Adapter Methods
|
|
- `getAllNouns()` → Use `getNouns()` with pagination
|
|
- `getAllVerbs()` → Use `getVerbs()` with pagination
|
|
- `getNounsByNounType(nounType)` → Use `getNouns({ nounType })`
|
|
- `getVerbsBySource(sourceId)` → Use `getVerbs({ sourceId })`
|
|
- `getVerbsByTarget(targetId)` → Use `getVerbs({ targetId })`
|
|
- `getVerbsByType(type)` → Use `getVerbs({ verbType: type })`
|
|
|
|
### 2. HNSW Index Methods
|
|
- `getNouns()` → Use `getNounsPaginated()`
|
|
|
|
## Migration Strategy
|
|
|
|
### Phase 1: Update Internal BrainyData Usage (High Priority)
|
|
**Files to update:**
|
|
- `src/brainyData.ts` - 8 usages of deprecated methods
|
|
- `src/augmentations/memoryAugmentations.ts` - 1 usage
|
|
- `src/mcp/brainyMCPAdapter.ts` - 2 usages
|
|
|
|
**Specific Changes:**
|
|
|
|
#### src/brainyData.ts
|
|
```typescript
|
|
// OLD:
|
|
const nouns = await this.storage!.getAllNouns()
|
|
|
|
// NEW:
|
|
const nouns: HNSWNoun[] = []
|
|
let cursor: SearchCursor | undefined
|
|
do {
|
|
const result = await this.storage!.getNouns({}, { limit: 1000, cursor })
|
|
nouns.push(...result.results)
|
|
cursor = result.cursor
|
|
} while (cursor)
|
|
```
|
|
|
|
#### src/augmentations/memoryAugmentations.ts
|
|
```typescript
|
|
// OLD:
|
|
const nodes = await this.storage.getAllNouns()
|
|
|
|
// NEW:
|
|
const nodes = await this.storage.getNouns({}, { limit: Number.MAX_SAFE_INTEGER })
|
|
```
|
|
|
|
#### src/mcp/brainyMCPAdapter.ts
|
|
```typescript
|
|
// OLD:
|
|
const outgoing = await (this.brainyData as any).getVerbsBySource?.(id) || []
|
|
|
|
// NEW:
|
|
const outgoing = await this.brainyData.getVerbs({ sourceId: id }, { limit: Number.MAX_SAFE_INTEGER }) || []
|
|
```
|
|
|
|
### Phase 2: Update Storage Adapter Implementations (Medium Priority)
|
|
**Files to update:**
|
|
- `src/storage/adapters/memoryStorage.ts`
|
|
- `src/storage/adapters/fileSystemStorage.ts`
|
|
- `src/storage/adapters/opfsStorage.ts`
|
|
- `src/storage/adapters/s3CompatibleStorage.ts`
|
|
|
|
**Strategy:** Keep deprecated methods but mark them as internal-only and implement them using the new filtered methods.
|
|
|
|
### Phase 3: Update Type Definitions (Low Priority)
|
|
**Files to update:**
|
|
- `src/coreTypes.ts` - Remove deprecated method signatures
|
|
- `src/storage/baseStorage.ts` - Remove deprecated implementations
|
|
|
|
### Phase 4: Update Tests (Low Priority)
|
|
**Files to update:**
|
|
- `tests/*.test.ts` - Update test files that use deprecated methods
|
|
|
|
## Implementation Order
|
|
|
|
### 1. Immediate (Safe Changes)
|
|
- ✅ Remove unused files (tensorflowUtils.ts, etc.)
|
|
- ✅ Fix TODO items with version handling
|
|
- Update internal BrainyData usage with pagination
|
|
|
|
### 2. Short-term (1-2 weeks)
|
|
- Update augmentation and MCP adapter usage
|
|
- Add deprecation warnings to method implementations
|
|
- Update storage adapter internal implementations
|
|
|
|
### 3. Long-term (Next major version)
|
|
- Remove deprecated method signatures from interfaces
|
|
- Remove deprecated method implementations
|
|
- Update all test files
|
|
|
|
## Backward Compatibility
|
|
|
|
### Option 1: Gradual Deprecation (Recommended)
|
|
- Keep deprecated methods but add console warnings
|
|
- Implement them using new filtered methods internally
|
|
- Remove in next major version (0.42.0 → 1.0.0)
|
|
|
|
### Option 2: Immediate Removal
|
|
- Remove deprecated methods now
|
|
- Update all usage immediately
|
|
- Risk: Breaking changes for external users
|
|
|
|
## Testing Strategy
|
|
|
|
1. **Unit Tests**: Update existing tests to use new methods
|
|
2. **Integration Tests**: Verify pagination works correctly
|
|
3. **Performance Tests**: Ensure new implementations don't regress performance
|
|
4. **Memory Tests**: Verify large dataset handling improves
|
|
|
|
## Rollback Plan
|
|
|
|
If issues arise:
|
|
1. Revert to previous implementations
|
|
2. Add memory limits as temporary solution
|
|
3. Implement pagination more gradually
|
|
|
|
## Success Criteria
|
|
|
|
- ✅ All deprecated method usage removed from core files
|
|
- ✅ No memory issues with large datasets
|
|
- ✅ Performance maintained or improved
|
|
- ✅ All tests passing
|
|
- ✅ Backward compatibility preserved (if Option 1)
|
|
|
|
## Timeline
|
|
|
|
- **Week 1**: Complete Phase 1 (internal usage)
|
|
- **Week 2**: Complete Phase 2 (storage adapters)
|
|
- **Week 3**: Complete Phase 3 (type definitions)
|
|
- **Week 4**: Complete Phase 4 (tests) and validation
|
|
|
|
## Notes
|
|
|
|
- The deprecated methods are still widely used, so this migration requires careful planning
|
|
- Consider adding configuration option to control pagination limits
|
|
- Document the breaking changes clearly in CHANGELOG.md
|
|
- Consider providing migration utilities for external users |