David Snelling
42571c5883
- Introduced new documentation files under `docs/`: - `model-bundling-analysis.md`: Provides detailed analysis of current, bundled, hybrid, and dynamic model loading approaches, including pros, cons, and recommendations. - `model-management.md`: Explains how Brainy manages Universal Sentence Encoder models, including setup, usage, and troubleshooting. - `optional-model-bundling.md`: Details the `@soulcraft/brainy-models` package for offline reliability with pre-bundled models. - Added `src/utils/robustModelLoader.ts`: - Implements enhanced model loading with retry mechanisms, timeout handling, fallback URLs, and optional local model bundling. - Supports Node.js and browser environments with exponential backoff logic. - Key Updates: - **Hybrid Loading Strategy**: Recommended for balancing reliability and flexibility via hybrid online/offline mechanisms. - **Enhanced Fallback Scenarios**: Robust loader improves network-dependent reliability for embedding workflows. - **Offline Reliability Support**: Optional model bundling eliminates dependency on external services, supporting air-gapped and edge environments. **Purpose**: Introduce a hybrid model loading approach with robust options for
5.1 KiB
5.1 KiB
Model Bundling Analysis
Current Approach vs. Bundling Options
Current Approach: Dynamic Loading from TensorFlow Hub
How it works:
- Small reference files (~3KB) point to TensorFlow Hub URLs
- Full model (~25MB) downloaded on first use from TensorFlow Hub
- Relies on TensorFlow.js built-in caching
Pros:
- Small package size (only ~3KB reference files)
- Always uses latest model from TensorFlow Hub
- Works across all environments (browser, Node.js, serverless)
- No licensing concerns (model hosted by Google)
Cons:
- Network dependency: Requires internet connection on first use
- Reliability issues: Single point of failure (TensorFlow Hub)
- Performance: Initial load can be slow (~25MB download)
- Timeout issues: No retry mechanisms or timeout handling
- Deployment issues: Can fail in restricted network environments
Option 1: Full Model Bundling
How it would work:
- Include the full 25MB model files in the npm package
- Load model directly from local files
- No network dependency after installation
Pros:
- Maximum reliability: No network dependency
- Fast loading: Immediate availability
- Offline support: Works without internet
- Predictable performance: No network variability
Cons:
- Large package size: +25MB to npm package
- Storage overhead: Every installation includes full model
- Update complexity: Model updates require package updates
- Licensing considerations: Need to verify redistribution rights
- CDN costs: Increased bandwidth costs for npm registry
Option 2: Hybrid Approach (Recommended)
How it would work:
- Provide optional separate model package (
@soulcraft/brainy-models) - Enhanced loader tries local bundled model first, falls back to TensorFlow Hub
- Robust retry mechanisms and fallback URLs
- Configurable loading strategy
Pros:
- Best of both worlds: Reliability when bundled, fallback when not
- Flexible deployment: Users choose based on their needs
- Backward compatibility: Existing installations continue to work
- Improved reliability: Retry mechanisms and fallbacks
- Optional bundling: Users can opt-in to local models
Cons:
- Complexity: More complex loading logic
- Documentation: Need to explain both approaches
- Testing: Need to test both scenarios
Option 3: Enhanced Dynamic Loading (Minimal Change)
How it would work:
- Keep current approach but add robust retry mechanisms
- Add multiple fallback URLs
- Implement timeout handling and exponential backoff
- Better error handling and logging
Pros:
- Minimal disruption: Small changes to existing code
- Improved reliability: Addresses current issues
- Maintains small package size: No size increase
- Easy to implement: Can be done quickly
Cons:
- Still network dependent: Fundamental reliability issue remains
- Limited offline support: Still requires internet on first use
- Fallback URL maintenance: Need to maintain list of working URLs
Recommendation: Hybrid Approach
Based on the analysis, I recommend implementing Option 2: Hybrid Approach because:
- Addresses the core issue: Provides reliability through local bundling option
- Maintains flexibility: Users can choose their preferred approach
- Backward compatible: Existing users aren't affected
- Future-proof: Can evolve based on user feedback
Implementation Plan
Phase 1: Enhanced Dynamic Loading (Immediate)
- Implement robust model loader with retries and timeouts
- Add fallback URLs for Universal Sentence Encoder
- Improve error handling and logging
- Impact: Significantly improves reliability with minimal changes
Phase 2: Optional Model Bundling (Future)
- Create separate
@soulcraft/brainy-modelspackage - Add detection logic for bundled models
- Update documentation with bundling options
- Impact: Provides maximum reliability for users who need it
Phase 3: Advanced Features (Future)
- Model compression and optimization
- Progressive loading strategies
- Custom model support
- Impact: Further performance and flexibility improvements
Configuration Options
// Enhanced loading with retries (Phase 1)
const encoder = new UniversalSentenceEncoder({
maxRetries: 3,
timeout: 60000,
useExponentialBackoff: true,
verbose: true
})
// With optional bundled model (Phase 2)
const encoder = new UniversalSentenceEncoder({
preferLocalModel: true,
fallbackUrls: ['https://backup-url.com/model'],
maxRetries: 3
})
Risk Assessment
Low Risk
- Enhanced dynamic loading (Phase 1)
- Backward compatibility maintained
- No breaking changes
Medium Risk
- Optional model bundling (Phase 2)
- Need to verify licensing for redistribution
- Additional testing complexity
High Risk
- Full model bundling (Option 1)
- Significant package size increase
- Potential npm registry issues
Conclusion
The hybrid approach provides the best balance of reliability, flexibility, and maintainability. Starting with enhanced dynamic loading (Phase 1) addresses the immediate reliability issues with minimal risk, while keeping the door open for optional bundling in the future.