- Update RobustModelLoader to properly handle @tensorflow-models/universal-sentence-encoder
- Add support for loading USE-lite model with tokenizer from local files
- Fix file:// URL handling issues in Node.js environment
- Improve fallback mechanism for model loading
- Add better error messages and logging for debugging
- Add better error handling for @soulcraft/brainy-models package loading
- Log model metadata when available for debugging
- Try alternative loading methods if primary method fails
- Update fallback URLs to working endpoints
- Add more comprehensive path checking for bundled models
- Improve error messages to help diagnose loading issues
- Remove Rollup bundling in favor of direct TypeScript compilation
- Move from bundled models to dynamic model loading with configurable paths
- Add Docker deployment examples and documentation
- Implement robust model loader with fallback mechanisms
- Update storage adapters for better cross-environment compatibility
- Add comprehensive tests for model loading and package installation
- Simplify package.json scripts and remove complex build configurations
- Clean up deprecated demo files and old bundling scripts
BREAKING CHANGE: Models are no longer bundled with the package. They are now loaded dynamically from CDN or custom paths.
Add @soulcraft/brainy-models as optional dependency for zero-config offline reliability. Enhance robustModelLoader with hierarchical loading strategy (local → online → fail). Add comprehensive production deployment documentation and update README with clear benefits.
This solves critical production issues where Universal Sentence Encoder fails to load in Docker/Cloud Run environments due to network timeouts or blocked URLs. The solution provides 100% offline reliability while maintaining backward compatibility and requires no code changes from users.
- **Compatibility Enhancements**:
- Added support to detect and inject missing `"format"` field in `model.json` files for TensorFlow.js compatibility.
- Modified model loading logic to handle both `tfjs-graph-model` and `tfjs-layers-model` formats.
- **New Features**:
- Introduced additional fallback paths for locating models to increase reliability in varying environments.
- Added support for mock implementations of the Universal Sentence Encoder in test environments.
- **Bug Fixes**:
- Fixed module loading resolution in `FileSystemStorage` with improved initialization and error handling for Node.js environments.
- Resolved issues with test assertions to improve validation logic in core tests.
**Purpose**: Improve model loading reliability, expand compatibility with TensorFlow.js models, and enhance test environment support.
- Introduced new documentation files under `docs/`:
- `model-bundling-analysis.md`: Provides detailed analysis of current, bundled, hybrid, and dynamic model loading approaches, including pros, cons, and recommendations.
- `model-management.md`: Explains how Brainy manages Universal Sentence Encoder models, including setup, usage, and troubleshooting.
- `optional-model-bundling.md`: Details the `@soulcraft/brainy-models` package for offline reliability with pre-bundled models.
- Added `src/utils/robustModelLoader.ts`:
- Implements enhanced model loading with retry mechanisms, timeout handling, fallback URLs, and optional local model bundling.
- Supports Node.js and browser environments with exponential backoff logic.
- Key Updates:
- **Hybrid Loading Strategy**: Recommended for balancing reliability and flexibility via hybrid online/offline mechanisms.
- **Enhanced Fallback Scenarios**: Robust loader improves network-dependent reliability for embedding workflows.
- **Offline Reliability Support**: Optional model bundling eliminates dependency on external services, supporting air-gapped and edge environments.
**Purpose**: Introduce a hybrid model loading approach with robust options for
