- Add complete Universal Sentence Encoder Lite model (27MB)
- Include vocab.json for tokenization support
- Update package to work with @tensorflow-models/universal-sentence-encoder
- Ensure offline model loading capability for Docker deployments
- Published to npm as @soulcraft/brainy-models@0.8.0
Remove development artifacts, test files, and redundant directories:
- Delete debug/reproduction scripts and temporary test files
- Remove brainy-models-package/ (redundant with main models/ directory)
- Remove test-consumer/ development testing directory
- Remove build artifacts (coverage/, test-results.json)
- Remove large brainy-data/ test artifact directory
This cleanup reduces repository size significantly and prepares the project for a clean release.
- **Documentation**:
- Added a detailed explanation in `model-management.md` for resolving `"format"` field compatibility issues in TensorFlow.js.
- Introduced a dual-layer protection approach to mitigate errors like `RangeError: byte length of Float32Array should be a multiple of 4`.
- **Scripts**:
- Removed the redundant `release:minor` script entry from `package.json`.
- Enhanced `_deploy` script consistency.
- **Protection Mechanisms**:
- Updated `download-full-models.js` to inject a missing `"format"` field during downloads.
- Enhanced `RobustModelLoader` to validate and restore the `"format"` field automatically at runtime, ensuring persistence and compatibility
- Removed `demo-optional-model-bundling.js`:
- Obsolete demonstration of model bundling and offline loading.
- Replaced by comprehensive documentation and tools in `@soulcraft/brainy-models`.
- Added `create-github-release.js`:
- Automates GitHub release creation for `@soulcraft/brainy-models-package`.
- Includes features for generating release notes, tagging, and uploading with GitHub CLI.
- Updated `package-lock.json`:
- Reflects new dependencies and updates for GitHub release automation.
**Purpose**: Streamline repository by removing redundant scripts and introducing automated GitHub release workflows for efficient version management.
- Added new scripts under `brainy-models-package/scripts`:
- **`compress-models.js`**: Implements model compression with float16 and int8 precision to create optimized variants of Universal Sentence Encoder models.
- **`download-full-models.js`**: Downloads the complete Universal Sentence Encoder model for offline usage.
- **`download-model.js`**: Downloads reference files for TensorFlow Hub-based Universal Sentence Encoder.
- Introduced a demonstration script:
- **`demo-optional-model-bundling.js`**: Highlights the solution of bundling models to eliminate network dependency, ensuring reliability and offline capability.
- Key Features:
- **Compression**:
- Reduced model size with float16 (balanced precision and size) and int8 (low-memory environments) options.
- Generated compression summaries for quick insights into model variants and saved space.
- **Offline Reliability**:
- Bundled versions eliminate first-load delays, network dependencies, and failures.
- Ensures rapid initialization in offline and memory-constrained scenarios.
- **Dynamic Optimization**:
- Tailored optimization profiles for various use cases: general, low-memory, and high-performance.
- **Demonstration and Documentation**:
- Comprehensive demo showcasing benefits of bundled models over online loading.
- Examples for usage, testing, and integration with Brainy.
**Purpose**: Introduce essential scripts and tools to enable efficient, offline-ready model usage, streamlining the embedding workflow while ensuring reliability in production and resource-constrained environments.
