Commit graph

2 commits

Author SHA1 Message Date
e476d45fac **feat(models): add pre-bundled Universal Sentence Encoder for offline use**
- Introduced `@soulcraft/brainy-models` package with pre-bundled TensorFlow models for enhanced offline reliability.
- Added `index.d.ts` and `index.js` allowing offline embedding workflows with the Universal Sentence Encoder model.
- Included utility scripts for model compression, size retrieval, and availability checks.
- Added `metadata.json` and `model.json` defining the Universal Sentence Encoder configuration with offline bundling.
- Ensured comprehensive model documentation, error handling, and robust logging for seamless integration.
- Supported optional model quantization placeholders for future TensorFlow.js enhancements.

**Purpose**: Enable fully offline-ready embedding workflows via pre-bundled Universal Sentence Encoder models, ensuring maximum reliability and air-gapped environment compatibility.
2025-08-01 16:22:58 -07:00
563b983fcc **feat(models): add scripts for model compression, bundling, and optimization**
- Added new scripts under `brainy-models-package/scripts`:
  - **`compress-models.js`**: Implements model compression with float16 and int8 precision to create optimized variants of Universal Sentence Encoder models.
  - **`download-full-models.js`**: Downloads the complete Universal Sentence Encoder model for offline usage.
  - **`download-model.js`**: Downloads reference files for TensorFlow Hub-based Universal Sentence Encoder.

- Introduced a demonstration script:
  - **`demo-optional-model-bundling.js`**: Highlights the solution of bundling models to eliminate network dependency, ensuring reliability and offline capability.

- Key Features:
  - **Compression**:
    - Reduced model size with float16 (balanced precision and size) and int8 (low-memory environments) options.
    - Generated compression summaries for quick insights into model variants and saved space.
  - **Offline Reliability**:
    - Bundled versions eliminate first-load delays, network dependencies, and failures.
    - Ensures rapid initialization in offline and memory-constrained scenarios.
  - **Dynamic Optimization**:
    - Tailored optimization profiles for various use cases: general, low-memory, and high-performance.
  - **Demonstration and Documentation**:
    - Comprehensive demo showcasing benefits of bundled models over online loading.
    - Examples for usage, testing, and integration with Brainy.

**Purpose**: Introduce essential scripts and tools to enable efficient, offline-ready model usage, streamlining the embedding workflow while ensuring reliability in production and resource-constrained environments.
2025-08-01 15:35:08 -07:00