- Refactored `TextEncoder` and `TextDecoder` in `src/utils/tensorflowUtils.ts` to handle environments without global availability, logging warnings and providing fallback implementations.
- Enhanced threading utilities in `src/utils/environment.ts`:
- Introduced `areWorkerThreadsAvailableSync()` and `isThreadingAvailableAsync()` for synchronous and asynchronous checks.
- Improved compatibility across browser and Node.js environments.
- Migrated `fileSystemStorage.ts` to use dynamic imports for Node.js modules, removing synchronous loading mechanisms to align with ES module standards.
- Updated `cli-package/package.json` keywords to include `browser` and `container`, improving discoverability.
- Added `worker_threads` as an external dependency in `cli-package/rollup.config.js` and `rollup.config.js`.
- Updated `.gitignore` to ignore all `tgz` files dynamically under main and CLI package.
This update improves threading compatibility, enhances environment-specific support, and aligns with modern module loading practices.
- Added `test-browser-worker.html` to demonstrate threading with Web Workers in browser environments.
- Added `test-fallback.html` to verify fallback functionality when threading is unavailable.
- Created `THREADING.md` to document unified threading implementation, including Node.js Worker Threads, Web Workers, and fallback mechanisms.
- Updated Node.js version requirement in `README.md` and `package.json` to `>=24.0.0` for compatibility with improved Worker Threads API.
- Enhanced `workerUtils.ts` to implement threading with a worker pool for Node.js and added Web Worker execution logic for browsers.
- Enabled environment-aware threading availability in `environment.ts`.
- Updated `rollup.config.js` to include CLI configurations for streamlined builds.
- Adjusted `.gitignore` to exclude test artifacts and package files.
These changes provide comprehensive testing and documentation of threading functionality, improve cross-environment compatibility, and enhance developer workflows.
- Added batch embedding support with `defaultBatchEmbeddingFunction`, leveraging shared model instances for optimized performance.
- Integrated `isInitializing` flag to prevent recursive initialization and ensure smooth concurrent operation handling during `BrainyData` initialization.
- Pre-loaded Universal Sentence Encoder in `BrainyData` to prevent delays during embedding.
- Introduced fallback mechanisms in embedding initialization for better error resiliency and model reusability.
- Updated `addBatch` with support for batchSize and refactored text/vector processing logic for clearer separation and memory management.
- Improved GPU and CPU backend selection in Universal Sentence Encoder for compatibility across environments.
- Enhanced memory management by cleaning tensors after embedding operations.
- Updated README with instructions for batch embedding, threading updates, and GPU/CPU optimizations.
