- Introduced `test-fallback-function.js` and `test-fallback-simple.js` to validate `executeInThread` fallback functionality with both named and anonymous compute-intensive functions.
- Added `test-tensorflow-textencoder.js` for TensorFlow.js and TextEncoder tests in a Node.js environment.
- Created `test-tensorflow-textencoder.html` for browser-based TensorFlow.js and TextEncoder tests.
- Implemented cross-environment test support in `cli-package/src/test-tensorflow-textencoder.ts` for CLI functionality.
- Enhanced `src/utils/embedding.ts`, `textEncoding.ts`, and `brainy-wrapper.js` to include updated global `TextEncoder` and `TextDecoder` utilities for compatibility and worker improvements.
- Standardized and expanded utility methods in `PlatformNode` for broader support, including `isFloat32Array` and `isTypedArray` checks.
- Updated Node.js requirement to `>= 24.4.0` across documentation and configuration files for compatibility improvements.
This update introduces comprehensive testing for fallback mechanisms, TensorFlow.js, and TextEncoder across multiple environments, ensuring robustness and compatibility.
- Updated Node.js version requirement to `>=24.3.0` in `README.md` and `package-lock.json`.
- Upgraded `TypeScript` to version `5.4.5` and synced references.
- Updated multiple dependencies, including:
- `@aws-sdk/client-s3` to `^3.540.0`.
- `uuid` to `^9.0.1`.
- Development tools such as `eslint` to `^8.57.0`, `typescript-eslint` plugins to `^7.4.0`, and more.
- Refreshed transitive dependencies with up-to-date versions for consistency and performance.
This update ensures compatibility with the latest Node.js features, aligns TypeScript references, and keeps the project dependencies up-to-date for improved stability and functionality.
- Refactored `TextEncoder` and `TextDecoder` in `src/utils/tensorflowUtils.ts` to handle environments without global availability, logging warnings and providing fallback implementations.
- Enhanced threading utilities in `src/utils/environment.ts`:
- Introduced `areWorkerThreadsAvailableSync()` and `isThreadingAvailableAsync()` for synchronous and asynchronous checks.
- Improved compatibility across browser and Node.js environments.
- Migrated `fileSystemStorage.ts` to use dynamic imports for Node.js modules, removing synchronous loading mechanisms to align with ES module standards.
- Updated `cli-package/package.json` keywords to include `browser` and `container`, improving discoverability.
- Added `worker_threads` as an external dependency in `cli-package/rollup.config.js` and `rollup.config.js`.
- Updated `.gitignore` to ignore all `tgz` files dynamically under main and CLI package.
This update improves threading compatibility, enhances environment-specific support, and aligns with modern module loading practices.
- Added `test-browser-worker.html` to demonstrate threading with Web Workers in browser environments.
- Added `test-fallback.html` to verify fallback functionality when threading is unavailable.
- Created `THREADING.md` to document unified threading implementation, including Node.js Worker Threads, Web Workers, and fallback mechanisms.
- Updated Node.js version requirement in `README.md` and `package.json` to `>=24.0.0` for compatibility with improved Worker Threads API.
- Enhanced `workerUtils.ts` to implement threading with a worker pool for Node.js and added Web Worker execution logic for browsers.
- Enabled environment-aware threading availability in `environment.ts`.
- Updated `rollup.config.js` to include CLI configurations for streamlined builds.
- Adjusted `.gitignore` to exclude test artifacts and package files.
These changes provide comprehensive testing and documentation of threading functionality, improve cross-environment compatibility, and enhance developer workflows.
- Added batch embedding support with `defaultBatchEmbeddingFunction`, leveraging shared model instances for optimized performance.
- Integrated `isInitializing` flag to prevent recursive initialization and ensure smooth concurrent operation handling during `BrainyData` initialization.
- Pre-loaded Universal Sentence Encoder in `BrainyData` to prevent delays during embedding.
- Introduced fallback mechanisms in embedding initialization for better error resiliency and model reusability.
- Updated `addBatch` with support for batchSize and refactored text/vector processing logic for clearer separation and memory management.
- Improved GPU and CPU backend selection in Universal Sentence Encoder for compatibility across environments.
- Enhanced memory management by cleaning tensors after embedding operations.
- Updated README with instructions for batch embedding, threading updates, and GPU/CPU optimizations.
