Commit graph

4 commits

Author SHA1 Message Date
19ed3dd081 **refactor(tests): consolidate and replace outdated environment test scripts**
- Removed obsolete scripts: `test-all-environments.js`, `test-fallback-function.js`, `test-fallback-simple.js`, `test-fix.js`, `test-tensorflow-textencoder.js`, `test-unified-encoding.js`, and `test-worker-utils.js`.
- Introduced `scripts/comprehensive-test.js` as a unified testing script covering all environments: Browser, Node.js, and CLI.
- Added `examples/cli-wrapper-example.js` to demonstrate a proper CLI implementation with TensorFlow.js initialization.

This refactor simplifies the testing structure by consolidating redundant scripts into a single comprehensive script while ensuring robust cross-environment coverage.
2025-07-14 11:12:51 -07:00
bba9a0c219 feat(src/brainyData, src/utils): enhance embedding efficiency with batch processing and initialize safeguards
- Added batch embedding support with `defaultBatchEmbeddingFunction`, leveraging shared model instances for optimized performance.
- Integrated `isInitializing` flag to prevent recursive initialization and ensure smooth concurrent operation handling during `BrainyData` initialization.
- Pre-loaded Universal Sentence Encoder in `BrainyData` to prevent delays during embedding.
- Introduced fallback mechanisms in embedding initialization for better error resiliency and model reusability.
- Updated `addBatch` with support for batchSize and refactored text/vector processing logic for clearer separation and memory management.
- Improved GPU and CPU backend selection in Universal Sentence Encoder for compatibility across environments.
- Enhanced memory management by cleaning tensors after embedding operations.
- Updated README with instructions for batch embedding, threading updates, and GPU/CPU optimizations.
2025-06-27 14:06:59 -07:00
6a8d044970 feat(src/utils): add GPU acceleration and improve threading for embeddings and distance calculations
- Integrated GPU acceleration via WebGL in `embedding.ts` and `distance.ts` for optimized performance.
- Enhanced fallback mechanisms for CPU processing when GPU is unavailable or fails to initialize.
- Added threading support for embedding and distance calculations in `embedding.ts` and `distance.ts` to improve performance and avoid blocking the main thread.
- Refactored TensorFlow.js imports to dynamically load modules (`@tensorflow/tfjs-core`, `@tensorflow/tfjs-backend-webgl`, `@tensorflow/tfjs-backend-cpu`) for modular dependency usage.
- Introduced GPU-accelerated batch distance calculations in `distance.ts` with appropriate error handling.
- Applied consistent formatting to improve code readability and maintainability while adhering to project style guidelines.
2025-06-26 19:04:37 -07:00
5a8a6c1ba3 Initial commit 2025-06-24 11:41:30 -07:00