feat: add GPU acceleration for embeddings with smart device auto-detection

Browse source

Add comprehensive GPU support for embedding generation while maintaining optimized CPU processing for distance calculations:

- Add device option to TransformerEmbeddingOptions (auto, cpu, webgpu, cuda, gpu)
- Implement smart auto-detection of best available GPU (WebGPU for browsers, CUDA for Node.js)
- Add automatic CPU fallback if GPU initialization fails
- Fix misleading GPU acceleration claims in distance functions and HNSW search
- Update documentation to accurately reflect GPU usage (embeddings only)
- Add comprehensive example demonstrating GPU acceleration usage
- Maintain full backward compatibility with existing code

Performance improvements: 3-5x faster embedding generation when GPU is available, while keeping faster CPU processing for 384-dim vector distance calculations.

...

This commit is contained in:

David Snelling 2025-08-05 20:00:04 -07:00

parent d58e9e50c2

commit 64e2c63c2a

7 changed files with 572 additions and 420 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

797

package-lock.json generated

View file

File diff suppressed because it is too large Load diff