- Remove @huggingface/transformers dependency (539MB native binaries)
- Add direct ONNX Runtime Web embedding engine
- Bundle all-MiniLM-L6-v2-q8 model (24MB, no runtime downloads)
- Works with Node.js, Bun, and bun build --compile
- Air-gap compatible: fully self-contained, no internet required
New WASM embedding components:
- WASMEmbeddingEngine: Main integration class
- WordPieceTokenizer: Pure TypeScript tokenizer
- EmbeddingPostProcessor: Mean pooling + L2 normalization
- ONNXInferenceEngine: Direct ONNX Runtime Web wrapper
- AssetLoader: Model file loading
Tests added:
- 11 WASM embedding integration tests
- 8 Bun compatibility tests
New npm scripts:
- test:wasm - Run WASM embedding tests
- test:bun - Run tests with Bun
- test:bun:compile - Build and run compiled binary
Fixed confusing messaging where comments mentioned Node 24 while actual requirement is Node 22:
- environment.ts: Fixed areWorkerThreadsAvailableSync() to check for >= 22 instead of >= 24
- workerUtils.ts: Updated comments to reference Node.js 22 LTS instead of 24
- embedding.ts: Updated ONNX threading comment to reference Node.js 22 LTS
Why Node 22 not Node 24:
- ONNX Runtime has stability issues in Node 24 (V8 HandleScope crashes)
- Node 22 LTS provides maximum stability with our embedding model
- These stale Node 24 references were causing user confusion
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Remove top-level Node.js imports that break bundlers
- Use universal adapters for crypto operations
- Add dynamic imports for Node.js-specific modules
- Add browser field to package.json for bundler hints
- Maintain full Node.js functionality while enabling browser usage
This allows Brainy to be used with modern bundlers (Vite, Webpack, etc.)
without requiring Node.js polyfills. Browser environments get core features
while Node.js retains all capabilities including filesystem and networking.
- Updated all fs, path, crypto, os, url, util, events, http, https, net, child_process, stream, and zlib imports
- Changed both static imports and dynamic imports to use node: protocol
- This makes Brainy more bundler-friendly by explicitly marking Node.js built-ins
- Prevents bundlers from attempting to polyfill or bundle these modules
- Reduces bundle size for web applications using Brainy
- Improves tree-shaking and dead code elimination
Benefits for external bundlers:
- Clear distinction between Node.js built-ins and external dependencies
- No ambiguity about what needs polyfilling
- Smaller bundles for browser builds
- Better compatibility with modern bundlers (Webpack 5, Vite, Rollup, esbuild)
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
- Unified embedding system with single EmbeddingManager
- Q8 model support with 75% smaller footprint (23MB vs 90MB)
- Intelligent precision auto-selection based on environment
- Clean cached embeddings with TTL and memory management
- Zero-config setup with smart defaults
- Complete storage structure documentation
- Removed legacy worker and hybrid managers
- Streamlined model configuration and precision management
- Changed confusing 'dtype' to 'precision' for model variant selection
- Fixed Q8 quantized model loading in transformers.js pipeline
- Added proper model file detection for q8 vs fp32 models
- Updated all references across codebase to use new parameter
- Maintains backward compatibility while providing clearer API
Changed default dtype from q8 to fp32 across all embedding implementations:
- embedding.ts: Default dtype changed to fp32
- worker-embedding.ts: Use fp32 for consistency
- universal-memory-manager.ts: Use fp32 for consistency
- lightweight-embedder.ts: Use fp32 for consistency
- hybridModelManager.ts: Use fp32 for all configurations
This ensures we use the exact same model (model.onnx) everywhere,
maintaining data compatibility and avoiding 404 errors for quantized models.
Models should download automatically when not present locally. Fixed the
environment variable check to only block downloads when explicitly set to 'false'
rather than blocking when undefined.
