feat: add optional Q8 quantized model support

- Add Q8 quantized models (75% smaller than FP32)
- Enhance download scripts with model variant selection
- Add smart model loading with availability detection
- Implement runtime warnings for Q8 compatibility
- Update documentation with Q8 usage examples
- Maintain 100% backward compatibility (FP32 default)

BREAKING CHANGE: None - FP32 remains default

🧠 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
David Snelling 2025-08-29 11:09:40 -07:00
parent 223311f4e7
commit 32df3ee6ae
8 changed files with 259 additions and 40 deletions

View file

@ -121,6 +121,37 @@ await brain.find("Documentation about authentication from last month")
- **Worker-based embeddings** - Non-blocking operations
- **Automatic caching** - Intelligent result caching
### Performance Optimization
**Q8 Quantized Models** - 75% smaller, faster loading (v2.8.0+)
```javascript
// Default: Full precision (fp32) - maximum compatibility
const brain = new BrainyData()
// Optimized: Quantized models (q8) - 75% smaller, 99% accuracy
const brainOptimized = new BrainyData({
embeddingOptions: { dtype: 'q8' }
})
```
**Model Comparison:**
- **FP32 (default)**: 90MB, 100% accuracy, maximum compatibility
- **Q8 (optional)**: 23MB, ~99% accuracy, faster loading
**When to use Q8:**
- ✅ New projects where size/speed matters
- ✅ Memory-constrained environments
- ✅ Mobile or edge deployments
- ❌ Existing projects with FP32 data (incompatible embeddings)
**Air-gap deployment:**
```bash
npm run download-models # Both models (recommended)
npm run download-models:q8 # Q8 only (space-constrained)
npm run download-models:fp32 # FP32 only (compatibility)
```
## 📚 Core API
### `search()` - Vector Similarity