feat: add optional Q8 quantized model support
- Add Q8 quantized models (75% smaller than FP32) - Enhance download scripts with model variant selection - Add smart model loading with availability detection - Implement runtime warnings for Q8 compatibility - Update documentation with Q8 usage examples - Maintain 100% backward compatibility (FP32 default) BREAKING CHANGE: None - FP32 remains default 🧠 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
223311f4e7
commit
32df3ee6ae
8 changed files with 259 additions and 40 deletions
31
README.md
31
README.md
|
|
@ -121,6 +121,37 @@ await brain.find("Documentation about authentication from last month")
|
|||
- **Worker-based embeddings** - Non-blocking operations
|
||||
- **Automatic caching** - Intelligent result caching
|
||||
|
||||
### Performance Optimization
|
||||
|
||||
**Q8 Quantized Models** - 75% smaller, faster loading (v2.8.0+)
|
||||
|
||||
```javascript
|
||||
// Default: Full precision (fp32) - maximum compatibility
|
||||
const brain = new BrainyData()
|
||||
|
||||
// Optimized: Quantized models (q8) - 75% smaller, 99% accuracy
|
||||
const brainOptimized = new BrainyData({
|
||||
embeddingOptions: { dtype: 'q8' }
|
||||
})
|
||||
```
|
||||
|
||||
**Model Comparison:**
|
||||
- **FP32 (default)**: 90MB, 100% accuracy, maximum compatibility
|
||||
- **Q8 (optional)**: 23MB, ~99% accuracy, faster loading
|
||||
|
||||
**When to use Q8:**
|
||||
- ✅ New projects where size/speed matters
|
||||
- ✅ Memory-constrained environments
|
||||
- ✅ Mobile or edge deployments
|
||||
- ❌ Existing projects with FP32 data (incompatible embeddings)
|
||||
|
||||
**Air-gap deployment:**
|
||||
```bash
|
||||
npm run download-models # Both models (recommended)
|
||||
npm run download-models:q8 # Q8 only (space-constrained)
|
||||
npm run download-models:fp32 # FP32 only (compatibility)
|
||||
```
|
||||
|
||||
## 📚 Core API
|
||||
|
||||
### `search()` - Vector Similarity
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue