soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

feat(docs): add comprehensive user guides and installation instructions for Brainy

Browse source
This commit is contained in:
David Snelling 2025-08-03 17:33:52 -07:00
parent 8f6f657ba0
commit 24076c3eba
15 changed files with 1859 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

67
docs/getting-started/README.md Normal file
View file

@ -0,0 +1,67 @@
# Getting Started with Brainy
Welcome to Brainy! This section provides everything you need to get up and running with the world's smartest vector database.
## 🚀 Quick Navigation
### [📦 Installation Guide](installation.md)
Learn how to install Brainy and set up your environment.
- Package installation options
- Environment requirements
- Verification steps
### [⚡ Quick Start Guide](quick-start.md)
Get your first Brainy application running in minutes.
- Zero-configuration setup
- Basic usage examples
- Auto-configuration features
### [🛠️ Environment Setup](environment-setup.md)
Configure your development environment for optimal performance.
- Development vs production settings
- Environment-specific optimizations
- Storage configuration
### [👶 First Steps](first-steps.md)
A guided tutorial through Brainy's core features.
- Your first vector database
- Adding and searching data
- Understanding search results
- Graph relationships
## 🎯 What You'll Learn
By the end of this section, you'll understand:
- ✅ How to install and set up Brainy
- ✅ Brainy's zero-configuration auto-optimization
- ✅ Basic vector operations and search
- ✅ How to choose storage options
- ✅ Performance optimization basics
## 🏃‍♂️ I'm in a Hurry!
If you just want to get started immediately:
1. **Install**: `npm install @soulcraft/brainy`
2. **Use**:
```typescript
import { createAutoBrainy } from '@soulcraft/brainy'
const brainy = createAutoBrainy()
```
3. **Done!** Everything else is auto-configured.
See the [Quick Start Guide](quick-start.md) for complete examples.
## 🔄 Next Steps
After completing the getting started guides:
- 📖 [User Guides](../user-guides/) - Learn advanced features
- ⚡ [Optimization Guides](../optimization-guides/) - Scale to millions
- 🔧 [API Reference](../api-reference/) - Complete API documentation
- 💡 [Examples](../examples/) - Real-world code examples

321
docs/getting-started/installation.md Normal file
View file

@ -0,0 +1,321 @@
# Installation Guide
This guide covers installing Brainy and setting up your development environment.
## 📦 Package Installation
### Core Package
```bash
npm install @soulcraft/brainy
```
The core package includes everything you need:
- ✅ Vector database with HNSW indexing
- ✅ Auto-configuration and optimization
- ✅ All storage adapters (Memory, FileSystem, OPFS, S3)
- ✅ TensorFlow.js integration
- ✅ Cross-environment compatibility
### Optional Packages
#### CLI Tools
```bash
npm install -g @soulcraft/brainy-cli
```
Command-line interface for:
- Database management
- Bulk operations
- Performance testing
- Data visualization
#### Web Service
```bash
npm install @soulcraft/brainy-web-service
```
REST API wrapper for:
- HTTP endpoints
- Remote database access
- Microservice integration
## 🌐 Environment Requirements
### Node.js
- **Minimum**: Node.js 24.4.0+
- **Recommended**: Node.js 20+ or latest LTS
- **Package Manager**: npm, yarn, or pnpm
```bash
node --version # Should be 24.4.0+
npm --version # Any recent version
```
### Browser
Modern browsers with ES Modules support:
- **Chrome**: 86+
- **Edge**: 86+
- **Opera**: 72+
- **Firefox**: 78+
- **Safari**: 14+
#### Optional Browser Features
- **OPFS Support**: For persistent storage (Chrome 86+, Edge 86+)
- **Web Workers**: For parallel processing (all modern browsers)
- **WebGL**: For GPU acceleration (most modern browsers)
### Memory Requirements
| Use Case | Minimum RAM | Recommended RAM |
|----------|-------------|-----------------|
| Development | 512MB | 2GB |
| Small datasets (<10k vectors) | 1GB | 4GB |
| Medium datasets (<100k vectors) | 2GB | 8GB |
| Large datasets (1M+ vectors) | 4GB | 16GB+ |
### Storage Requirements
| Dataset Size | Minimum Storage | Recommended Storage |
|-------------|-----------------|-------------------|
| <10k vectors | 100MB | 500MB |
| <100k vectors | 1GB | 5GB |
| <1M vectors | 10GB | 50GB |
| 1M+ vectors | 50GB+ | Dataset size × 3 |
## ✅ Installation Verification
### Basic Verification
```typescript
import { BrainyData } from '@soulcraft/brainy'
console.log('Brainy installed successfully!')
// Test auto-configuration
import { createAutoBrainy } from '@soulcraft/brainy'
const brainy = createAutoBrainy()
console.log('Auto-configuration works!')
```
### Environment Detection Test
```typescript
import { environment } from '@soulcraft/brainy'
console.log(`Environment: ${
environment.isBrowser ? 'Browser' :
environment.isNode ? 'Node.js' :
'Unknown'
}`)
console.log(`Threading available: ${environment.isThreadingAvailable()}`)
```
### Storage Test
```typescript
import { createAutoBrainy } from '@soulcraft/brainy'
const brainy = createAutoBrainy()
// Add a test vector
await brainy.addVector({
id: 'test-1',
vector: [0.1, 0.2, 0.3],
text: 'Installation test'
})
// Search for it
const results = await brainy.search([0.1, 0.2, 0.3], 1)
console.log('Storage test passed:', results.length > 0)
```
## 🔧 Development Setup
### TypeScript Configuration
Add to your `tsconfig.json`:
```json
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "node",
"lib": ["ES2022", "DOM", "WebWorker"],
"allowSyntheticDefaultImports": true,
"esModuleInterop": true,
"skipLibCheck": true
}
}
```
### Bundler Configuration
#### Vite
```typescript
// vite.config.ts
import { defineConfig } from 'vite'
export default defineConfig({
optimizeDeps: {
include: ['@soulcraft/brainy']
},
define: {
global: 'globalThis'
}
})
```
#### Webpack
```javascript
// webpack.config.js
module.exports = {
resolve: {
fallback: {
"buffer": require.resolve("buffer"),
"util": require.resolve("util")
}
},
plugins: [
new webpack.ProvidePlugin({
Buffer: ['buffer', 'Buffer'],
process: 'process/browser'
})
]
}
```
#### Rollup
```javascript
// rollup.config.js
import { nodeResolve } from '@rollup/plugin-node-resolve'
import { nodePolyfills } from 'rollup-plugin-polyfill-node'
export default {
plugins: [
nodePolyfills(),
nodeResolve({ browser: true, preferBuiltins: false })
]
}
```
## 🚀 Production Setup
### Environment Variables
For S3 storage in production:
```bash
# AWS Configuration
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
AWS_REGION=us-east-1
# Optional S3 Configuration
S3_BUCKET_NAME=your-vector-storage
S3_ENDPOINT=https://s3.amazonaws.com
```
### Docker
```dockerfile
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
EXPOSE 3000
# Set memory limit for large datasets
ENV NODE_OPTIONS="--max-old-space-size=8192"
CMD ["node", "index.js"]
```
### Performance Optimizations
```typescript
// Production configuration
import { createAutoBrainy } from '@soulcraft/brainy'
const brainy = createAutoBrainy({
// S3 storage for persistence
bucketName: process.env.S3_BUCKET_NAME,
region: process.env.AWS_REGION
})
// System auto-configures based on:
// - Available memory
// - CPU cores
// - Dataset size
// - Environment type
```
## 🔍 Troubleshooting
### Common Issues
#### "Module not found" errors
**Solution**: Ensure your bundler is configured for ES modules:
```json
{
"type": "module"
}
```
#### Out of memory errors
**Solution**: Increase Node.js memory limit:
```bash
node --max-old-space-size=8192 your-script.js
```
#### TensorFlow.js loading issues
**Solution**: The auto-patcher handles this, but if needed:
```typescript
import '@soulcraft/brainy/setup' // Import before other modules
import { BrainyData } from '@soulcraft/brainy'
```
#### Browser compatibility issues
**Solution**: Check browser requirements and enable feature detection:
```typescript
import { environment } from '@soulcraft/brainy'
if (!environment.isBrowser) {
console.error('This app requires a modern browser')
}
```
### Getting Help
- 📚 [Troubleshooting Guide](../troubleshooting/)
- 🐛 [GitHub Issues](https://github.com/soulcraft-research/brainy/issues)
- 💬 [GitHub Discussions](https://github.com/soulcraft-research/brainy/discussions)
## ✅ Next Steps
Once installation is complete:
1. **[Quick Start Guide](quick-start.md)** - Your first Brainy app
2. **[Environment Setup](environment-setup.md)** - Optimize your environment
3. **[First Steps](first-steps.md)** - Learn core concepts

241
docs/getting-started/quick-start.md Normal file
View file

@ -0,0 +1,241 @@
# Quick Start Guide
Get your first Brainy application running in just a few minutes with zero configuration required!
## ⚡ The 2-Minute Setup
### 1. Install Brainy
```bash
npm install @soulcraft/brainy
```
### 2. Create Your First Vector Database
```typescript
import { createAutoBrainy } from '@soulcraft/brainy'
// That's it! Everything is auto-configured
const brainy = createAutoBrainy()
// Add some data
await brainy.addVector({
id: '1',
vector: [0.1, 0.2, 0.3],
text: 'Hello world'
})
// Search for similar vectors
const results = await brainy.search([0.1, 0.2, 0.3], 10)
console.log('Found:', results)
```
🎉 **Congratulations!** You now have a production-ready vector database with:
- ✅ Automatic environment detection
- ✅ Optimized memory management
- ✅ Intelligent caching
- ✅ Performance auto-tuning
## 🎯 Choose Your Scenario
### Scenario 1: Development & Testing
```typescript
import { createAutoBrainy } from '@soulcraft/brainy'
// Perfect for development - uses memory storage
const brainy = createAutoBrainy()
// Add test data
await brainy.addVector({ id: '1', vector: [0.1, 0.2, 0.3] })
await brainy.addVector({ id: '2', vector: [0.4, 0.5, 0.6] })
// Search
const results = await brainy.search([0.1, 0.2, 0.3], 5)
```
### Scenario 2: Production with Persistence
```typescript
import { createAutoBrainy } from '@soulcraft/brainy'
// Auto-detects AWS credentials from environment variables
const brainy = createAutoBrainy({
bucketName: 'my-vector-storage'
})
// Data persists in S3 - survives restarts
await brainy.addVector({ id: '1', vector: [0.1, 0.2, 0.3] })
```
### Scenario 3: Scale-Specific Setup
```typescript
import { createQuickBrainy } from '@soulcraft/brainy'
// Choose your scale: 'small', 'medium', 'large', 'enterprise'
const brainy = await createQuickBrainy('large', {
bucketName: 'my-big-vector-db'
})
// System auto-configures for 1M+ vectors
```
### Scenario 4: Text-Based Semantic Search
```typescript
import { createAutoBrainy } from '@soulcraft/brainy'
const brainy = createAutoBrainy()
// Add text - automatically converted to vectors
await brainy.addText('1', 'Machine learning is fascinating')
await brainy.addText('2', 'Deep learning models are powerful')
await brainy.addText('3', 'Cats make great pets')
// Search by meaning, not keywords
const results = await brainy.searchText('AI and neural networks', 2)
// Returns: machine learning and deep learning results
```
## 🧠 What Auto-Configuration Does
When you use `createAutoBrainy()`, the system automatically:
### 🎯 **Environment Detection**
- Detects Browser, Node.js, or Serverless environment
- Configures threading (Web Workers vs Worker Threads)
- Sets appropriate memory limits
### 💾 **Smart Storage Selection**
- **Browser**: OPFS (persistent) → Memory (fallback)
- **Node.js**: FileSystem → S3 (if configured)
- **Serverless**: S3 (if configured) → Memory
### ⚡ **Performance Optimization**
- **Memory Management**: Uses available RAM optimally
- **Semantic Partitioning**: Clusters similar vectors automatically
- **Distributed Search**: Parallel processing on multi-core systems
- **Multi-Level Caching**: Hot/Warm/Cold caching strategy
### 📊 **Adaptive Learning**
- Monitors search performance in real-time
- Adjusts parameters every 50 searches
- Learns from your data patterns
- Continuously improves performance
## 📋 Complete Examples
### Example 1: Document Search System
```typescript
import { createAutoBrainy } from '@soulcraft/brainy'
const brainy = createAutoBrainy()
// Add documents
const docs = [
{ id: 'doc1', text: 'Climate change affects global weather patterns' },
{ id: 'doc2', text: 'Machine learning models can predict weather' },
{ id: 'doc3', text: 'Solar panels reduce carbon emissions' }
]
for (const doc of docs) {
await brainy.addText(doc.id, doc.text)
}
// Semantic search
const results = await brainy.searchText('environmental sustainability', 3)
console.log('Relevant documents:', results)
```
### Example 2: Recommendation System
```typescript
import { createAutoBrainy } from '@soulcraft/brainy'
const brainy = createAutoBrainy()
// Add user preferences as vectors
await brainy.addVector({
id: 'user1',
vector: [0.8, 0.1, 0.9, 0.2], // [action, comedy, drama, horror]
metadata: { name: 'Alice', age: 25 }
})
await brainy.addVector({
id: 'user2',
vector: [0.1, 0.9, 0.2, 0.8],
metadata: { name: 'Bob', age: 30 }
})
// Find similar users
const similar = await brainy.search([0.7, 0.2, 0.8, 0.1], 2)
console.log('Similar users:', similar)
```
### Example 3: Production API
```typescript
import { createAutoBrainy } from '@soulcraft/brainy'
import express from 'express'
const app = express()
const brainy = createAutoBrainy({
bucketName: process.env.S3_BUCKET_NAME
})
app.post('/add', async (req, res) => {
const { id, text } = req.body
await brainy.addText(id, text)
res.json({ success: true })
})
app.get('/search', async (req, res) => {
const { query, limit = 10 } = req.query
const results = await brainy.searchText(query, limit)
res.json({ results })
})
app.listen(3000, () => {
console.log('Vector search API running on port 3000')
})
```
## 🚀 Performance Benchmarks
With auto-configuration, you can expect:
| Dataset Size | Search Time | Memory Usage | Setup Time |
|-------------|-------------|--------------|------------|
| 1k vectors | <10ms | <100MB | <1 second |
| 10k vectors | ~50ms | ~300MB | <5 seconds |
| 100k vectors | ~200ms | ~1GB | ~30 seconds |
| 1M vectors | ~500ms | ~4GB | ~5 minutes |
*Benchmarks on modern hardware. Actual performance varies by environment.*
## 🔄 Next Steps
Now that you have Brainy running:
### Learn More Features
- **[First Steps Guide](first-steps.md)** - Core concepts and features
- **[User Guides](../user-guides/)** - Advanced search techniques
- **[Optimization Guides](../optimization-guides/)** - Scale to millions
### Production Deployment
- **[Environment Setup](environment-setup.md)** - Configure for production
- **[API Reference](../api-reference/)** - Complete API documentation
- **[Examples](../examples/)** - Real-world integration patterns
### Get Help
- **[Troubleshooting](../troubleshooting/)** - Common issues and solutions
- **[GitHub Issues](https://github.com/soulcraft-research/brainy/issues)** - Bug reports
- **[GitHub Discussions](https://github.com/soulcraft-research/brainy/discussions)** - Community support
## 💡 Pro Tips
1. **Start Simple**: Use `createAutoBrainy()` first, optimize later
2. **Monitor Performance**: Check metrics with `brainy.getPerformanceMetrics()`
3. **Use S3 for Production**: Persistent storage survives restarts
4. **Let it Learn**: Performance improves automatically over time
5. **Scale Gradually**: Start with 'small' scenario, upgrade as needed
**Ready to build something amazing?** 🚀