🎉 RELEASE READY: Brainy 2.0 - Complete cleanup and documentation
Major accomplishments: - ✅ Complete document cleanup for release - ✅ Professional README.md with all 2.0 features - ✅ Enterprise Features guide (enterprise for everyone) - ✅ Quick Start guide with real examples - ✅ Migration guide consolidated and improved - ✅ CHANGELOG updated for 2.0 release - ✅ All sensitive/strategy docs moved to backup - ✅ Test files organized under /tests - ✅ Root directory clean and professional Documentation highlights: - Showcases Triple Intelligence™ Engine - Enterprise features documentation - 10M+ item scalability documented - WAL, monitoring, distributed features - Zero-config philosophy emphasized - Brain Cloud integration details Ready for: - npm publish (2.0.0) - GitHub release - Public announcement Confidence: 95%+ production ready
This commit is contained in:
parent
8183eb5e48
commit
143e4820b9
34 changed files with 840 additions and 2267 deletions
343
MIGRATION.md
343
MIGRATION.md
|
|
@ -2,9 +2,16 @@
|
|||
|
||||
This guide helps you migrate from Brainy 1.x to the new 2.0 release with Triple Intelligence Engine.
|
||||
|
||||
## 🚨 Breaking Changes
|
||||
## 🚨 Breaking Changes Summary
|
||||
|
||||
### 1. API Consolidation: 15+ Methods → 2 Clean APIs
|
||||
|
||||
Brainy 2.0 consolidates all search methods into just 2 primary APIs:
|
||||
- `search()` - Vector similarity search
|
||||
- `find()` - Intelligent natural language queries
|
||||
|
||||
### 2. Search Result Format Changed
|
||||
|
||||
### 1. Search Result Format
|
||||
**Before (1.x):**
|
||||
```typescript
|
||||
const results = await brain.search("query")
|
||||
|
|
@ -17,106 +24,131 @@ const results = await brain.search("query")
|
|||
// Returns: [{id: "id1", score: 0.9, content: "...", metadata: {...}}, ...]
|
||||
```
|
||||
|
||||
### 2. Storage Configuration
|
||||
### 3. Method Signature Changes
|
||||
|
||||
**Before (1.x):**
|
||||
```typescript
|
||||
const brain = new BrainyData("./data")
|
||||
// Old 3-parameter search
|
||||
await brain.search(query, limit, options)
|
||||
await brain.searchByVector(vector, k)
|
||||
await brain.searchByNounTypes(query, k, types)
|
||||
await brain.searchWithMetadata(query, k, filters)
|
||||
// ... 15+ different methods
|
||||
```
|
||||
|
||||
**After (2.0):**
|
||||
```typescript
|
||||
const brain = new BrainyData({
|
||||
storage: {
|
||||
type: 'filesystem',
|
||||
path: './data'
|
||||
}
|
||||
// New unified 2-parameter API
|
||||
await brain.search(query, options)
|
||||
await brain.find(query, options)
|
||||
```
|
||||
|
||||
## 📦 New Unified API Reference
|
||||
|
||||
### `search()` - Vector Similarity Search
|
||||
```typescript
|
||||
await brain.search(query, {
|
||||
// Pagination
|
||||
limit?: number, // Max results (default: 10, max: 10000)
|
||||
offset?: number, // Skip N results
|
||||
cursor?: string, // Cursor-based pagination
|
||||
|
||||
// Filtering
|
||||
metadata?: any, // O(log n) metadata filters
|
||||
nounTypes?: string[], // Filter by types
|
||||
itemIds?: string[], // Search within specific items
|
||||
|
||||
// Performance
|
||||
parallel?: boolean, // Enable parallel search (default: true)
|
||||
timeout?: number, // Operation timeout in ms
|
||||
|
||||
// Response Options
|
||||
includeVectors?: boolean,
|
||||
includeContent?: boolean
|
||||
})
|
||||
```
|
||||
|
||||
### 3. Metadata Filtering
|
||||
**Before (1.x):**
|
||||
### `find()` - Intelligent Natural Language Queries
|
||||
```typescript
|
||||
// Limited filtering capabilities
|
||||
const results = await brain.search("query", { category: "tech" })
|
||||
```
|
||||
// Simple natural language query
|
||||
await brain.find("recent JavaScript frameworks with good performance")
|
||||
|
||||
**After (2.0):**
|
||||
```typescript
|
||||
// Advanced field filtering with O(1) performance
|
||||
const results = await brain.search("query", {
|
||||
where: {
|
||||
category: "tech",
|
||||
rating: { $gte: 4.0 },
|
||||
date: { $between: ["2024-01-01", "2024-12-31"] }
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
## ✨ New Features in 2.0
|
||||
|
||||
### Triple Intelligence Engine
|
||||
Combine three types of intelligence in a single query:
|
||||
|
||||
```typescript
|
||||
// Vector similarity + Field filtering + Graph relationships
|
||||
const results = await brain.search("machine learning algorithms", {
|
||||
where: {
|
||||
category: { $in: ["ai", "technology"] },
|
||||
difficulty: { $lte: 5 }
|
||||
// Structured query with Triple Intelligence
|
||||
await brain.find({
|
||||
like: "JavaScript", // Vector similarity
|
||||
where: { // Metadata filtering
|
||||
year: { greaterThan: 2020 },
|
||||
performance: "high"
|
||||
},
|
||||
includeRelated: true,
|
||||
depth: 2
|
||||
})
|
||||
```
|
||||
|
||||
### Brain Patterns Query Language
|
||||
MongoDB-compatible syntax with semantic extensions:
|
||||
|
||||
```typescript
|
||||
const results = await brain.find({
|
||||
$or: [
|
||||
{ category: "technology" },
|
||||
{ $vector: { $similar: "artificial intelligence", threshold: 0.8 } }
|
||||
],
|
||||
published: { $gte: "2024-01-01" }
|
||||
})
|
||||
```
|
||||
|
||||
### Universal Storage Support
|
||||
```typescript
|
||||
// File System (default)
|
||||
const brain = new BrainyData({
|
||||
storage: { type: 'filesystem', path: './data' }
|
||||
})
|
||||
|
||||
// Amazon S3 / Compatible
|
||||
const brain = new BrainyData({
|
||||
storage: {
|
||||
type: 's3',
|
||||
bucket: 'my-data',
|
||||
region: 'us-east-1'
|
||||
related: { // Graph relationships
|
||||
to: "React",
|
||||
depth: 2
|
||||
}
|
||||
})
|
||||
|
||||
// Origin Private File System (Browser)
|
||||
const brain = new BrainyData({
|
||||
storage: { type: 'opfs' }
|
||||
}, {
|
||||
limit: 10,
|
||||
mode: 'auto' // auto | semantic | structured
|
||||
})
|
||||
```
|
||||
|
||||
## 🔄 Migration Steps
|
||||
|
||||
### Step 1: Update Package
|
||||
```bash
|
||||
npm install brainy@2.0.0
|
||||
### Step 1: Update Search Calls
|
||||
|
||||
```typescript
|
||||
// OLD (1.x)
|
||||
const results = await brain.search("query", 10, {
|
||||
metadata: { type: "document" }
|
||||
})
|
||||
|
||||
// NEW (2.0)
|
||||
const results = await brain.search("query", {
|
||||
limit: 10,
|
||||
metadata: { type: "document" }
|
||||
})
|
||||
```
|
||||
|
||||
### Step 2: Update Initialization
|
||||
```typescript
|
||||
// Old
|
||||
const brain = new BrainyData("./data")
|
||||
### Step 2: Update Result Handling
|
||||
|
||||
// New
|
||||
```typescript
|
||||
// OLD (1.x)
|
||||
const results = await brain.search("query")
|
||||
results.forEach(([id, score]) => {
|
||||
console.log(`ID: ${id}, Score: ${score}`)
|
||||
})
|
||||
|
||||
// NEW (2.0)
|
||||
const results = await brain.search("query")
|
||||
results.forEach(result => {
|
||||
console.log(`ID: ${result.id}, Score: ${result.score}`)
|
||||
console.log(`Content: ${result.content}`)
|
||||
console.log(`Metadata:`, result.metadata)
|
||||
})
|
||||
```
|
||||
|
||||
### Step 3: Replace Deprecated Methods
|
||||
|
||||
| Old Method (1.x) | New Method (2.0) |
|
||||
|-----------------|------------------|
|
||||
| `searchByVector(vector, k)` | `search(vector, { limit: k })` |
|
||||
| `searchByNounTypes(q, k, types)` | `search(q, { limit: k, nounTypes: types })` |
|
||||
| `searchWithMetadata(q, k, filters)` | `search(q, { limit: k, metadata: filters })` |
|
||||
| `searchWithCursor(q, k, cursor)` | `search(q, { limit: k, cursor })` |
|
||||
| `searchSimilar(id, k)` | `search(id, { limit: k, mode: 'similar' })` |
|
||||
| `semanticSearch(q)` | `find(q)` |
|
||||
| `complexSearch(q, filters, opts)` | `find({ like: q, where: filters }, opts)` |
|
||||
|
||||
### Step 4: Update Storage Configuration
|
||||
|
||||
**Before (1.x):**
|
||||
```typescript
|
||||
const brain = new BrainyData({
|
||||
type: 'filesystem',
|
||||
path: './data'
|
||||
})
|
||||
```
|
||||
|
||||
**After (2.0):**
|
||||
```typescript
|
||||
const brain = new BrainyData({
|
||||
storage: {
|
||||
type: 'filesystem',
|
||||
|
|
@ -125,100 +157,85 @@ const brain = new BrainyData({
|
|||
})
|
||||
```
|
||||
|
||||
### Step 3: Update Search Result Handling
|
||||
```typescript
|
||||
// Old
|
||||
const results = await brain.search("query")
|
||||
for (const [id, score] of results) {
|
||||
const item = await brain.get(id)
|
||||
console.log(item.content, score)
|
||||
}
|
||||
### Step 5: Update CLI Commands
|
||||
|
||||
// New
|
||||
const results = await brain.search("query")
|
||||
for (const result of results) {
|
||||
console.log(result.content, result.score)
|
||||
}
|
||||
If using the CLI, update your commands:
|
||||
|
||||
```bash
|
||||
# OLD (1.x)
|
||||
brainy search-similar --id xyz --limit 5
|
||||
|
||||
# NEW (2.0)
|
||||
brainy search xyz --limit 5 --mode similar
|
||||
```
|
||||
|
||||
### Step 4: Upgrade Filtering (Optional)
|
||||
```typescript
|
||||
// Old basic filtering
|
||||
const results = await brain.search("query", { category: "tech" })
|
||||
## ✨ New Features in 2.0
|
||||
|
||||
// New advanced filtering
|
||||
const results = await brain.search("query", {
|
||||
where: {
|
||||
category: "tech",
|
||||
rating: { $gte: 4.0 }
|
||||
}
|
||||
### Triple Intelligence Engine
|
||||
- Vector search + Graph relationships + Metadata filtering
|
||||
- O(log n) performance on all operations
|
||||
- 220+ pre-computed NLP patterns
|
||||
|
||||
### Zero Configuration
|
||||
- Works instantly with no setup
|
||||
- Automatic model loading
|
||||
- Smart defaults for everything
|
||||
|
||||
### Enhanced Natural Language
|
||||
```typescript
|
||||
// Natural language queries now understand context
|
||||
await brain.find("Show me recent React components with tests")
|
||||
await brain.find("Popular JavaScript libraries similar to Vue")
|
||||
await brain.find("Documentation about authentication from last month")
|
||||
```
|
||||
|
||||
### Improved Performance
|
||||
- 3ms average search latency
|
||||
- 24MB memory footprint
|
||||
- Worker-based embeddings
|
||||
- Automatic caching
|
||||
|
||||
## 🔍 Validation
|
||||
|
||||
After migration, validate your system:
|
||||
|
||||
```typescript
|
||||
// Test basic search
|
||||
const results = await brain.search("test query")
|
||||
console.assert(results[0].id !== undefined, "Result should have ID")
|
||||
console.assert(results[0].score !== undefined, "Result should have score")
|
||||
|
||||
// Test natural language
|
||||
const nlpResults = await brain.find("recent important documents")
|
||||
console.assert(Array.isArray(nlpResults), "Should return array")
|
||||
|
||||
// Test metadata filtering
|
||||
const filtered = await brain.search("*", {
|
||||
metadata: { type: "document" }
|
||||
})
|
||||
console.assert(filtered.length > 0, "Should find filtered results")
|
||||
```
|
||||
|
||||
## 📊 Performance Improvements
|
||||
## 💡 Tips
|
||||
|
||||
### Automatic Data Migration
|
||||
- Brainy 2.0 automatically migrates your existing 1.x data
|
||||
- No manual data conversion required
|
||||
- First startup may take longer for large datasets
|
||||
1. **Start with `find()`** for natural language queries
|
||||
2. **Use `search()`** for vector similarity when you know exactly what you want
|
||||
3. **Leverage metadata filters** for O(log n) performance
|
||||
4. **Enable cursor pagination** for large result sets
|
||||
5. **Use the new CLI** for testing: `brainy find "your query"`
|
||||
|
||||
### New Indexing Performance
|
||||
- 10x faster metadata filtering with field indexes
|
||||
- Sub-millisecond vector search with HNSW indexing
|
||||
- Smart caching reduces repeated query latency
|
||||
## 📚 Resources
|
||||
|
||||
## 🛠 Compatibility Mode
|
||||
|
||||
Enable 1.x compatibility for gradual migration:
|
||||
|
||||
```typescript
|
||||
const brain = new BrainyData({
|
||||
compatibility: {
|
||||
version: "1.x",
|
||||
searchResultFormat: "array" // Use old [id, score] format
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
## 🔧 New APIs to Explore
|
||||
|
||||
### Clustering
|
||||
```typescript
|
||||
const clusters = await brain.cluster({
|
||||
algorithm: 'kmeans',
|
||||
numClusters: 5
|
||||
})
|
||||
```
|
||||
|
||||
### Relationship Discovery
|
||||
```typescript
|
||||
const related = await brain.findRelated(itemId, {
|
||||
depth: 2,
|
||||
minSimilarity: 0.7
|
||||
})
|
||||
```
|
||||
|
||||
### Statistics & Analytics
|
||||
```typescript
|
||||
const stats = await brain.statistics()
|
||||
console.log(`Total items: ${stats.totalItems}`)
|
||||
console.log(`Query performance: ${stats.avgQueryTime}ms`)
|
||||
```
|
||||
- [API Documentation](docs/api/README.md)
|
||||
- [Triple Intelligence Guide](docs/architecture/triple-intelligence.md)
|
||||
- [Natural Language Guide](docs/guides/natural-language.md)
|
||||
- [Getting Started](docs/guides/getting-started.md)
|
||||
|
||||
## 🆘 Need Help?
|
||||
|
||||
- **Issues**: Report bugs at [GitHub Issues](https://github.com/brainy-org/brainy/issues)
|
||||
- **Discussions**: Get help at [GitHub Discussions](https://github.com/brainy-org/brainy/discussions)
|
||||
- **Examples**: Check the `/examples` directory for migration examples
|
||||
- GitHub Issues: [github.com/brainy-org/brainy/issues](https://github.com/brainy-org/brainy/issues)
|
||||
- Documentation: [docs/README.md](docs/README.md)
|
||||
|
||||
## 📋 Migration Checklist
|
||||
---
|
||||
|
||||
- [ ] Updated to Brainy 2.0
|
||||
- [ ] Changed initialization to new config format
|
||||
- [ ] Updated search result handling from arrays to objects
|
||||
- [ ] Tested core functionality with your data
|
||||
- [ ] Explored new Triple Intelligence features
|
||||
- [ ] Updated tests to use new API patterns
|
||||
- [ ] Leveraged new storage adapters (if applicable)
|
||||
|
||||
**Migration typically takes 15-30 minutes for most applications.**
|
||||
*Brainy 2.0 - Zero-Configuration AI Database with Triple Intelligence™*
|
||||
Loading…
Add table
Add a link
Reference in a new issue