docs: consolidate and archive redundant documentation
- Archived 13 API design iterations to docs/api-design-archive/ - Consolidated augmentation docs to docs/augmentations-archive/ - Maintained ONE definitive API doc at docs/api/README.md - Cleaned up documentation structure for 2.0 release - Preserved all historical documents for reference
This commit is contained in:
parent
ef8f35ec2a
commit
994276f09f
35 changed files with 444 additions and 239 deletions
|
|
@ -60,11 +60,11 @@ await brain.searchWithinNouns(ids[], query) // Search within specific nouns
|
|||
|
||||
### Triple Intelligence 🧠
|
||||
```typescript
|
||||
await brain.find(query) // Unified Vector + Graph + Field search
|
||||
await brain.find(query) // Unified Vector + Graph + Metadata search
|
||||
// Examples:
|
||||
await brain.find('documents about AI') // Natural language
|
||||
await brain.find({ like: 'sample-id' }) // Similar to ID
|
||||
await brain.find({ where: { type: 'doc' }}) // Field filter
|
||||
await brain.find({ where: { type: 'doc' }}) // Metadata filter
|
||||
await brain.find({ connected: { to: id }}) // Graph traversal
|
||||
```
|
||||
|
||||
|
|
@ -47,14 +47,14 @@ deleteVerbs(ids[]) // Delete multiple verbs
|
|||
// Core Search
|
||||
search(query, k?, options?) // Primary vector search
|
||||
searchText(text, k?, options?) // Natural language search
|
||||
find(query) // Triple Intelligence (Vector+Graph+Field) 🧠
|
||||
find(query) // Triple Intelligence (Vector+Graph+Metadata) 🧠
|
||||
findSimilar(id, k?, options?) // Find similar to existing noun
|
||||
|
||||
// Advanced Search
|
||||
searchByNounTypes(types[], query, k?) // Filter by noun types
|
||||
searchWithinItems(query, ids[], k?) // Search within specific nouns
|
||||
searchWithCursor(query, cursor) // Paginated search
|
||||
searchByStandardField(field, value, k?) // Field-based search
|
||||
searchByStandardField(field, value, k?) // Metadata-based search
|
||||
|
||||
// Graph Search
|
||||
searchVerbs(query, options?) // Search relationships
|
||||
|
|
@ -206,7 +206,7 @@ These are now private (use new methods above):
|
|||
- `addItem()`, `addToBoth()`, `addBatch()`, `getBatch()`
|
||||
|
||||
### ✅ Triple Intelligence
|
||||
- New `find()` method unifies Vector + Graph + Field search
|
||||
- New `find()` method unifies Vector + Graph + Metadata search
|
||||
- Most powerful search capability in one simple method
|
||||
|
||||
### ✅ Zero-Configuration
|
||||
|
|
@ -48,7 +48,7 @@ find({
|
|||
// Vector search
|
||||
like: 'text' | vector | {id: 'noun-id'},
|
||||
|
||||
// Field filtering with BRAINY OPERATORS (NOT MongoDB!)
|
||||
// Metadata filtering with BRAINY OPERATORS (NOT MongoDB!)
|
||||
where: {
|
||||
// Direct equality
|
||||
field: value,
|
||||
|
|
@ -261,7 +261,7 @@ brain.generateRandomGraph(nodes, edges) // Test data
|
|||
3. **Simple neuralImport** - One method, smart detection
|
||||
4. **Visualization exports** - For D3, Cytoscape, GraphML
|
||||
5. **search() is just convenience** - Not a complete alias
|
||||
6. **find() has Triple Intelligence** - Vector + Graph + Field
|
||||
6. **find() has Triple Intelligence** - Vector + Graph + Metadata
|
||||
7. **Proper operator names** - greaterThan not $gt
|
||||
8. **Complete clustering API** - Fast, large, streaming options
|
||||
9. **Synapses and Conduits** - External and internal sync
|
||||
|
|
@ -126,7 +126,7 @@ Simple vector similarity search (convenience wrapper)
|
|||
// Vector similarity
|
||||
like?: string | number[] | {id: string}, // Text, vector, or noun ID
|
||||
|
||||
// Field filtering
|
||||
// Metadata filtering
|
||||
where?: {
|
||||
field: value, // Exact match
|
||||
field: {$in: [values]}, // In array
|
||||
|
|
@ -242,7 +242,7 @@ mode // Current mode (readonly)
|
|||
|
||||
✅ **Neural Import** - Smart AI-powered data import
|
||||
✅ **Clustering** - Automatic and manual clustering
|
||||
✅ **Triple Intelligence** - Vector + Graph + Field combined
|
||||
✅ **Triple Intelligence** - Vector + Graph + Metadata combined
|
||||
✅ **Verb Scoring** - Intelligent relationship scoring
|
||||
✅ **Synapses** - External platform connectors
|
||||
✅ **Conduits** - Brainy-to-Brainy sync
|
||||
|
|
@ -48,7 +48,7 @@ Just TWO methods - simple and powerful:
|
|||
|
||||
```typescript
|
||||
search(query, k?) // Convenience: same as find({like: query, limit: k})
|
||||
find(query) // TRIPLE INTELLIGENCE: Vector + Graph + Field
|
||||
find(query) // TRIPLE INTELLIGENCE: Vector + Graph + Metadata
|
||||
```
|
||||
|
||||
### Find Query (with CORRECT Brainy Operators):
|
||||
|
|
@ -229,7 +229,7 @@ BrainyData.warmup() // Warmup
|
|||
|
||||
1. **Zero-Config** - Works instantly, no setup
|
||||
2. **Auto-Embedding** - Text automatically becomes vectors
|
||||
3. **Triple Intelligence** - Vector + Graph + Field combined
|
||||
3. **Triple Intelligence** - Vector + Graph + Metadata combined
|
||||
4. **Brainy Operators** - Clean, legal, no MongoDB style
|
||||
5. **Complete Neural API** - All clustering/viz features
|
||||
6. **Simple Import** - One method, auto-detects everything
|
||||
|
|
@ -53,7 +53,7 @@ find('documents about AI')
|
|||
// Similar to existing noun
|
||||
find({ like: 'noun-id-123' })
|
||||
|
||||
// Field filtering
|
||||
// Metadata filtering
|
||||
find({ where: { type: 'article' }})
|
||||
|
||||
// Graph traversal
|
||||
|
|
@ -62,7 +62,7 @@ find({ connected: { to: 'id', via: 'references' }})
|
|||
// Combined queries (Triple Intelligence!)
|
||||
find({
|
||||
like: 'sample-doc', // Vector similarity
|
||||
where: { status: 'published' }, // Field filter
|
||||
where: { status: 'published' }, // Metadata filter
|
||||
connected: { via: 'cites' }, // Graph relationships
|
||||
limit: 10 // Pagination
|
||||
})
|
||||
|
|
@ -127,7 +127,7 @@ shutdown() // Cleanup
|
|||
### Why So Simple?
|
||||
|
||||
1. **`addNoun()` handles everything** - Text? Auto-embeds. Vector? Uses directly.
|
||||
2. **`find()` is the ultimate search** - Combines vector, graph, and field search
|
||||
2. **`find()` is the ultimate search** - Combines vector, graph, and metadata search
|
||||
3. **`search()` is just convenience** - Simple alias to `find()` for basic queries
|
||||
4. **No duplicate methods** - One way to do each thing
|
||||
|
||||
|
|
@ -136,7 +136,7 @@ shutdown() // Cleanup
|
|||
The `find()` method is your Swiss Army knife:
|
||||
- Text search → Auto-embeds and searches
|
||||
- Vector search → `{ like: 'id' }` or `{ like: vector }`
|
||||
- Field search → `{ where: { field: value }}`
|
||||
- Metadata search → `{ where: { field: value }}`
|
||||
- Graph search → `{ connected: { to/from: 'id' }}`
|
||||
- Combine them all → Triple Intelligence!
|
||||
|
||||
|
|
@ -54,7 +54,7 @@ search(query, k?) // Simple vector search
|
|||
// Equivalent to: find({like: query, limit: k})
|
||||
|
||||
find(query) // TRIPLE INTELLIGENCE 🧠
|
||||
// Combines Vector + Graph + Field search
|
||||
// Combines Vector + Graph + Metadata search
|
||||
```
|
||||
|
||||
### Find Query Structure
|
||||
|
|
@ -329,7 +329,7 @@ brain.initialized // Is initialized?
|
|||
1. **Brainy Operators** - NOT MongoDB style ($gt, $lt)
|
||||
2. **Neural API** - Complete with visualization export
|
||||
3. **Simple Search** - Just `search()` and `find()`
|
||||
4. **Triple Intelligence** - Vector + Graph + Field in `find()`
|
||||
4. **Triple Intelligence** - Vector + Graph + Metadata in `find()`
|
||||
5. **Auto-embedding** - `addNoun()` accepts text directly
|
||||
6. **Unified Methods** - `getNouns()` handles all plural queries
|
||||
7. **Clean Architecture** - Augmentation system for extensibility
|
||||
|
|
@ -60,7 +60,7 @@ searchByNounTypes(types[], query) // Filter by noun types
|
|||
searchWithinItems(query, ids[], k?) // Search within specific nouns
|
||||
searchVerbs(query) // Search relationships
|
||||
searchNounsByVerbs(conditions) // Graph-based noun search
|
||||
searchByStandardField(field, value) // Field-based search
|
||||
searchByStandardField(field, value) // Metadata-based search
|
||||
```
|
||||
|
||||
### Distributed Search
|
||||
|
|
@ -47,14 +47,14 @@ deleteVerbs(ids[]) // Delete multiple verbs
|
|||
// Core Search
|
||||
search(query, k?, options?) // Primary vector search
|
||||
searchText(text, k?, options?) // Natural language search
|
||||
find(query) // Triple Intelligence (Vector+Graph+Field) 🧠
|
||||
find(query) // Triple Intelligence (Vector+Graph+Metadata) 🧠
|
||||
findSimilar(id, k?, options?) // Find similar to existing noun
|
||||
|
||||
// Advanced Search
|
||||
searchByNounTypes(types[], query, k?) // Filter by noun types
|
||||
searchWithinItems(query, ids[], k?) // Search within specific nouns
|
||||
searchWithCursor(query, cursor) // Paginated search
|
||||
searchByStandardField(field, value, k?) // Field-based search
|
||||
searchByStandardField(field, value, k?) // Metadata-based search
|
||||
|
||||
// Graph Search
|
||||
searchVerbs(query, options?) // Search relationships
|
||||
|
|
@ -206,5 +206,5 @@ These are now private (use new methods above):
|
|||
- `addItem()`, `addToBoth()`, `addBatch()`, `getBatch()`
|
||||
|
||||
### ✅ Triple Intelligence
|
||||
- New `find()` method unifies Vector + Graph + Field search
|
||||
- New `find()` method unifies Vector + Graph + Metadata search
|
||||
- Most powerful search capability in one simple method
|
||||
28
docs/api-design-archive/README.md
Normal file
28
docs/api-design-archive/README.md
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
# API Design Archive
|
||||
|
||||
This directory contains historical API design documents from the Brainy 2.0 development process.
|
||||
|
||||
## Purpose
|
||||
These documents represent the iterative refinement of the Brainy 2.0 API during development. They are preserved here for historical reference and to document the design decisions made along the way.
|
||||
|
||||
## Current API Documentation
|
||||
The definitive API documentation is now located at:
|
||||
- **`/docs/api/README.md`** - The ONE official API reference
|
||||
|
||||
## Archived Documents
|
||||
These documents show the evolution of the API design:
|
||||
- Various iterations of API structure
|
||||
- Exploration of different naming conventions
|
||||
- Refinement of the Triple Intelligence concept
|
||||
- Transition from MongoDB operators to Brainy operators
|
||||
- Evolution of the Neural API
|
||||
|
||||
## Key Decisions Made
|
||||
1. **Terminology**: Vector + Graph + Metadata (not Field)
|
||||
2. **Operators**: Brainy operators (greaterThan, lessThan) not MongoDB ($gt, $lt)
|
||||
3. **Methods**: Specific noun/verb naming (addNoun, getNoun)
|
||||
4. **Unification**: Single getNouns() method instead of multiple variants
|
||||
5. **Neural API**: Complete clustering and visualization features
|
||||
|
||||
## Note
|
||||
These documents are archived, not deleted, to preserve the development history and rationale behind API decisions.
|
||||
|
|
@ -1,236 +1,395 @@
|
|||
# API Reference
|
||||
# 🧠 Brainy 2.0 API Reference
|
||||
|
||||
Complete API documentation for Brainy's multi-dimensional AI database.
|
||||
> **The definitive API documentation for Brainy 2.0**
|
||||
> Clean • Powerful • Zero-Configuration
|
||||
|
||||
## Core APIs
|
||||
## Quick Start
|
||||
|
||||
### [BrainyData](./brainy-data.md)
|
||||
The main entry point for all operations.
|
||||
|
||||
### [Triple Intelligence](./triple-intelligence.md)
|
||||
Unified query system for vector, graph, and field search.
|
||||
|
||||
### [Storage](./storage.md)
|
||||
Storage adapter interfaces and implementations.
|
||||
|
||||
### [Entity Registry](./entity-registry.md)
|
||||
High-performance entity deduplication system.
|
||||
|
||||
### [Neural API](./neural-api.md)
|
||||
Natural language processing and similarity operations.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Initialization
|
||||
```typescript
|
||||
import { BrainyData } from 'brainy'
|
||||
|
||||
const brain = new BrainyData({
|
||||
storage: { type: 'filesystem', path: './data' },
|
||||
vectors: { dimensions: 384 }
|
||||
})
|
||||
import { BrainyData } from '@soulcraft/brainy'
|
||||
|
||||
const brain = new BrainyData() // Zero config!
|
||||
await brain.init()
|
||||
```
|
||||
|
||||
### Basic Operations
|
||||
// Add data (text auto-embeds!)
|
||||
await brain.addNoun('The future of AI is here')
|
||||
|
||||
#### Add Entities (Nouns) and Relationships (Verbs)
|
||||
```typescript
|
||||
// Add entities (nouns) with automatic embedding
|
||||
const id = await brain.addNoun("Machine learning is fascinating", {
|
||||
category: "technology",
|
||||
timestamp: Date.now()
|
||||
})
|
||||
|
||||
// Add relationships (verbs) between entities
|
||||
const sourceId = await brain.addNoun("Research Paper")
|
||||
const targetId = await brain.addNoun("Neural Networks")
|
||||
await brain.addVerb(sourceId, targetId, "discusses", {
|
||||
confidence: 0.95,
|
||||
section: "methodology"
|
||||
})
|
||||
|
||||
// Batch operations
|
||||
const entities = ["Entity 1", "Entity 2", "Entity 3"]
|
||||
for (const entity of entities) {
|
||||
await brain.addNoun(entity, { type: "batch" })
|
||||
}
|
||||
```
|
||||
|
||||
#### Search and Find
|
||||
```typescript
|
||||
// Simple semantic search
|
||||
const results = await brain.search("AI and machine learning")
|
||||
|
||||
// Natural language queries with find()
|
||||
const nlpResults = await brain.find("research papers about neural networks from 2024")
|
||||
// Automatically interprets: document type, topic, and time range
|
||||
|
||||
// Advanced triple intelligence search with structured query
|
||||
const structured = await brain.find({
|
||||
like: "neural networks",
|
||||
where: { category: "research" },
|
||||
connected: { to: "team-id", depth: 2 },
|
||||
limit: 20
|
||||
})
|
||||
|
||||
// Complex natural language with multiple conditions
|
||||
const complex = await brain.find("highly cited papers on deep learning with over 100 citations published in Nature")
|
||||
// Automatically extracts: citation count, topic, publication venue
|
||||
```
|
||||
|
||||
#### Get and Update
|
||||
```typescript
|
||||
// Get noun by ID
|
||||
const noun = await brain.getNoun("noun-id")
|
||||
|
||||
// Get verb (relationship) by ID
|
||||
const verb = await brain.getVerb("verb-id")
|
||||
|
||||
// Update noun metadata
|
||||
await brain.updateNounMetadata("noun-id", {
|
||||
verified: true,
|
||||
lastModified: Date.now()
|
||||
})
|
||||
|
||||
// Delete noun (soft delete by default)
|
||||
await brain.deleteNoun("noun-id")
|
||||
|
||||
// Delete verb (relationship)
|
||||
await brain.deleteVerb("verb-id")
|
||||
```
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### Augmentations
|
||||
```typescript
|
||||
import {
|
||||
WALAugmentation,
|
||||
EntityRegistryAugmentation,
|
||||
BatchProcessingAugmentation
|
||||
} from 'brainy'
|
||||
|
||||
const brain = new BrainyData({
|
||||
augmentations: [
|
||||
new WALAugmentation(),
|
||||
new EntityRegistryAugmentation({ maxCacheSize: 100000 }),
|
||||
new BatchProcessingAugmentation({ batchSize: 100 })
|
||||
]
|
||||
// Search with Triple Intelligence
|
||||
const results = await brain.find({
|
||||
like: 'artificial intelligence',
|
||||
where: { year: { greaterThan: 2020 } },
|
||||
connected: { via: 'references' }
|
||||
})
|
||||
```
|
||||
|
||||
### Event System
|
||||
## Core Concepts
|
||||
|
||||
### 🧬 Nouns
|
||||
Vectors with metadata - the fundamental data unit in Brainy.
|
||||
|
||||
### 🔗 Verbs
|
||||
Relationships between nouns - the connections that create knowledge graphs.
|
||||
|
||||
### 🧠 Triple Intelligence
|
||||
Vector search + Graph traversal + Metadata filtering in one unified query.
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
|
||||
### Data Operations
|
||||
|
||||
#### Nouns (Vectors with Metadata)
|
||||
|
||||
##### `addNoun(dataOrVector, metadata?)`
|
||||
Add a single noun to the database.
|
||||
- **dataOrVector**: `string | number[]` - Text (auto-embeds) or pre-computed vector
|
||||
- **metadata**: `object` - Associated metadata
|
||||
- **Returns**: `Promise<string>` - The noun's ID
|
||||
|
||||
##### `getNoun(id)`
|
||||
Retrieve a noun by ID.
|
||||
- **id**: `string` - The noun's ID
|
||||
- **Returns**: `Promise<VectorDocument | null>`
|
||||
|
||||
##### `updateNoun(id, dataOrVector?, metadata?)`
|
||||
Update an existing noun.
|
||||
- **id**: `string` - The noun's ID
|
||||
- **dataOrVector**: `string | number[]` - New data/vector (optional)
|
||||
- **metadata**: `object` - New metadata (optional)
|
||||
- **Returns**: `Promise<void>`
|
||||
|
||||
##### `deleteNoun(id)`
|
||||
Delete a noun.
|
||||
- **id**: `string` - The noun's ID
|
||||
- **Returns**: `Promise<boolean>`
|
||||
|
||||
##### `getNouns(options)`
|
||||
Get multiple nouns (unified method).
|
||||
- **options**: Can be:
|
||||
- `string[]` - Array of IDs
|
||||
- `{filter: object}` - Metadata filter
|
||||
- `{limit: number, offset: number}` - Pagination
|
||||
- **Returns**: `Promise<VectorDocument[]>`
|
||||
|
||||
#### Verbs (Relationships)
|
||||
|
||||
##### `addVerb(source, target, type, metadata?)`
|
||||
Create a relationship between nouns.
|
||||
- **source**: `string` - Source noun ID
|
||||
- **target**: `string` - Target noun ID
|
||||
- **type**: `string` - Relationship type
|
||||
- **metadata**: `object` - Relationship metadata (optional)
|
||||
- **Returns**: `Promise<string>` - The verb's ID
|
||||
|
||||
##### `getVerbsBySource(sourceId)`
|
||||
Get all outgoing relationships.
|
||||
- **sourceId**: `string` - Source noun ID
|
||||
- **Returns**: `Promise<Verb[]>`
|
||||
|
||||
##### `getVerbsByTarget(targetId)`
|
||||
Get all incoming relationships.
|
||||
- **targetId**: `string` - Target noun ID
|
||||
- **Returns**: `Promise<Verb[]>`
|
||||
|
||||
---
|
||||
|
||||
### Search Operations
|
||||
|
||||
#### `search(query, k?)`
|
||||
Simple vector similarity search.
|
||||
- **query**: `string | number[]` - Text or vector
|
||||
- **k**: `number` - Number of results (default: 10)
|
||||
- **Returns**: `Promise<SearchResult[]>`
|
||||
|
||||
> 💡 This is equivalent to: `find({like: query, limit: k})`
|
||||
|
||||
#### `find(query)` - Triple Intelligence 🧠
|
||||
The ultimate search method combining vector, graph, and metadata search.
|
||||
|
||||
```typescript
|
||||
brain.on('addNoun', (noun) => {
|
||||
console.log('Noun added:', noun.id)
|
||||
})
|
||||
|
||||
brain.on('addVerb', (verb) => {
|
||||
console.log('Relationship created:', verb.type)
|
||||
})
|
||||
|
||||
brain.on('search', (query, results) => {
|
||||
console.log(`Search for "${query}" returned ${results.length} results`)
|
||||
})
|
||||
|
||||
brain.on('error', (error) => {
|
||||
console.error('Error occurred:', error)
|
||||
find({
|
||||
// Vector similarity
|
||||
like: 'text query' | vector | {id: 'noun-id'},
|
||||
|
||||
// Metadata filtering (Brainy operators)
|
||||
where: {
|
||||
field: value, // Exact match
|
||||
field: {
|
||||
equals: value,
|
||||
greaterThan: value,
|
||||
lessThan: value,
|
||||
greaterEqual: value,
|
||||
lessEqual: value,
|
||||
oneOf: [val1, val2], // In array
|
||||
notOneOf: [val1, val2], // Not in array
|
||||
contains: value, // Array/string contains
|
||||
startsWith: value,
|
||||
endsWith: value,
|
||||
matches: /pattern/, // Pattern match
|
||||
between: [min, max]
|
||||
}
|
||||
},
|
||||
|
||||
// Graph traversal
|
||||
connected: {
|
||||
to: 'noun-id', // Target noun
|
||||
from: 'noun-id', // Source noun
|
||||
via: 'relationship-type', // Relationship type
|
||||
depth: 2 // Traversal depth
|
||||
},
|
||||
|
||||
// Control
|
||||
limit: 10, // Max results
|
||||
offset: 0, // Skip results
|
||||
explain: false // Include explanation
|
||||
})
|
||||
```
|
||||
|
||||
### Statistics
|
||||
---
|
||||
|
||||
### Neural API
|
||||
|
||||
Access advanced AI features via `brain.neural`:
|
||||
|
||||
#### `brain.neural.similar(a, b)`
|
||||
Calculate semantic similarity between two items.
|
||||
- **Returns**: `Promise<number>` - Similarity score (0-1)
|
||||
|
||||
#### `brain.neural.clusters(options?)`
|
||||
Automatically cluster nouns.
|
||||
- **Returns**: `Promise<Cluster[]>` - Generated clusters
|
||||
|
||||
#### `brain.neural.hierarchy(id)`
|
||||
Build semantic hierarchy from a noun.
|
||||
- **Returns**: `Promise<HierarchyTree>` - Hierarchy structure
|
||||
|
||||
#### `brain.neural.neighbors(id, k?)`
|
||||
Find k-nearest neighbors.
|
||||
- **Returns**: `Promise<Noun[]>` - Nearest neighbors
|
||||
|
||||
#### `brain.neural.outliers(threshold?)`
|
||||
Detect outlier nouns.
|
||||
- **Returns**: `Promise<string[]>` - Outlier IDs
|
||||
|
||||
#### `brain.neural.visualize(options?)`
|
||||
Generate visualization data for external tools.
|
||||
```typescript
|
||||
const stats = await brain.statistics()
|
||||
console.log(`
|
||||
Total items: ${stats.totalItems}
|
||||
Index size: ${stats.indexSize}
|
||||
Average query time: ${stats.avgQueryTime}ms
|
||||
`)
|
||||
visualize({
|
||||
maxNodes: 100,
|
||||
dimensions: 2 | 3,
|
||||
algorithm: 'force' | 'hierarchical' | 'radial',
|
||||
includeEdges: true
|
||||
})
|
||||
// Returns format for D3, Cytoscape, or GraphML
|
||||
```
|
||||
|
||||
## Type Definitions
|
||||
---
|
||||
|
||||
### Core Types
|
||||
```typescript
|
||||
interface SearchResult {
|
||||
id: string
|
||||
score: number
|
||||
content?: string
|
||||
metadata?: Record<string, any>
|
||||
}
|
||||
### Import & Export
|
||||
|
||||
interface TripleQuery {
|
||||
like?: string | Vector | any
|
||||
where?: Record<string, any>
|
||||
connected?: ConnectionQuery
|
||||
limit?: number
|
||||
threshold?: number
|
||||
}
|
||||
#### `neuralImport(data, options?)`
|
||||
AI-powered smart import that auto-detects format.
|
||||
- **data**: `any` - Data to import
|
||||
- **options**: Import configuration
|
||||
- `confidenceThreshold`: Minimum confidence (0-1)
|
||||
- `autoApply`: Automatically add to database
|
||||
- `skipDuplicates`: Skip existing entities
|
||||
- **Returns**: Detected entities and relationships
|
||||
|
||||
interface Vector {
|
||||
values: number[]
|
||||
dimensions: number
|
||||
}
|
||||
```
|
||||
#### `backup()`
|
||||
Create a full backup.
|
||||
- **Returns**: `Promise<BackupData>`
|
||||
|
||||
## Error Handling
|
||||
#### `restore(backup)`
|
||||
Restore from backup.
|
||||
- **backup**: `BackupData` - Previous backup
|
||||
- **Returns**: `Promise<void>`
|
||||
|
||||
All methods follow consistent error handling:
|
||||
---
|
||||
|
||||
```typescript
|
||||
try {
|
||||
await brain.addNoun("content", metadata)
|
||||
} catch (error) {
|
||||
if (error.code === 'STORAGE_ERROR') {
|
||||
// Handle storage issues
|
||||
} else if (error.code === 'VALIDATION_ERROR') {
|
||||
// Handle validation issues
|
||||
}
|
||||
}
|
||||
```
|
||||
### Intelligence Features
|
||||
|
||||
## Performance Guidelines
|
||||
#### Verb Scoring
|
||||
Train the relationship scoring model:
|
||||
- `provideFeedbackForVerbScoring(feedback)` - Train model
|
||||
- `getVerbScoringStats()` - Get statistics
|
||||
- `exportVerbScoringLearningData()` - Export training
|
||||
- `importVerbScoringLearningData(data)` - Import training
|
||||
|
||||
### Batching
|
||||
Always use batch operations for bulk data:
|
||||
```typescript
|
||||
// Good - efficient batch processing
|
||||
const items = ["item1", "item2", "item3"]
|
||||
for (const item of items) {
|
||||
await brain.addNoun(item, { batch: true })
|
||||
}
|
||||
#### Embeddings
|
||||
- `embed(text)` - Generate embedding vector
|
||||
- `calculateSimilarity(a, b, metric?)` - Calculate similarity
|
||||
|
||||
// For relationships
|
||||
const relationships = [
|
||||
{ source: id1, target: id2, type: "related" },
|
||||
{ source: id2, target: id3, type: "similar" }
|
||||
]
|
||||
for (const rel of relationships) {
|
||||
await brain.addVerb(rel.source, rel.target, rel.type)
|
||||
}
|
||||
```
|
||||
---
|
||||
|
||||
### Caching
|
||||
Configure caching for your use case:
|
||||
### Configuration & Management
|
||||
|
||||
#### Operational Modes
|
||||
- `setReadOnly(bool)` - Toggle read-only mode
|
||||
- `setWriteOnly(bool)` - Toggle write-only mode
|
||||
- `setFrozen(bool)` - Freeze all modifications
|
||||
|
||||
#### Cache & Performance
|
||||
- `getCacheStats()` - Get cache statistics
|
||||
- `clearCache()` - Clear search cache
|
||||
- `size()` - Get total noun count
|
||||
- `getStatistics()` - Get full statistics
|
||||
|
||||
#### Data Management
|
||||
- `clear(options?)` - Clear all data
|
||||
- `clearNouns()` - Clear nouns only
|
||||
- `clearVerbs()` - Clear verbs only
|
||||
- `rebuildMetadataIndex()` - Rebuild index
|
||||
|
||||
---
|
||||
|
||||
### Lifecycle
|
||||
|
||||
#### Initialization
|
||||
```typescript
|
||||
const brain = new BrainyData({
|
||||
cache: {
|
||||
search: { maxSize: 100, ttl: 60000 },
|
||||
metadata: { maxSize: 1000, ttl: 300000 }
|
||||
}
|
||||
storage: 'auto', // auto | memory | filesystem | s3
|
||||
dimensions: 384, // Vector dimensions
|
||||
cache: true, // Enable caching
|
||||
index: true // Enable indexing
|
||||
})
|
||||
|
||||
await brain.init() // Required before use!
|
||||
```
|
||||
|
||||
### Indexing
|
||||
Ensure fields used in queries are indexed:
|
||||
#### Cleanup
|
||||
```typescript
|
||||
// Configure indexed fields
|
||||
const brain = new BrainyData({
|
||||
indexedFields: ['category', 'author', 'timestamp']
|
||||
await brain.shutdown() // Graceful shutdown
|
||||
```
|
||||
|
||||
#### Static Methods
|
||||
- `BrainyData.preloadModel()` - Preload ML model
|
||||
- `BrainyData.warmup()` - Warmup system
|
||||
|
||||
---
|
||||
|
||||
## Query Operators Reference
|
||||
|
||||
Brainy uses its own clean, readable operators:
|
||||
|
||||
| Brainy Operator | Description | Example |
|
||||
|-----------------|-------------|---------|
|
||||
| `equals` | Exact match | `{age: {equals: 25}}` |
|
||||
| `greaterThan` | Greater than | `{age: {greaterThan: 18}}` |
|
||||
| `lessThan` | Less than | `{price: {lessThan: 100}}` |
|
||||
| `greaterEqual` | Greater or equal | `{score: {greaterEqual: 90}}` |
|
||||
| `lessEqual` | Less or equal | `{rating: {lessEqual: 3}}` |
|
||||
| `oneOf` | In array | `{color: {oneOf: ['red', 'blue']}}` |
|
||||
| `notOneOf` | Not in array | `{status: {notOneOf: ['deleted']}}` |
|
||||
| `contains` | Contains value | `{tags: {contains: 'ai'}}` |
|
||||
| `startsWith` | String prefix | `{name: {startsWith: 'John'}}` |
|
||||
| `endsWith` | String suffix | `{email: {endsWith: '@gmail.com'}}` |
|
||||
| `matches` | Pattern match | `{text: {matches: /^[A-Z]/}}` |
|
||||
| `between` | Range | `{year: {between: [2020, 2024]}}` |
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### Basic Usage
|
||||
```typescript
|
||||
// Add data
|
||||
const id = await brain.addNoun('Quantum computing breakthrough', {
|
||||
category: 'technology',
|
||||
year: 2024,
|
||||
importance: 'high'
|
||||
})
|
||||
|
||||
// Simple search
|
||||
const results = await brain.search('quantum physics', 5)
|
||||
|
||||
// Complex query with Triple Intelligence
|
||||
const articles = await brain.find({
|
||||
like: 'quantum computing',
|
||||
where: {
|
||||
year: { greaterThan: 2022 },
|
||||
importance: { oneOf: ['high', 'critical'] }
|
||||
},
|
||||
connected: {
|
||||
via: 'references',
|
||||
depth: 2
|
||||
},
|
||||
limit: 10
|
||||
})
|
||||
```
|
||||
|
||||
## Migration from v1.x
|
||||
### Creating Knowledge Graphs
|
||||
```typescript
|
||||
// Add entities
|
||||
const ai = await brain.addNoun('Artificial Intelligence')
|
||||
const ml = await brain.addNoun('Machine Learning')
|
||||
const dl = await brain.addNoun('Deep Learning')
|
||||
|
||||
See the [Migration Guide](../MIGRATION.md) for upgrading from Brainy 1.x to 2.0.
|
||||
// Create relationships
|
||||
await brain.addVerb(ml, ai, 'subset_of')
|
||||
await brain.addVerb(dl, ml, 'subset_of')
|
||||
await brain.addVerb(dl, ai, 'enables')
|
||||
|
||||
// Traverse the graph
|
||||
const aiEcosystem = await brain.find({
|
||||
connected: { from: ai, depth: 3 }
|
||||
})
|
||||
```
|
||||
|
||||
### Using Neural Features
|
||||
```typescript
|
||||
// Find similar concepts
|
||||
const similarity = await brain.neural.similar(
|
||||
'renewable energy',
|
||||
'sustainable power'
|
||||
)
|
||||
|
||||
// Auto-cluster documents
|
||||
const clusters = await brain.neural.clusters({
|
||||
method: 'kmeans',
|
||||
k: 5
|
||||
})
|
||||
|
||||
// Generate visualization
|
||||
const vizData = await brain.neural.visualize({
|
||||
maxNodes: 200,
|
||||
algorithm: 'force',
|
||||
dimensions: 3
|
||||
})
|
||||
// Use vizData with D3.js, Cytoscape, etc.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Features
|
||||
|
||||
### ✨ Zero Configuration
|
||||
Works instantly with sensible defaults. No setup required.
|
||||
|
||||
### 🧠 Triple Intelligence
|
||||
Combines vector search, graph traversal, and metadata filtering in one query.
|
||||
|
||||
### 🚀 Auto-Embedding
|
||||
Text automatically converts to vectors - no manual embedding needed.
|
||||
|
||||
### 📊 Built-in Visualization
|
||||
Export data formatted for popular visualization libraries.
|
||||
|
||||
### 🔒 Clean Operators
|
||||
Readable, intuitive operators - no cryptic symbols.
|
||||
|
||||
### 🎯 Everything Included
|
||||
All features in the MIT licensed package - no premium tiers.
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
- **GitHub**: [github.com/soulcraft/brainy](https://github.com/soulcraft/brainy)
|
||||
- **Documentation**: [docs.soulcraft.com/brainy](https://docs.soulcraft.com/brainy)
|
||||
- **License**: MIT
|
||||
|
||||
---
|
||||
|
||||
*Brainy 2.0 - Intelligence for Everyone*
|
||||
|
|
@ -462,7 +462,7 @@ for (const order of orders) {
|
|||
// Similar to graph databases but with added benefits:
|
||||
// 1. Automatic vector embeddings for similarity
|
||||
// 2. Natural language querying
|
||||
// 3. Unified with field filtering
|
||||
// 3. Unified with metadata filtering
|
||||
|
||||
// Enhanced graph queries
|
||||
const results = await brain.find("similar users who purchased similar products")
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Architecture Overview
|
||||
|
||||
Brainy is a multi-dimensional AI database that combines vector similarity, graph relationships, and field filtering into a unified query system. This document provides a comprehensive overview of the system architecture.
|
||||
Brainy is a multi-dimensional AI database that combines vector similarity, graph relationships, and metadata filtering into a unified query system. This document provides a comprehensive overview of the system architecture.
|
||||
|
||||
## Core Components
|
||||
|
||||
|
|
@ -23,7 +23,7 @@ Brainy's revolutionary feature that unifies three types of search:
|
|||
const results = await brain.find({
|
||||
like: "machine learning papers", // Vector similarity
|
||||
connected: { to: "research-team", depth: 2 }, // Graph traversal
|
||||
where: { published: { $gte: "2024-01-01" } } // Field filtering
|
||||
where: { published: { $gte: "2024-01-01" } } // Metadata filtering
|
||||
})
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
# Triple Intelligence System
|
||||
|
||||
The Triple Intelligence System is Brainy's revolutionary query engine that unifies vector similarity, graph relationships, and field filtering into a single, optimized query interface.
|
||||
The Triple Intelligence System is Brainy's revolutionary query engine that unifies vector similarity, graph relationships, and metadata filtering into a single, optimized query interface.
|
||||
|
||||
## Overview
|
||||
|
||||
Traditional databases force you to choose between vector search, graph traversal, OR field filtering. Brainy combines all three intelligences into one magical API that automatically optimizes execution for maximum performance.
|
||||
Traditional databases force you to choose between vector search, graph traversal, OR metadata filtering. Brainy combines all three intelligences into one magical API that automatically optimizes execution for maximum performance.
|
||||
|
||||
## Query Interface
|
||||
|
||||
|
|
@ -116,7 +116,7 @@ const results = await brain.find({
|
|||
like: "recent posts", // Applied to filtered set
|
||||
limit: 5
|
||||
})
|
||||
// Field filter first, then vector search on results
|
||||
// Metadata filter first, then vector search on results
|
||||
```
|
||||
|
||||
## Fusion Ranking
|
||||
|
|
@ -193,7 +193,7 @@ Brainy uses itself to optimize queries:
|
|||
|
||||
Triple Intelligence leverages all available indexes:
|
||||
- **HNSW Index**: For vector similarity
|
||||
- **Metadata Index**: For field filtering
|
||||
- **Metadata Index**: For metadata filtering
|
||||
- **Graph Index**: For relationship traversal
|
||||
|
||||
## Advanced Features
|
||||
|
|
|
|||
18
docs/augmentations-archive/README.md
Normal file
18
docs/augmentations-archive/README.md
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
# Augmentations Documentation Archive
|
||||
|
||||
This directory contains historical augmentation documentation from the Brainy 2.0 development process.
|
||||
|
||||
## Current Documentation
|
||||
The definitive augmentation documentation is now located at:
|
||||
- **`/docs/augmentations/README.md`** - Main augmentation guide
|
||||
- **`/docs/architecture/augmentations.md`** - Architecture details
|
||||
|
||||
## Archived Documents
|
||||
These documents represent the evolution of the augmentation system design:
|
||||
- Various implementation approaches
|
||||
- Pipeline architecture exploration
|
||||
- Storage augmentation patterns
|
||||
- Example implementations
|
||||
|
||||
## Note
|
||||
These documents are archived to preserve development history while maintaining a clean documentation structure.
|
||||
|
|
@ -15,7 +15,7 @@ Unified query system that automatically combines:
|
|||
// All three intelligences work together automatically
|
||||
const results = await brain.find({
|
||||
like: 'AI research', // Vector search
|
||||
where: { year: 2024 }, // Field filtering
|
||||
where: { year: 2024 }, // Metadata filtering
|
||||
connected: { to: authorId } // Graph traversal
|
||||
})
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Getting Started with Brainy
|
||||
|
||||
This guide will help you get up and running with Brainy, the multi-dimensional AI database that combines vector similarity, graph relationships, and field filtering.
|
||||
This guide will help you get up and running with Brainy, the multi-dimensional AI database that combines vector similarity, graph relationships, and metadata filtering.
|
||||
|
||||
## Installation
|
||||
|
||||
|
|
@ -90,7 +90,7 @@ results.forEach(result => {
|
|||
const results = await brain.find("show me technology articles about AI from 2023")
|
||||
// Automatically interprets: topic, category, and time range
|
||||
|
||||
// Structured queries with vector similarity and field filtering
|
||||
// Structured queries with vector similarity and metadata filtering
|
||||
const structured = await brain.find({
|
||||
like: "artificial intelligence",
|
||||
where: {
|
||||
|
|
|
|||
|
|
@ -162,7 +162,7 @@ The natural language is converted to a structured Triple Intelligence query:
|
|||
### 5. Execution
|
||||
The structured query is executed using Triple Intelligence, combining:
|
||||
- Vector similarity search
|
||||
- Field filtering
|
||||
- Metadata filtering
|
||||
- Graph traversal
|
||||
|
||||
## Advanced Features
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@
|
|||
- ✅ Natural language queries ("find developers")
|
||||
- ✅ Vector similarity search (`similar: 'text'`)
|
||||
- ✅ Graph traversal (`connected: { to: 'node' }`)
|
||||
- ✅ Field filtering (`where: { field: 'value' }`)
|
||||
- ✅ Metadata filtering (`where: { field: 'value' }`)
|
||||
- ✅ Combined intelligence with fusion scoring
|
||||
- ✅ Performance optimized for complex queries
|
||||
- **Status: PRODUCTION READY**
|
||||
|
|
@ -94,7 +94,7 @@ const viz = await neural.visualize()
|
|||
```
|
||||
|
||||
### Core Innovation Validated
|
||||
- ✅ **Triple Intelligence**: Vector + Graph + Field search unified
|
||||
- ✅ **Triple Intelligence**: Vector + Graph + Metadata search unified
|
||||
- ✅ **Intelligent Verb Scoring**: Smart relationship weights
|
||||
- ✅ **Neural APIs**: Ready for external visualization libraries
|
||||
- ✅ **Zero Config**: Works perfectly out of the box
|
||||
|
|
@ -118,7 +118,7 @@ const viz = await neural.visualize()
|
|||
- **Scalability**: Tested with complex datasets
|
||||
|
||||
### Innovation Leadership ✅
|
||||
- **First True Triple Intelligence**: Vector + Graph + Field unified
|
||||
- **First True Triple Intelligence**: Vector + Graph + Metadata unified
|
||||
- **Smart by Default**: No configuration required
|
||||
- **Revolutionary Data Model**: Noun-verb taxonomy
|
||||
- **Neural API**: Ready for external clustering/visualization libraries
|
||||
|
|
@ -133,7 +133,7 @@ const viz = await neural.visualize()
|
|||
|
||||
Brainy 2.0 represents a fundamental leap forward in vector database technology:
|
||||
|
||||
1. **Triple Intelligence** solves the problem of having to choose between vector, graph, or field search
|
||||
1. **Triple Intelligence** solves the problem of having to choose between vector, graph, or metadata search
|
||||
2. **Intelligent Verb Scoring** automatically computes optimal relationship weights
|
||||
3. **Neural APIs** enable external libraries to build advanced visualizations
|
||||
4. **Zero Configuration** makes it accessible to all developers
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@
|
|||
|
||||
### Core Features
|
||||
- ✅ **Noun-Verb Taxonomy** - Complete implementation with addNoun() and addVerb()
|
||||
- ✅ **Triple Intelligence Engine** - Vector + Graph + Field unified queries
|
||||
- ✅ **Triple Intelligence Engine** - Vector + Graph + Metadata unified queries
|
||||
- ✅ **Natural Language find()** - Basic NLP with 220+ embedded patterns
|
||||
- ✅ **HNSW Vector Search** - O(log n) similarity search
|
||||
- ✅ **Field Indexing** - O(1) metadata lookups via FieldIndex class
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ After thorough investigation of the codebase, here's what's ACTUALLY implemented
|
|||
|
||||
### Core Features
|
||||
- ✅ **Noun-Verb Taxonomy** - Complete with addNoun() and addVerb()
|
||||
- ✅ **Triple Intelligence Engine** - Vector + Graph + Field unified queries
|
||||
- ✅ **Triple Intelligence Engine** - Vector + Graph + Metadata unified queries
|
||||
- ✅ **Natural Language find()** - Basic NLP with 220+ embedded patterns
|
||||
- ✅ **HNSW Vector Search** - O(log n) similarity search with partitioning support
|
||||
- ✅ **Field Indexing** - O(1) metadata lookups via FieldIndex class
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
|
||||
### Core Features (100% Complete)
|
||||
- ✅ **Noun-Verb Taxonomy**: Revolutionary data model
|
||||
- ✅ **Triple Intelligence**: Vector + Graph + Field unified queries
|
||||
- ✅ **Triple Intelligence**: Vector + Graph + Metadata unified queries
|
||||
- ✅ **HNSW Indexing**: O(log n) vector search
|
||||
- ✅ **384 Dimensions**: Fixed with all-MiniLM-L6-v2
|
||||
- ✅ **Zero-Config**: Works out of the box
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@
|
|||
- Real transformer models (no mocking)
|
||||
|
||||
### 6. Natural Language ✅
|
||||
- `tests/triple-intelligence.test.ts` - Vector + Graph + Field queries
|
||||
- `tests/triple-intelligence.test.ts` - Vector + Graph + Metadata queries
|
||||
- Natural language query understanding
|
||||
|
||||
### 7. Error Handling ✅
|
||||
|
|
|
|||
|
|
@ -225,7 +225,7 @@ const brain = new BrainyData({
|
|||
const results = await brain.search("query", { limit: 10 })
|
||||
```
|
||||
|
||||
3. **Consider field filtering first**
|
||||
3. **Consider metadata filtering first**
|
||||
```typescript
|
||||
// Filter by metadata first, then semantic search
|
||||
const results = await brain.search("query", {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue