fix: exclude __words__ keyword index from corruption detection and getStats()
The __words__ keyword index stores 50-5000 entries per entity (one per word), which inflated avg entries/entity well above the corruption threshold of 100. This caused: 1. validateConsistency() to falsely detect corruption on every startup, triggering unnecessary clearAllIndexData() + rebuild() cycles 2. getStats() to log false "Metadata index may be corrupted" warnings and report inflated totalEntries/totalIds stats Both methods now skip __words__ when counting, so stats and health checks reflect metadata fields only (noun, type, createdAt, etc.). Keyword search is unaffected since the __words__ field index itself is not modified.
This commit is contained in:
parent
32dbdcec61
commit
364360d447
128 changed files with 5637 additions and 5682 deletions
|
|
@ -1,8 +1,8 @@
|
|||
# 🔌 Brainy v4.0.0 Augmentations Complete Reference
|
||||
# 🔌 Brainy Augmentations Complete Reference
|
||||
|
||||
> **All augmentations that power Brainy's extensibility - with locations, usage, and examples**
|
||||
>
|
||||
> **⚠️ v4.0.0 Update**: Updated for metadata structure changes and billion-scale optimizations
|
||||
> **⚠️ Update**: Updated for metadata structure changes and billion-scale optimizations
|
||||
|
||||
## Quick Start
|
||||
|
||||
|
|
@ -10,37 +10,37 @@
|
|||
import { Brainy } from '@soulcraft/brainy'
|
||||
|
||||
const brain = new Brainy({
|
||||
// Augmentations auto-configure based on environment
|
||||
storage: 'auto', // Storage augmentation
|
||||
cache: true, // Cache augmentation
|
||||
index: true // Index augmentation
|
||||
// Augmentations auto-configure based on environment
|
||||
storage: 'auto', // Storage augmentation
|
||||
cache: true, // Cache augmentation
|
||||
index: true // Index augmentation
|
||||
})
|
||||
|
||||
await brain.init() // Augmentations initialize automatically
|
||||
await brain.init() // Augmentations initialize automatically
|
||||
```
|
||||
|
||||
## v4.0.0 Augmentation Architecture
|
||||
## Augmentation Architecture
|
||||
|
||||
### Key Improvements for Billion-Scale Performance
|
||||
|
||||
1. **Metadata/Vector Separation**: Augmentations now work with separated metadata and vectors
|
||||
- Metadata stored separately from vector data
|
||||
- 99.2% memory reduction for type tracking
|
||||
- Two-file storage pattern for optimal I/O
|
||||
- Metadata stored separately from vector data
|
||||
- 99.2% memory reduction for type tracking
|
||||
- Two-file storage pattern for optimal I/O
|
||||
|
||||
2. **Type System Enforcement**: All metadata requires type fields
|
||||
- `NounMetadata` requires `noun: NounType`
|
||||
- `VerbMetadata` requires `verb: VerbType`
|
||||
- Type inference system available as public API
|
||||
- `NounMetadata` requires `noun: NounType`
|
||||
- `VerbMetadata` requires `verb: VerbType`
|
||||
- Type inference system available as public API
|
||||
|
||||
3. **Storage Adapter Pattern**: Internal vs public method distinction
|
||||
- `_methods`: Return pure structures (HNSWNoun, HNSWVerb)
|
||||
- Public methods: Return WithMetadata types
|
||||
- MetadataEnforcer Proxy ensures proper access
|
||||
- `_methods`: Return pure structures (HNSWNoun, HNSWVerb)
|
||||
- Public methods: Return WithMetadata types
|
||||
- MetadataEnforcer Proxy ensures proper access
|
||||
|
||||
### What This Means for Augmentation Users
|
||||
|
||||
**✅ If you use built-in augmentations**: No changes needed! They're all updated for v4.0.0.
|
||||
**✅ If you use built-in augmentations**: No changes needed! They're all updated.
|
||||
|
||||
**⚠️ If you created custom storage augmentations**: Update your storage adapter to:
|
||||
- Wrap metadata with required `noun`/`verb` fields
|
||||
|
|
@ -69,81 +69,81 @@ Augmentations are modular extensions that add functionality to Brainy without cl
|
|||
## Storage Augmentations (8 total)
|
||||
|
||||
### MemoryStorageAugmentation
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Auto-enabled**: When `storage: 'memory'` or in test environments
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Auto-enabled**: When `storage: 'memory'` or in test environments
|
||||
**Purpose**: In-memory storage for testing and temporary data
|
||||
```typescript
|
||||
const brain = new Brainy({ storage: 'memory' })
|
||||
```
|
||||
|
||||
### FileSystemStorageAugmentation
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Auto-enabled**: When `storage: 'filesystem'` or Node.js detected
|
||||
### FileSystemStorageAugmentation
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Auto-enabled**: When `storage: 'filesystem'` or Node.js detected
|
||||
**Purpose**: Persistent file-based storage for Node.js applications
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'filesystem', path: './data' }
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'filesystem', path: './data' }
|
||||
})
|
||||
```
|
||||
|
||||
### OPFSStorageAugmentation
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Auto-enabled**: When `storage: 'opfs'` or browser with OPFS support
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Auto-enabled**: When `storage: 'opfs'` or browser with OPFS support
|
||||
**Purpose**: Browser-based persistent storage using Origin Private File System
|
||||
```typescript
|
||||
const brain = new Brainy({ storage: 'opfs' })
|
||||
```
|
||||
|
||||
### S3StorageAugmentation
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Manual**: Requires AWS credentials
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Manual**: Requires AWS credentials
|
||||
**Purpose**: AWS S3-compatible cloud storage
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 's3',
|
||||
bucket: 'my-bucket',
|
||||
region: 'us-east-1',
|
||||
credentials: { accessKeyId, secretAccessKey }
|
||||
}
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 's3',
|
||||
bucket: 'my-bucket',
|
||||
region: 'us-east-1',
|
||||
credentials: { accessKeyId, secretAccessKey }
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
### R2StorageAugmentation
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Manual**: Requires Cloudflare credentials
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Manual**: Requires Cloudflare credentials
|
||||
**Purpose**: Cloudflare R2 storage (S3-compatible)
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'r2',
|
||||
accountId: 'xxx',
|
||||
bucket: 'my-bucket',
|
||||
credentials: { accessKeyId, secretAccessKey }
|
||||
}
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'r2',
|
||||
accountId: 'xxx',
|
||||
bucket: 'my-bucket',
|
||||
credentials: { accessKeyId, secretAccessKey }
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
### GCSStorageAugmentation
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Manual**: Requires Google Cloud credentials
|
||||
**Location**: `src/augmentations/storageAugmentations.ts`
|
||||
**Manual**: Requires Google Cloud credentials
|
||||
**Purpose**: Google Cloud Storage
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'gcs',
|
||||
bucket: 'my-bucket',
|
||||
projectId: 'my-project'
|
||||
}
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'gcs',
|
||||
bucket: 'my-bucket',
|
||||
projectId: 'my-project'
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
### StorageAugmentation (base)
|
||||
**Location**: `src/augmentations/storageAugmentation.ts`
|
||||
**Location**: `src/augmentations/storageAugmentation.ts`
|
||||
**Purpose**: Base class for custom storage implementations
|
||||
|
||||
### DynamicStorageAugmentation
|
||||
**Location**: `src/augmentations/storageAugmentation.ts`
|
||||
**Location**: `src/augmentations/storageAugmentation.ts`
|
||||
**Purpose**: Runtime storage adapter switching
|
||||
|
||||
---
|
||||
|
|
@ -151,17 +151,17 @@ const brain = new Brainy({
|
|||
## Performance Augmentations (7 total)
|
||||
|
||||
### CacheAugmentation
|
||||
**Location**: `src/augmentations/cacheAugmentation.ts`
|
||||
**Auto-enabled**: When `cache: true` (default)
|
||||
**Location**: `src/augmentations/cacheAugmentation.ts`
|
||||
**Auto-enabled**: When `cache: true` (default)
|
||||
**Purpose**: LRU cache for search results and frequent queries
|
||||
```typescript
|
||||
brain.clearCache() // Exposed via API
|
||||
brain.getCacheStats() // Cache hit/miss statistics
|
||||
brain.clearCache() // Exposed via API
|
||||
brain.getCacheStats() // Cache hit/miss statistics
|
||||
```
|
||||
|
||||
### IndexAugmentation
|
||||
**Location**: `src/augmentations/indexAugmentation.ts`
|
||||
**Auto-enabled**: When `index: true` (default)
|
||||
**Location**: `src/augmentations/indexAugmentation.ts`
|
||||
**Auto-enabled**: When `index: true` (default)
|
||||
**Purpose**: Metadata indexing for O(1) field lookups
|
||||
```typescript
|
||||
brain.rebuildMetadataIndex() // Exposed via API
|
||||
|
|
@ -170,41 +170,41 @@ brain.find({ where: { category: 'tech' } })
|
|||
```
|
||||
|
||||
### MetricsAugmentation
|
||||
**Location**: `src/augmentations/metricsAugmentation.ts`
|
||||
**Auto-enabled**: Always active
|
||||
**Location**: `src/augmentations/metricsAugmentation.ts`
|
||||
**Auto-enabled**: Always active
|
||||
**Purpose**: Performance metrics and statistics collection
|
||||
```typescript
|
||||
brain.getStats() // Comprehensive metrics
|
||||
brain.getStats() // Comprehensive metrics
|
||||
```
|
||||
|
||||
### MonitoringAugmentation
|
||||
**Location**: `src/augmentations/monitoringAugmentation.ts`
|
||||
**Manual**: Register for detailed monitoring
|
||||
**Location**: `src/augmentations/monitoringAugmentation.ts`
|
||||
**Manual**: Register for detailed monitoring
|
||||
**Purpose**: Real-time performance monitoring and alerts
|
||||
|
||||
### BatchProcessingAugmentation
|
||||
**Location**: `src/augmentations/batchProcessingAugmentation.ts`
|
||||
**Auto-enabled**: For batch operations
|
||||
**Location**: `src/augmentations/batchProcessingAugmentation.ts`
|
||||
**Auto-enabled**: For batch operations
|
||||
**Purpose**: Optimizes bulk add/update/delete operations
|
||||
```typescript
|
||||
brain.addNouns([...]) // Automatically batched
|
||||
brain.addNouns([...]) // Automatically batched
|
||||
```
|
||||
|
||||
### RequestDeduplicatorAugmentation
|
||||
**Location**: `src/augmentations/requestDeduplicatorAugmentation.ts`
|
||||
**Auto-enabled**: Always active
|
||||
**Location**: `src/augmentations/requestDeduplicatorAugmentation.ts`
|
||||
**Auto-enabled**: Always active
|
||||
**Purpose**: Prevents duplicate concurrent operations
|
||||
|
||||
### ConnectionPoolAugmentation
|
||||
**Location**: `src/augmentations/connectionPoolAugmentation.ts`
|
||||
**Auto-enabled**: For network storage
|
||||
**Location**: `src/augmentations/connectionPoolAugmentation.ts`
|
||||
**Auto-enabled**: For network storage
|
||||
**Purpose**: Connection pooling for cloud storage adapters
|
||||
|
||||
---
|
||||
|
||||
## Data Integrity Augmentations (3 total)
|
||||
|
||||
**Auto-enabled**: When `wal: true`
|
||||
**Auto-enabled**: When `wal: true`
|
||||
**Purpose**: Write-ahead logging for crash recovery
|
||||
```typescript
|
||||
const brain = new Brainy({ wal: true })
|
||||
|
|
@ -212,8 +212,8 @@ const brain = new Brainy({ wal: true })
|
|||
```
|
||||
|
||||
### EntityRegistryAugmentation
|
||||
**Location**: `src/augmentations/entityRegistryAugmentation.ts`
|
||||
**Auto-enabled**: For streaming operations
|
||||
**Location**: `src/augmentations/entityRegistryAugmentation.ts`
|
||||
**Auto-enabled**: For streaming operations
|
||||
**Purpose**: High-speed deduplication for real-time data
|
||||
```typescript
|
||||
// Prevents duplicate entities in streaming scenarios
|
||||
|
|
@ -221,8 +221,8 @@ brain.add(data) // Automatically deduplicated
|
|||
```
|
||||
|
||||
### AutoRegisterEntitiesAugmentation
|
||||
**Location**: `src/augmentations/entityRegistryAugmentation.ts`
|
||||
**Manual**: For automatic entity discovery
|
||||
**Location**: `src/augmentations/entityRegistryAugmentation.ts`
|
||||
**Manual**: For automatic entity discovery
|
||||
**Purpose**: Auto-discovers and registers entities from data
|
||||
|
||||
---
|
||||
|
|
@ -230,20 +230,20 @@ brain.add(data) // Automatically deduplicated
|
|||
## Intelligence Augmentations (2 total)
|
||||
|
||||
### NeuralImportAugmentation
|
||||
**Location**: `src/augmentations/neuralImport.ts`
|
||||
**Manual**: Via `brain.neuralImport()`
|
||||
**Location**: `src/augmentations/neuralImport.ts`
|
||||
**Manual**: Via `brain.neuralImport()`
|
||||
**Purpose**: AI-powered smart data import
|
||||
```typescript
|
||||
const result = await brain.neuralImport(data, {
|
||||
confidenceThreshold: 0.7,
|
||||
autoApply: true
|
||||
confidenceThreshold: 0.7,
|
||||
autoApply: true
|
||||
})
|
||||
// Automatically detects entities and relationships
|
||||
```
|
||||
|
||||
### IntelligentVerbScoringAugmentation
|
||||
**Location**: `src/augmentations/intelligentVerbScoringAugmentation.ts`
|
||||
**Auto-enabled**: When verbs are used
|
||||
**Location**: `src/augmentations/intelligentVerbScoringAugmentation.ts`
|
||||
**Auto-enabled**: When verbs are used
|
||||
**Purpose**: ML-based relationship strength scoring
|
||||
```typescript
|
||||
brain.verbScoring.train(feedback)
|
||||
|
|
@ -255,8 +255,8 @@ brain.verbScoring.getScore(verbId)
|
|||
## Communication Augmentations (4 total)
|
||||
|
||||
### APIServerAugmentation
|
||||
**Location**: `src/augmentations/apiServerAugmentation.ts`
|
||||
**Manual**: For server deployments
|
||||
**Location**: `src/augmentations/apiServerAugmentation.ts`
|
||||
**Manual**: For server deployments
|
||||
**Purpose**: REST/WebSocket/MCP API server
|
||||
```typescript
|
||||
const augmentation = new APIServerAugmentation()
|
||||
|
|
@ -265,8 +265,8 @@ await brain.registerAugmentation(augmentation)
|
|||
```
|
||||
|
||||
### WebSocketConduitAugmentation
|
||||
**Location**: `src/augmentations/conduitAugmentations.ts`
|
||||
**Manual**: For Brainy-to-Brainy sync
|
||||
**Location**: `src/augmentations/conduitAugmentations.ts`
|
||||
**Manual**: For Brainy-to-Brainy sync
|
||||
**Purpose**: Real-time sync between Brainy instances
|
||||
```typescript
|
||||
const conduit = new WebSocketConduitAugmentation()
|
||||
|
|
@ -274,13 +274,13 @@ await conduit.establishConnection('ws://other-brain')
|
|||
```
|
||||
|
||||
### ServerSearchConduitAugmentation
|
||||
**Location**: `src/augmentations/serverSearchAugmentations.ts`
|
||||
**Manual**: For client-server search
|
||||
**Location**: `src/augmentations/serverSearchAugmentations.ts`
|
||||
**Manual**: For client-server search
|
||||
**Purpose**: Search remote Brainy instance, cache locally
|
||||
|
||||
### ServerSearchActivationAugmentation
|
||||
**Location**: `src/augmentations/serverSearchAugmentations.ts`
|
||||
**Manual**: Works with ServerSearchConduit
|
||||
**Location**: `src/augmentations/serverSearchAugmentations.ts`
|
||||
**Manual**: Works with ServerSearchConduit
|
||||
**Purpose**: Triggers and manages server search operations
|
||||
|
||||
---
|
||||
|
|
@ -288,18 +288,18 @@ await conduit.establishConnection('ws://other-brain')
|
|||
## External Integration (2 total)
|
||||
|
||||
### SynapseAugmentation (base)
|
||||
**Location**: `src/augmentations/synapseAugmentation.ts`
|
||||
**Location**: `src/augmentations/synapseAugmentation.ts`
|
||||
**Purpose**: Base class for external platform integrations
|
||||
```typescript
|
||||
// Example: NotionSynapse, SlackSynapse, etc.
|
||||
class NotionSynapse extends SynapseAugmentation {
|
||||
async fetchData() { /* Notion API calls */ }
|
||||
async pushData() { /* Sync to Notion */ }
|
||||
async fetchData() { /* Notion API calls */ }
|
||||
async pushData() { /* Sync to Notion */ }
|
||||
}
|
||||
```
|
||||
|
||||
### ExampleFileSystemSynapse
|
||||
**Location**: `src/augmentations/synapseAugmentation.ts`
|
||||
**Location**: `src/augmentations/synapseAugmentation.ts`
|
||||
**Purpose**: Example implementation for file system sync
|
||||
|
||||
---
|
||||
|
|
@ -309,11 +309,11 @@ class NotionSynapse extends SynapseAugmentation {
|
|||
### Auto-Configuration
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
// These auto-register augmentations:
|
||||
storage: 'auto', // Storage augmentation
|
||||
cache: true, // Cache augmentation
|
||||
index: true, // Index augmentation
|
||||
metrics: true // Metrics augmentation
|
||||
// These auto-register augmentations:
|
||||
storage: 'auto', // Storage augmentation
|
||||
cache: true, // Cache augmentation
|
||||
index: true, // Index augmentation
|
||||
metrics: true // Metrics augmentation
|
||||
})
|
||||
```
|
||||
|
||||
|
|
@ -333,29 +333,29 @@ await brain.init()
|
|||
import { BaseAugmentation } from '@soulcraft/brainy'
|
||||
|
||||
class MyAugmentation extends BaseAugmentation {
|
||||
readonly name = 'my-augmentation'
|
||||
readonly timing = 'after' // before | after | both
|
||||
readonly operations = ['addNoun', 'search'] // Which ops to hook
|
||||
readonly priority = 10 // Execution order (lower = earlier)
|
||||
|
||||
protected async onInit(): Promise<void> {
|
||||
// Initialize your augmentation
|
||||
}
|
||||
|
||||
async execute<T>(
|
||||
operation: string,
|
||||
params: any,
|
||||
context?: AugmentationContext
|
||||
): Promise<T | void> {
|
||||
// Your augmentation logic
|
||||
if (operation === 'addNoun') {
|
||||
console.log('Noun added:', params)
|
||||
}
|
||||
}
|
||||
|
||||
protected async onShutdown(): Promise<void> {
|
||||
// Cleanup
|
||||
}
|
||||
readonly name = 'my-augmentation'
|
||||
readonly timing = 'after' // before | after | both
|
||||
readonly operations = ['addNoun', 'search'] // Which ops to hook
|
||||
readonly priority = 10 // Execution order (lower = earlier)
|
||||
|
||||
protected async onInit(): Promise<void> {
|
||||
// Initialize your augmentation
|
||||
}
|
||||
|
||||
async execute<T>(
|
||||
operation: string,
|
||||
params: any,
|
||||
context?: AugmentationContext
|
||||
): Promise<T | void> {
|
||||
// Your augmentation logic
|
||||
if (operation === 'addNoun') {
|
||||
console.log('Noun added:', params)
|
||||
}
|
||||
}
|
||||
|
||||
protected async onShutdown(): Promise<void> {
|
||||
// Cleanup
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
# 🛠️ Brainy Augmentation Developer Guide
|
||||
|
||||
> **How to create, test, and use augmentations in Brainy v4.0.0**
|
||||
> **How to create, test, and use augmentations in Brainy**
|
||||
>
|
||||
> **⚠️ v4.0.0 Update**: This guide has been updated with breaking changes for metadata structure and type system improvements.
|
||||
> **⚠️ Update**: This guide has been updated with breaking changes for metadata structure and type system improvements.
|
||||
|
||||
## v4.0.0 Migration Guide
|
||||
## Migration Guide
|
||||
|
||||
### What Changed?
|
||||
|
||||
|
|
@ -25,22 +25,22 @@
|
|||
```typescript
|
||||
// ❌ v3.x
|
||||
const verb = {
|
||||
type: 'relatedTo',
|
||||
sourceId: 'a',
|
||||
targetId: 'b'
|
||||
type: 'relatedTo',
|
||||
sourceId: 'a',
|
||||
targetId: 'b'
|
||||
}
|
||||
if (verb.type === 'relatedTo') { ... }
|
||||
|
||||
// ✅ v4.0.0
|
||||
// ✅ Current
|
||||
const verb = {
|
||||
verb: 'relatedTo',
|
||||
sourceId: 'a',
|
||||
targetId: 'b'
|
||||
verb: 'relatedTo',
|
||||
sourceId: 'a',
|
||||
targetId: 'b'
|
||||
}
|
||||
const metadata: VerbMetadata = {
|
||||
verb: 'relatedTo',
|
||||
sourceId: 'a',
|
||||
targetId: 'b'
|
||||
verb: 'relatedTo',
|
||||
sourceId: 'a',
|
||||
targetId: 'b'
|
||||
}
|
||||
if (verb.verb === 'relatedTo') { ... }
|
||||
```
|
||||
|
|
@ -51,40 +51,40 @@ if (verb.verb === 'relatedTo') { ... }
|
|||
import { BaseAugmentation, BrainyAugmentation, AugmentationContext } from '@soulcraft/brainy'
|
||||
|
||||
export class MyFirstAugmentation extends BaseAugmentation {
|
||||
readonly name = 'my-first-augmentation'
|
||||
readonly timing = 'after' as const // When to run: before | after | both
|
||||
readonly operations = ['add'] as const // Which operations to hook
|
||||
readonly priority = 10 // Execution order (lower = first)
|
||||
readonly name = 'my-first-augmentation'
|
||||
readonly timing = 'after' as const // When to run: before | after | both
|
||||
readonly operations = ['add'] as const // Which operations to hook
|
||||
readonly priority = 10 // Execution order (lower = first)
|
||||
|
||||
protected async onInit(): Promise<void> {
|
||||
// Initialize your augmentation
|
||||
console.log('MyFirstAugmentation initialized!')
|
||||
}
|
||||
protected async onInit(): Promise<void> {
|
||||
// Initialize your augmentation
|
||||
console.log('MyFirstAugmentation initialized!')
|
||||
}
|
||||
|
||||
async execute<T = any>(
|
||||
operation: string,
|
||||
params: any,
|
||||
context?: AugmentationContext
|
||||
): Promise<T | void> {
|
||||
// Your augmentation logic
|
||||
if (operation === 'add') {
|
||||
console.log('Noun added:', params.noun)
|
||||
async execute<T = any>(
|
||||
operation: string,
|
||||
params: any,
|
||||
context?: AugmentationContext
|
||||
): Promise<T | void> {
|
||||
// Your augmentation logic
|
||||
if (operation === 'add') {
|
||||
console.log('Noun added:', params.noun)
|
||||
|
||||
// v4.0.0: Access metadata correctly
|
||||
if (params.noun?.metadata) {
|
||||
console.log('Noun type:', params.noun.metadata.noun) // Required field
|
||||
}
|
||||
// Access metadata correctly
|
||||
if (params.noun?.metadata) {
|
||||
console.log('Noun type:', params.noun.metadata.noun) // Required field
|
||||
}
|
||||
|
||||
// You can access the brain instance
|
||||
const stats = await context?.brain.getStats()
|
||||
console.log('Total nouns:', stats.totalNouns)
|
||||
}
|
||||
}
|
||||
// You can access the brain instance
|
||||
const stats = await context?.brain.getStats()
|
||||
console.log('Total nouns:', stats.totalNouns)
|
||||
}
|
||||
}
|
||||
|
||||
protected async onShutdown(): Promise<void> {
|
||||
// Cleanup
|
||||
console.log('MyFirstAugmentation shutting down')
|
||||
}
|
||||
protected async onShutdown(): Promise<void> {
|
||||
// Cleanup
|
||||
console.log('MyFirstAugmentation shutting down')
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -113,23 +113,23 @@ await brain.add('Hello World')
|
|||
### 1. Registration Phase
|
||||
```typescript
|
||||
const aug = new MyAugmentation()
|
||||
brain.augmentations.register(aug) // Before brain.init()!
|
||||
brain.augmentations.register(aug) // Before brain.init()!
|
||||
```
|
||||
|
||||
### 2. Initialization Phase
|
||||
```typescript
|
||||
await brain.init() // Calls aug.initialize() internally
|
||||
await brain.init() // Calls aug.initialize() internally
|
||||
// Your onInit() method runs here
|
||||
```
|
||||
|
||||
### 3. Execution Phase
|
||||
```typescript
|
||||
await brain.add('data') // Your execute() method runs
|
||||
await brain.add('data') // Your execute() method runs
|
||||
```
|
||||
|
||||
### 4. Shutdown Phase
|
||||
```typescript
|
||||
await brain.shutdown() // Your onShutdown() method runs
|
||||
await brain.shutdown() // Your onShutdown() method runs
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -139,52 +139,52 @@ await brain.shutdown() // Your onShutdown() method runs
|
|||
### `before` - Modify Input
|
||||
```typescript
|
||||
class ValidationAugmentation extends BaseAugmentation {
|
||||
readonly timing = 'before' as const
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<any> {
|
||||
if (operation === 'add') {
|
||||
// Validate and/or modify params
|
||||
if (!params.content) {
|
||||
throw new Error('Content required')
|
||||
}
|
||||
// Return modified params
|
||||
return { ...params, validated: true }
|
||||
}
|
||||
}
|
||||
readonly timing = 'before' as const
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<any> {
|
||||
if (operation === 'add') {
|
||||
// Validate and/or modify params
|
||||
if (!params.content) {
|
||||
throw new Error('Content required')
|
||||
}
|
||||
// Return modified params
|
||||
return { ...params, validated: true }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `after` - React to Results
|
||||
```typescript
|
||||
class LoggingAugmentation extends BaseAugmentation {
|
||||
readonly timing = 'after' as const
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<void> {
|
||||
if (operation === 'search') {
|
||||
console.log(`Search for "${params.query}" returned ${params.result.length} results`)
|
||||
}
|
||||
// Don't return anything - just observe
|
||||
}
|
||||
readonly timing = 'after' as const
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<void> {
|
||||
if (operation === 'search') {
|
||||
console.log(`Search for "${params.query}" returned ${params.result.length} results`)
|
||||
}
|
||||
// Don't return anything - just observe
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `both` - Before AND After
|
||||
```typescript
|
||||
class TimingAugmentation extends BaseAugmentation {
|
||||
readonly timing = 'both' as const
|
||||
private startTime?: number
|
||||
|
||||
async execute<T>(operation: string, params: any, context?: AugmentationContext): Promise<void> {
|
||||
if (!this.startTime) {
|
||||
// Before execution
|
||||
this.startTime = Date.now()
|
||||
} else {
|
||||
// After execution
|
||||
const duration = Date.now() - this.startTime
|
||||
console.log(`${operation} took ${duration}ms`)
|
||||
this.startTime = undefined
|
||||
}
|
||||
}
|
||||
readonly timing = 'both' as const
|
||||
private startTime?: number
|
||||
|
||||
async execute<T>(operation: string, params: any, context?: AugmentationContext): Promise<void> {
|
||||
if (!this.startTime) {
|
||||
// Before execution
|
||||
this.startTime = Date.now()
|
||||
} else {
|
||||
// After execution
|
||||
const duration = Date.now() - this.startTime
|
||||
console.log(`${operation} took ${duration}ms`)
|
||||
this.startTime = undefined
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -195,28 +195,28 @@ class TimingAugmentation extends BaseAugmentation {
|
|||
### Core Operations You Can Hook
|
||||
```typescript
|
||||
readonly operations = [
|
||||
'add', // Adding data
|
||||
'update', // Updating data
|
||||
'delete', // Deleting data
|
||||
'get', // Retrieving data
|
||||
'search', // Searching
|
||||
'find', // Triple Intelligence queries
|
||||
'relate', // Adding relationships
|
||||
'unrelate', // Removing relationships
|
||||
'clear', // Clearing data
|
||||
'all' // Hook ALL operations
|
||||
'add', // Adding data
|
||||
'update', // Updating data
|
||||
'delete', // Deleting data
|
||||
'get', // Retrieving data
|
||||
'search', // Searching
|
||||
'find', // Triple Intelligence queries
|
||||
'relate', // Adding relationships
|
||||
'unrelate', // Removing relationships
|
||||
'clear', // Clearing data
|
||||
'all' // Hook ALL operations
|
||||
] as const
|
||||
```
|
||||
|
||||
### Example: Multi-Operation Hook
|
||||
```typescript
|
||||
class AuditAugmentation extends BaseAugmentation {
|
||||
readonly operations = ['add', 'update', 'delete'] as const
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<void> {
|
||||
// Log all data modifications
|
||||
await this.logToAuditTrail(operation, params)
|
||||
}
|
||||
readonly operations = ['add', 'update', 'delete'] as const
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<void> {
|
||||
// Log all data modifications
|
||||
await this.logToAuditTrail(operation, params)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -226,26 +226,26 @@ class AuditAugmentation extends BaseAugmentation {
|
|||
|
||||
```typescript
|
||||
class ContextAwareAugmentation extends BaseAugmentation {
|
||||
async execute<T>(
|
||||
operation: string,
|
||||
params: any,
|
||||
context?: AugmentationContext
|
||||
): Promise<void> {
|
||||
// Access the brain instance
|
||||
const brain = context?.brain
|
||||
if (!brain) return
|
||||
|
||||
// Use any brain method
|
||||
const stats = await brain.getStats()
|
||||
const size = await brain.size()
|
||||
const results = await brain.search('query')
|
||||
|
||||
// Access other augmentations
|
||||
const cache = brain.augmentations.get('cache')
|
||||
if (cache) {
|
||||
await cache.clear()
|
||||
}
|
||||
}
|
||||
async execute<T>(
|
||||
operation: string,
|
||||
params: any,
|
||||
context?: AugmentationContext
|
||||
): Promise<void> {
|
||||
// Access the brain instance
|
||||
const brain = context?.brain
|
||||
if (!brain) return
|
||||
|
||||
// Use any brain method
|
||||
const stats = await brain.getStats()
|
||||
const size = await brain.size()
|
||||
const results = await brain.search('query')
|
||||
|
||||
// Access other augmentations
|
||||
const cache = brain.augmentations.get('cache')
|
||||
if (cache) {
|
||||
await cache.clear()
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -256,92 +256,92 @@ class ContextAwareAugmentation extends BaseAugmentation {
|
|||
### 1. Backup Augmentation
|
||||
```typescript
|
||||
class BackupAugmentation extends BaseAugmentation {
|
||||
readonly name = 'backup'
|
||||
readonly timing = 'after' as const
|
||||
readonly operations = ['add', 'update', 'delete'] as const
|
||||
readonly priority = 5
|
||||
|
||||
private changes = 0
|
||||
private readonly backupThreshold = 100
|
||||
|
||||
async execute<T>(operation: string, params: any, context?: AugmentationContext): Promise<void> {
|
||||
this.changes++
|
||||
|
||||
if (this.changes >= this.backupThreshold) {
|
||||
await this.performBackup(context?.brain)
|
||||
this.changes = 0
|
||||
}
|
||||
}
|
||||
|
||||
private async performBackup(brain?: any): Promise<void> {
|
||||
if (!brain) return
|
||||
// Create instant COW snapshot
|
||||
const snapshotName = `backup-${Date.now()}`
|
||||
await brain.fork(snapshotName)
|
||||
console.log(`Automatic snapshot created: ${snapshotName}`)
|
||||
}
|
||||
readonly name = 'backup'
|
||||
readonly timing = 'after' as const
|
||||
readonly operations = ['add', 'update', 'delete'] as const
|
||||
readonly priority = 5
|
||||
|
||||
private changes = 0
|
||||
private readonly backupThreshold = 100
|
||||
|
||||
async execute<T>(operation: string, params: any, context?: AugmentationContext): Promise<void> {
|
||||
this.changes++
|
||||
|
||||
if (this.changes >= this.backupThreshold) {
|
||||
await this.performBackup(context?.brain)
|
||||
this.changes = 0
|
||||
}
|
||||
}
|
||||
|
||||
private async performBackup(brain?: any): Promise<void> {
|
||||
if (!brain) return
|
||||
// Create instant COW snapshot
|
||||
const snapshotName = `backup-${Date.now()}`
|
||||
await brain.fork(snapshotName)
|
||||
console.log(`Automatic snapshot created: ${snapshotName}`)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Rate Limiting Augmentation
|
||||
```typescript
|
||||
class RateLimitAugmentation extends BaseAugmentation {
|
||||
readonly name = 'rate-limit'
|
||||
readonly timing = 'before' as const
|
||||
readonly operations = ['search', 'find'] as const
|
||||
readonly priority = 100 // High priority - run first
|
||||
|
||||
private requests = new Map<string, number[]>()
|
||||
private readonly limit = 100 // 100 requests
|
||||
private readonly window = 60000 // per minute
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<void> {
|
||||
const now = Date.now()
|
||||
const key = params.userId || 'anonymous'
|
||||
|
||||
// Get request timestamps
|
||||
const timestamps = this.requests.get(key) || []
|
||||
|
||||
// Remove old timestamps
|
||||
const recent = timestamps.filter(t => now - t < this.window)
|
||||
|
||||
// Check limit
|
||||
if (recent.length >= this.limit) {
|
||||
throw new Error('Rate limit exceeded')
|
||||
}
|
||||
|
||||
// Add current request
|
||||
recent.push(now)
|
||||
this.requests.set(key, recent)
|
||||
}
|
||||
readonly name = 'rate-limit'
|
||||
readonly timing = 'before' as const
|
||||
readonly operations = ['search', 'find'] as const
|
||||
readonly priority = 100 // High priority - run first
|
||||
|
||||
private requests = new Map<string, number[]>()
|
||||
private readonly limit = 100 // 100 requests
|
||||
private readonly window = 60000 // per minute
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<void> {
|
||||
const now = Date.now()
|
||||
const key = params.userId || 'anonymous'
|
||||
|
||||
// Get request timestamps
|
||||
const timestamps = this.requests.get(key) || []
|
||||
|
||||
// Remove old timestamps
|
||||
const recent = timestamps.filter(t => now - t < this.window)
|
||||
|
||||
// Check limit
|
||||
if (recent.length >= this.limit) {
|
||||
throw new Error('Rate limit exceeded')
|
||||
}
|
||||
|
||||
// Add current request
|
||||
recent.push(now)
|
||||
this.requests.set(key, recent)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Encryption Augmentation
|
||||
```typescript
|
||||
class EncryptionAugmentation extends BaseAugmentation {
|
||||
readonly name = 'encryption'
|
||||
readonly timing = 'both' as const
|
||||
readonly operations = ['add', 'get'] as const
|
||||
readonly priority = 90 // Run early
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<any> {
|
||||
if (operation === 'add') {
|
||||
// Encrypt before storing
|
||||
if (params.metadata?.sensitive) {
|
||||
params.content = await this.encrypt(params.content)
|
||||
params.encrypted = true
|
||||
}
|
||||
return params
|
||||
}
|
||||
readonly name = 'encryption'
|
||||
readonly timing = 'both' as const
|
||||
readonly operations = ['add', 'get'] as const
|
||||
readonly priority = 90 // Run early
|
||||
|
||||
if (operation === 'get' && params.result?.encrypted) {
|
||||
// Decrypt after retrieval
|
||||
params.result.content = await this.decrypt(params.result.content)
|
||||
delete params.result.encrypted
|
||||
return params.result
|
||||
}
|
||||
}
|
||||
async execute<T>(operation: string, params: any): Promise<any> {
|
||||
if (operation === 'add') {
|
||||
// Encrypt before storing
|
||||
if (params.metadata?.sensitive) {
|
||||
params.content = await this.encrypt(params.content)
|
||||
params.encrypted = true
|
||||
}
|
||||
return params
|
||||
}
|
||||
|
||||
if (operation === 'get' && params.result?.encrypted) {
|
||||
// Decrypt after retrieval
|
||||
params.result.content = await this.decrypt(params.result.content)
|
||||
delete params.result.encrypted
|
||||
return params.result
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -355,26 +355,26 @@ import { Brainy } from '@soulcraft/brainy'
|
|||
import { MyAugmentation } from './my-augmentation'
|
||||
|
||||
describe('MyAugmentation', () => {
|
||||
it('should hook into addNoun', async () => {
|
||||
const brain = new Brainy({ storage: 'memory' })
|
||||
const aug = new MyAugmentation()
|
||||
|
||||
// Spy on the execute method
|
||||
const executeSpy = vi.spyOn(aug, 'execute')
|
||||
|
||||
brain.augmentations.register(aug)
|
||||
await brain.init()
|
||||
|
||||
// Trigger the augmentation
|
||||
await brain.add('test data')
|
||||
it('should hook into addNoun', async () => {
|
||||
const brain = new Brainy({ storage: 'memory' })
|
||||
const aug = new MyAugmentation()
|
||||
|
||||
// Verify it was called
|
||||
expect(executeSpy).toHaveBeenCalledWith(
|
||||
'add',
|
||||
expect.objectContaining({ content: 'test data' }),
|
||||
expect.any(Object)
|
||||
)
|
||||
})
|
||||
// Spy on the execute method
|
||||
const executeSpy = vi.spyOn(aug, 'execute')
|
||||
|
||||
brain.augmentations.register(aug)
|
||||
await brain.init()
|
||||
|
||||
// Trigger the augmentation
|
||||
await brain.add('test data')
|
||||
|
||||
// Verify it was called
|
||||
expect(executeSpy).toHaveBeenCalledWith(
|
||||
'add',
|
||||
expect.objectContaining({ content: 'test data' }),
|
||||
expect.any(Object)
|
||||
)
|
||||
})
|
||||
})
|
||||
```
|
||||
|
||||
|
|
@ -391,61 +391,61 @@ describe('MyAugmentation', () => {
|
|||
```typescript
|
||||
// Priority guidelines
|
||||
100: Critical (auth, rate limiting)
|
||||
50: Important (validation, transformation)
|
||||
10: Normal (logging, metrics)
|
||||
1: Optional (debugging, tracing)
|
||||
50: Important (validation, transformation)
|
||||
10: Normal (logging, metrics)
|
||||
1: Optional (debugging, tracing)
|
||||
```
|
||||
|
||||
### 3. Handle Errors Gracefully
|
||||
```typescript
|
||||
async execute<T>(operation: string, params: any): Promise<void> {
|
||||
try {
|
||||
await this.riskyOperation()
|
||||
} catch (error) {
|
||||
// Log but don't break the main operation
|
||||
console.error(`Augmentation error in ${this.name}:`, error)
|
||||
// Optionally report to monitoring
|
||||
this.reportError(error)
|
||||
}
|
||||
try {
|
||||
await this.riskyOperation()
|
||||
} catch (error) {
|
||||
// Log but don't break the main operation
|
||||
console.error(`Augmentation error in ${this.name}:`, error)
|
||||
// Optionally report to monitoring
|
||||
this.reportError(error)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Be Performance Conscious
|
||||
```typescript
|
||||
class CachedAugmentation extends BaseAugmentation {
|
||||
private cache = new Map<string, any>()
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<any> {
|
||||
const key = this.getCacheKey(params)
|
||||
|
||||
// Check cache first
|
||||
if (this.cache.has(key)) {
|
||||
return this.cache.get(key)
|
||||
}
|
||||
|
||||
// Expensive operation
|
||||
const result = await this.expensiveOperation(params)
|
||||
this.cache.set(key, result)
|
||||
|
||||
return result
|
||||
}
|
||||
private cache = new Map<string, any>()
|
||||
|
||||
async execute<T>(operation: string, params: any): Promise<any> {
|
||||
const key = this.getCacheKey(params)
|
||||
|
||||
// Check cache first
|
||||
if (this.cache.has(key)) {
|
||||
return this.cache.get(key)
|
||||
}
|
||||
|
||||
// Expensive operation
|
||||
const result = await this.expensiveOperation(params)
|
||||
this.cache.set(key, result)
|
||||
|
||||
return result
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Clean Up Resources
|
||||
```typescript
|
||||
protected async onShutdown(): Promise<void> {
|
||||
// Close connections
|
||||
await this.connection?.close()
|
||||
|
||||
// Clear intervals
|
||||
clearInterval(this.interval)
|
||||
|
||||
// Flush buffers
|
||||
await this.flush()
|
||||
|
||||
// Clear caches
|
||||
this.cache.clear()
|
||||
// Close connections
|
||||
await this.connection?.close()
|
||||
|
||||
// Clear intervals
|
||||
clearInterval(this.interval)
|
||||
|
||||
// Flush buffers
|
||||
await this.flush()
|
||||
|
||||
// Clear caches
|
||||
this.cache.clear()
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -457,10 +457,10 @@ protected async onShutdown(): Promise<void> {
|
|||
```
|
||||
my-augmentation/
|
||||
├── src/
|
||||
│ └── index.ts # Your augmentation
|
||||
├── dist/ # Built output
|
||||
│ └── index.ts # Your augmentation
|
||||
├── dist/ # Built output
|
||||
├── tests/
|
||||
│ └── augmentation.test.ts
|
||||
│ └── augmentation.test.ts
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
└── README.md
|
||||
|
|
@ -469,21 +469,21 @@ my-augmentation/
|
|||
### package.json
|
||||
```json
|
||||
{
|
||||
"name": "@mycompany/brainy-custom-augmentation",
|
||||
"version": "1.0.0",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
"keywords": ["brainy-augmentation"],
|
||||
"peerDependencies": {
|
||||
"@soulcraft/brainy": ">=2.0.0"
|
||||
},
|
||||
"brainy": {
|
||||
"type": "augmentation",
|
||||
"class": "CustomAugmentation",
|
||||
"timing": "after",
|
||||
"operations": ["add"],
|
||||
"priority": 10
|
||||
}
|
||||
"name": "@mycompany/brainy-custom-augmentation",
|
||||
"version": "1.0.0",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
"keywords": ["brainy-augmentation"],
|
||||
"peerDependencies": {
|
||||
"@soulcraft/brainy": ">=2.0.0"
|
||||
},
|
||||
"brainy": {
|
||||
"type": "augmentation",
|
||||
"class": "CustomAugmentation",
|
||||
"timing": "after",
|
||||
"operations": ["add"],
|
||||
"priority": 10
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -492,7 +492,7 @@ my-augmentation/
|
|||
# Coming in 2.1+
|
||||
npm run build
|
||||
npm test
|
||||
brainy publish # Publishes to brain-cloud registry
|
||||
brainy publish # Publishes to brain-cloud registry
|
||||
```
|
||||
|
||||
---
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue