feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
/**
|
|
|
|
|
* Brainy Plugin System
|
|
|
|
|
*
|
|
|
|
|
* Simple plugin architecture for two use cases:
|
2026-02-01 08:22:07 -08:00
|
|
|
* 1. Native acceleration (@soulcraft/cortex)
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
* 2. Custom storage adapters (e.g., Redis, DynamoDB, custom backends)
|
|
|
|
|
*
|
|
|
|
|
* Plugins are auto-detected by package name or registered manually.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
import type { StorageAdapter } from './coreTypes.js'
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Plugin interface — all brainy plugins must implement this.
|
|
|
|
|
*/
|
|
|
|
|
export interface BrainyPlugin {
|
|
|
|
|
/** Unique plugin name (typically the npm package name) */
|
|
|
|
|
name: string
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Called by brainy during init() to activate the plugin.
|
|
|
|
|
* Return true if activation succeeded, false to skip.
|
|
|
|
|
*/
|
|
|
|
|
activate(context: BrainyPluginContext): Promise<boolean>
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Called when brainy.close() is invoked. Optional cleanup.
|
|
|
|
|
*/
|
|
|
|
|
deactivate?(): Promise<void>
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Context passed to plugins during activation.
|
|
|
|
|
*/
|
|
|
|
|
export interface BrainyPluginContext {
|
|
|
|
|
/**
|
|
|
|
|
* Register a provider for a named subsystem.
|
|
|
|
|
*
|
|
|
|
|
* Well-known provider keys (used by cortex):
|
|
|
|
|
* - 'metadataIndex' — MetadataIndexManager replacement
|
|
|
|
|
* - 'graphIndex' — GraphAdjacencyIndex replacement
|
|
|
|
|
* - 'entityIdMapper' — EntityIdMapper replacement
|
|
|
|
|
* - 'cache' — UnifiedCache replacement
|
|
|
|
|
* - 'hnsw' — HNSWIndex replacement
|
|
|
|
|
* - 'roaring' — RoaringBitmap32 replacement
|
2026-02-01 13:03:15 -08:00
|
|
|
* - 'embeddings' — Embedding engine replacement (single text)
|
|
|
|
|
* - 'embedBatch' — Batch embedding engine (texts[] → vectors[])
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
* - 'distance' — Distance function overrides
|
|
|
|
|
* - 'msgpack' — Msgpack encode/decode
|
feat: add aggregation engine with incremental SUM/COUNT/AVG/MIN/MAX, GROUP BY, and time windows
Add a write-time incremental aggregation engine that maintains running
totals on every add/update/delete for O(1) read performance. Integrates
into brain.find({ aggregate }) for a unified query API.
Core features:
- AggregationIndex with defineAggregate()/removeAggregate() API
- Five aggregation operations: SUM, COUNT, AVG, MIN, MAX
- GROUP BY with multiple dimensions including time windows
- Time window bucketing: hour, day, week, month, quarter, year, custom
- Materialization of results as NounType.Measurement entities
- Debounced persistence of definitions and state to storage
- Definition change detection via FNV-1a hashing with auto-rebuild
- Infinite loop prevention for materialized entities
- 'aggregation' plugin provider key for native acceleration
- Lazy initialization (created on first defineAggregate() call)
- 73 tests (unit + integration) covering all functionality
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 16:57:53 -08:00
|
|
|
* - 'aggregation' — AggregationIndex replacement (incremental aggregates)
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
*
|
|
|
|
|
* Storage adapter keys:
|
|
|
|
|
* - 'storage:<name>' — Custom storage adapter factory
|
|
|
|
|
*/
|
|
|
|
|
registerProvider(key: string, implementation: unknown): void
|
|
|
|
|
|
|
|
|
|
/** Brainy version for compatibility checks */
|
|
|
|
|
readonly version: string
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Storage adapter factory — plugins register these to provide
|
|
|
|
|
* new storage backends that users reference by name.
|
|
|
|
|
*
|
|
|
|
|
* Example: A Redis plugin registers 'storage:redis', then users
|
|
|
|
|
* can use `new Brainy({ storage: 'redis', redis: { host: '...' } })`
|
|
|
|
|
*/
|
|
|
|
|
export interface StorageAdapterFactory {
|
|
|
|
|
create(config: Record<string, unknown>): StorageAdapter | Promise<StorageAdapter>
|
|
|
|
|
name: string
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Plugin registry — manages plugin lifecycle and provider resolution.
|
|
|
|
|
*/
|
|
|
|
|
export class PluginRegistry {
|
|
|
|
|
private plugins: Map<string, BrainyPlugin> = new Map()
|
|
|
|
|
private providers: Map<string, unknown> = new Map()
|
|
|
|
|
private activated: Set<string> = new Set()
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Register a plugin manually.
|
|
|
|
|
*/
|
|
|
|
|
register(plugin: BrainyPlugin): void {
|
|
|
|
|
this.plugins.set(plugin.name, plugin)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Activate all registered plugins.
|
|
|
|
|
*/
|
|
|
|
|
async activateAll(context: BrainyPluginContext): Promise<string[]> {
|
|
|
|
|
const activated: string[] = []
|
|
|
|
|
|
|
|
|
|
for (const [name, plugin] of this.plugins) {
|
|
|
|
|
if (this.activated.has(name)) continue
|
|
|
|
|
|
|
|
|
|
try {
|
|
|
|
|
const success = await plugin.activate(context)
|
|
|
|
|
if (success) {
|
|
|
|
|
this.activated.add(name)
|
|
|
|
|
activated.push(name)
|
|
|
|
|
}
|
|
|
|
|
} catch (error) {
|
|
|
|
|
console.warn(`[brainy] Plugin ${name} failed to activate:`, error)
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return activated
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Deactivate all plugins (called during close()).
|
|
|
|
|
*/
|
|
|
|
|
async deactivateAll(): Promise<void> {
|
|
|
|
|
for (const [name, plugin] of this.plugins) {
|
|
|
|
|
if (!this.activated.has(name)) continue
|
|
|
|
|
try {
|
|
|
|
|
await plugin.deactivate?.()
|
|
|
|
|
this.activated.delete(name)
|
|
|
|
|
} catch {
|
|
|
|
|
// Non-fatal
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Get a registered provider by key.
|
|
|
|
|
*/
|
|
|
|
|
getProvider<T = unknown>(key: string): T | undefined {
|
|
|
|
|
return this.providers.get(key) as T | undefined
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Check if a provider is registered.
|
|
|
|
|
*/
|
|
|
|
|
hasProvider(key: string): boolean {
|
|
|
|
|
return this.providers.has(key)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Register a provider (called by plugins via BrainyPluginContext).
|
|
|
|
|
*/
|
|
|
|
|
registerProvider(key: string, implementation: unknown): void {
|
|
|
|
|
this.providers.set(key, implementation)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Get a storage adapter factory by name.
|
|
|
|
|
*/
|
|
|
|
|
getStorageFactory(name: string): StorageAdapterFactory | undefined {
|
|
|
|
|
return this.providers.get(`storage:${name}`) as StorageAdapterFactory | undefined
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Get active plugin names */
|
|
|
|
|
getActivePlugins(): string[] {
|
|
|
|
|
return [...this.activated]
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Check if any plugins are active */
|
|
|
|
|
hasActivePlugins(): boolean {
|
|
|
|
|
return this.activated.size > 0
|
|
|
|
|
}
|
|
|
|
|
}
|