soulcraft/brainy
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

Add augmentation pipeline and memory augmentation documentation

Browse source 
Introduced an augmentation event pipeline supporting multiple execution modes (e.g., SEQUENTIAL, PARALLEL). Added new Memory Augmentation type for versatile data storage/retrieval (e.g., fileSystem, in-memory, Firestore). Updated `README.md` with pipeline usage, examples, and augmentations reorganization. Included new demo files for augmentation examples.
This commit is contained in:
David Snelling 2025-05-27 10:08:01 -07:00
parent de0531530f
commit d2ddbd2613
5 changed files with 1243 additions and 185 deletions
Download patch file Download diff file Expand all files Collapse all files

201
README.md
View file

@ -13,6 +13,7 @@ A vector database that runs in a browser or Node.js and utilizes Origin Private
- **TypeScript support**: Fully typed API with generics for metadata types
- **Multiple distance functions**: Supports cosine, Euclidean, Manhattan, and dot product distance metrics
- **Augmentation system**: Extensible architecture for adding specialized capabilities
- **Memory augmentation**: Store and retrieve data in different formats (fileSystem, in-memory, firestore)
- **Graph data model**: Structured representation of entities and relationships
## Installation
@ -197,6 +198,45 @@ interface IWebSocketSupport {
Brainy supports several specialized augmentation types:
#### Sense Augmentations
For processing raw, unstructured data:
```typescript
interface ISenseAugmentation extends IAugmentation {
processRawData(rawData: Buffer | string, dataType: string): AugmentationResponse<{
nouns: string[];
verbs: string[];
}>;
listenToFeed(
feedUrl: string,
callback: DataCallback<{ nouns: string[]; verbs: string[] }>
): Promise<void>;
}
```
#### Conduit Augmentations
For establishing data exchange channels:
```typescript
interface IConduitAugmentation extends IAugmentation {
establishConnection(
targetSystemId: string,
config: Record<string, unknown>
): AugmentationResponse<WebSocketConnection>;
readData(
query: Record<string, unknown>,
options?: Record<string, unknown>
): AugmentationResponse<unknown>;
writeData(
data: Record<string, unknown>,
options?: Record<string, unknown>
): AugmentationResponse<unknown>;
monitorStream(streamId: string, callback: DataCallback<unknown>): Promise<void>;
}
```
#### Cognition Augmentations
For reasoning, inference, and logical operations:
@ -212,20 +252,34 @@ interface ICognitionAugmentation extends IAugmentation {
}
```
#### Sense Augmentations
#### Memory Augmentations
For processing raw, unstructured data:
For storing data in different formats (e.g., fileSystem, in-memory, or firestore):
```typescript
interface ISenseAugmentation extends IAugmentation {
processRawData(rawData: Buffer | string, dataType: string): AugmentationResponse<{
nouns: string[];
verbs: string[];
}>;
listenToFeed(
feedUrl: string,
callback: DataCallback<{ nouns: string[]; verbs: string[] }>
): Promise<void>;
interface IMemoryAugmentation extends IAugmentation {
storeData(
key: string,
data: unknown,
options?: Record<string, unknown>
): AugmentationResponse<boolean>;
retrieveData(
key: string,
options?: Record<string, unknown>
): AugmentationResponse<unknown>;
updateData(
key: string,
data: unknown,
options?: Record<string, unknown>
): AugmentationResponse<boolean>;
deleteData(
key: string,
options?: Record<string, unknown>
): AugmentationResponse<boolean>;
listDataKeys(
pattern?: string,
options?: Record<string, unknown>
): AugmentationResponse<string[]>;
}
```
@ -251,21 +305,6 @@ interface IPerceptionAugmentation extends IAugmentation {
}
```
#### Activation Augmentations
For triggering actions and generating outputs:
```typescript
interface IActivationAugmentation extends IAugmentation {
triggerAction(
actionName: string,
parameters?: Record<string, unknown>
): AugmentationResponse<unknown>;
generateOutput(knowledgeId: string, format: string): AugmentationResponse<string | Record<string, unknown>>;
interactExternal(systemId: string, payload: Record<string, unknown>): AugmentationResponse<unknown>;
}
```
#### Dialog Augmentations
For natural language understanding and generation:
@ -287,28 +326,105 @@ interface IDialogAugmentation extends IAugmentation {
}
```
#### Conduit Augmentations
#### Activation Augmentations
For establishing data exchange channels:
For triggering actions and generating outputs:
```typescript
interface IConduitAugmentation extends IAugmentation {
establishConnection(
targetSystemId: string,
config: Record<string, unknown>
): AugmentationResponse<WebSocketConnection>;
readData(
query: Record<string, unknown>,
options?: Record<string, unknown>
interface IActivationAugmentation extends IAugmentation {
triggerAction(
actionName: string,
parameters?: Record<string, unknown>
): AugmentationResponse<unknown>;
writeData(
data: Record<string, unknown>,
options?: Record<string, unknown>
): AugmentationResponse<unknown>;
monitorStream(streamId: string, callback: DataCallback<unknown>): Promise<void>;
generateOutput(knowledgeId: string, format: string): AugmentationResponse<string | Record<string, unknown>>;
interactExternal(systemId: string, payload: Record<string, unknown>): AugmentationResponse<unknown>;
}
```
### Augmentation Event Pipeline
Brainy provides an event pipeline that allows registering and executing multiple augmentations of each type. The pipeline supports different execution modes and provides a flexible way to manage augmentations.
#### Using the Pipeline
```typescript
import { augmentationPipeline, ExecutionMode } from '@soulcraft/brainy';
// Register augmentations
augmentationPipeline.register(mySenseAugmentation);
augmentationPipeline.register(myConduitAugmentation);
augmentationPipeline.register(myCognitionAugmentation);
// Initialize all registered augmentations
await augmentationPipeline.initialize();
// Execute a sense pipeline
const processingResults = await augmentationPipeline.executeSensePipeline(
'processRawData',
['Some raw text data', 'text'],
{ mode: ExecutionMode.SEQUENTIAL, stopOnError: true }
);
// Execute a conduit pipeline
const connectionResults = await augmentationPipeline.executeConduitPipeline(
'establishConnection',
['external-system', { apiKey: 'your-api-key' }]
);
// Execute a cognition pipeline
const reasoningResults = await augmentationPipeline.executeCognitionPipeline(
'reason',
['What is the capital of France?', { additionalContext: 'geography' }],
{ mode: ExecutionMode.PARALLEL }
);
// Execute a memory pipeline
const storeResults = await augmentationPipeline.executeMemoryPipeline(
'storeData',
['user123', { name: 'John Doe', email: 'john@example.com' }]
);
const retrieveResults = await augmentationPipeline.executeMemoryPipeline(
'retrieveData',
['user123']
);
// Shut down all registered augmentations
await augmentationPipeline.shutDown();
```
#### Execution Modes
The pipeline supports several execution modes:
- `ExecutionMode.SEQUENTIAL`: Execute augmentations one after another (default)
- `ExecutionMode.PARALLEL`: Execute all augmentations simultaneously
- `ExecutionMode.FIRST_SUCCESS`: Execute augmentations until one succeeds
- `ExecutionMode.FIRST_RESULT`: Execute augmentations until one returns a result
#### Pipeline Options
You can configure the pipeline execution with options:
```typescript
interface PipelineOptions {
mode?: ExecutionMode; // Execution mode (default: SEQUENTIAL)
timeout?: number; // Timeout in milliseconds (default: 30000)
stopOnError?: boolean; // Whether to stop on error (default: false)
}
```
#### Creating a Custom Pipeline
You can create a custom pipeline instance if needed:
```typescript
import { AugmentationPipeline } from '@soulcraft/brainy';
const myPipeline = new AugmentationPipeline();
myPipeline.register(myCustomAugmentation);
```
## Graph Data Model
Brainy uses a graph-based data model to represent entities and relationships. This model consists of nouns (nodes) and verbs (edges).
@ -425,6 +541,9 @@ The repository also includes TypeScript examples for Node.js:
- `src/examples/basicUsage.ts`: Demonstrates basic vector operations
- `src/examples/customStorage.ts`: Shows how to use a custom storage adapter
- `src/examples/augmentationPipeline.ts`: Demonstrates the augmentation pipeline
- `src/examples/webSocketAugmentation.ts`: Shows how to create WebSocket-supporting augmentations
- `src/examples/memoryAugmentation.ts`: Demonstrates memory augmentations for different storage formats
## How It Works

574
src/augmentationPipeline.ts Normal file
View file

@ -0,0 +1,574 @@
/**
* Augmentation Event Pipeline
*
* This module provides a pipeline for managing and executing multiple augmentations
* of each type. It allows registering multiple augmentations and executing them
* in sequence or in parallel.
*/
import {
BrainyAugmentations,
IAugmentation,
IWebSocketSupport,
AugmentationResponse
} from './types/augmentations.js'
/**
* Type definitions for the augmentation registry
*/
type AugmentationRegistry = {
sense: BrainyAugmentations.ISenseAugmentation[];
conduit: BrainyAugmentations.IConduitAugmentation[];
cognition: BrainyAugmentations.ICognitionAugmentation[];
memory: BrainyAugmentations.IMemoryAugmentation[];
perception: BrainyAugmentations.IPerceptionAugmentation[];
dialog: BrainyAugmentations.IDialogAugmentation[];
activation: BrainyAugmentations.IActivationAugmentation[];
webSocket: IWebSocketSupport[];
}
/**
* Execution mode for the pipeline
*/
export enum ExecutionMode {
SEQUENTIAL = 'sequential',
PARALLEL = 'parallel',
FIRST_SUCCESS = 'firstSuccess',
FIRST_RESULT = 'firstResult'
}
/**
* Options for pipeline execution
*/
export interface PipelineOptions {
mode?: ExecutionMode;
timeout?: number;
stopOnError?: boolean;
}
/**
* Default pipeline options
*/
const DEFAULT_PIPELINE_OPTIONS: PipelineOptions = {
mode: ExecutionMode.SEQUENTIAL,
timeout: 30000,
stopOnError: false
}
/**
* AugmentationPipeline class
*
* Manages multiple augmentations of each type and provides methods to execute them.
*/
export class AugmentationPipeline {
private registry: AugmentationRegistry = {
sense: [],
conduit: [],
cognition: [],
memory: [],
perception: [],
dialog: [],
activation: [],
webSocket: []
}
/**
* Register an augmentation with the pipeline
*
* @param augmentation The augmentation to register
* @returns The pipeline instance for chaining
*/
public register<T extends IAugmentation>(augmentation: T): AugmentationPipeline {
let registered = false
// Check for specific augmentation types
if (this.isAugmentationType<BrainyAugmentations.ISenseAugmentation>(
augmentation,
'processRawData',
'listenToFeed'
)) {
this.registry.sense.push(augmentation)
registered = true
} else if (this.isAugmentationType<BrainyAugmentations.IConduitAugmentation>(
augmentation,
'establishConnection',
'readData',
'writeData',
'monitorStream'
)) {
this.registry.conduit.push(augmentation)
registered = true
} else if (this.isAugmentationType<BrainyAugmentations.ICognitionAugmentation>(
augmentation,
'reason',
'infer',
'executeLogic'
)) {
this.registry.cognition.push(augmentation)
registered = true
} else if (this.isAugmentationType<BrainyAugmentations.IMemoryAugmentation>(
augmentation,
'storeData',
'retrieveData',
'updateData',
'deleteData',
'listDataKeys'
)) {
this.registry.memory.push(augmentation)
registered = true
} else if (this.isAugmentationType<BrainyAugmentations.IPerceptionAugmentation>(
augmentation,
'interpret',
'organize',
'generateVisualization'
)) {
this.registry.perception.push(augmentation)
registered = true
} else if (this.isAugmentationType<BrainyAugmentations.IDialogAugmentation>(
augmentation,
'processUserInput',
'generateResponse',
'manageContext'
)) {
this.registry.dialog.push(augmentation)
registered = true
} else if (this.isAugmentationType<BrainyAugmentations.IActivationAugmentation>(
augmentation,
'triggerAction',
'generateOutput',
'interactExternal'
)) {
this.registry.activation.push(augmentation)
registered = true
}
// Check if the augmentation supports WebSocket
if (this.isAugmentationType<IWebSocketSupport>(
augmentation,
'connectWebSocket',
'sendWebSocketMessage',
'onWebSocketMessage',
'closeWebSocket'
)) {
this.registry.webSocket.push(augmentation as IWebSocketSupport)
registered = true
}
// If the augmentation wasn't registered as any known type, throw an error
if (!registered) {
throw new Error(`Unknown augmentation type: ${augmentation.name}`)
}
return this
}
/**
* Unregister an augmentation from the pipeline
*
* @param augmentationName The name of the augmentation to unregister
* @returns The pipeline instance for chaining
*/
public unregister(augmentationName: string): AugmentationPipeline {
let found = false
// Remove from all registries
for (const type in this.registry) {
const typedRegistry = this.registry[type as keyof AugmentationRegistry]
const index = typedRegistry.findIndex(aug => aug.name === augmentationName)
if (index !== -1) {
typedRegistry.splice(index, 1)
found = true
}
}
return this
}
/**
* Initialize all registered augmentations
*
* @returns A promise that resolves when all augmentations are initialized
*/
public async initialize(): Promise<void> {
const allAugmentations = this.getAllAugmentations()
await Promise.all(
allAugmentations.map(augmentation =>
augmentation.initialize().catch(error => {
console.error(`Failed to initialize augmentation ${augmentation.name}:`, error)
})
)
)
}
/**
* Shut down all registered augmentations
*
* @returns A promise that resolves when all augmentations are shut down
*/
public async shutDown(): Promise<void> {
const allAugmentations = this.getAllAugmentations()
await Promise.all(
allAugmentations.map(augmentation =>
augmentation.shutDown().catch(error => {
console.error(`Failed to shut down augmentation ${augmentation.name}:`, error)
})
)
)
}
/**
* Execute a sense pipeline
*
* @param method The method to execute on each sense augmentation
* @param args The arguments to pass to the method
* @param options The pipeline execution options
* @returns A promise that resolves with the results from all augmentations
*/
public async executeSensePipeline<
M extends keyof BrainyAugmentations.ISenseAugmentation & string,
R extends BrainyAugmentations.ISenseAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never
>(
method: M & (BrainyAugmentations.ISenseAugmentation[M] extends (...args: any[]) => any ? M : never),
args: Parameters<Extract<BrainyAugmentations.ISenseAugmentation[M], (...args: any[]) => any>>,
options: PipelineOptions = {}
): Promise<AugmentationResponse<R>[]> {
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options }
return this.executeTypedPipeline<BrainyAugmentations.ISenseAugmentation, M, R>(
this.registry.sense,
method,
args,
opts
)
}
/**
* Execute a conduit pipeline
*
* @param method The method to execute on each conduit augmentation
* @param args The arguments to pass to the method
* @param options The pipeline execution options
* @returns A promise that resolves with the results from all augmentations
*/
public async executeConduitPipeline<
M extends keyof BrainyAugmentations.IConduitAugmentation & string,
R extends BrainyAugmentations.IConduitAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never
>(
method: M & (BrainyAugmentations.IConduitAugmentation[M] extends (...args: any[]) => any ? M : never),
args: Parameters<Extract<BrainyAugmentations.IConduitAugmentation[M], (...args: any[]) => any>>,
options: PipelineOptions = {}
): Promise<AugmentationResponse<R>[]> {
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options }
return this.executeTypedPipeline<BrainyAugmentations.IConduitAugmentation, M, R>(
this.registry.conduit,
method,
args,
opts
)
}
/**
* Execute a cognition pipeline
*
* @param method The method to execute on each cognition augmentation
* @param args The arguments to pass to the method
* @param options The pipeline execution options
* @returns A promise that resolves with the results from all augmentations
*/
public async executeCognitionPipeline<
M extends keyof BrainyAugmentations.ICognitionAugmentation & string,
R extends BrainyAugmentations.ICognitionAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never
>(
method: M & (BrainyAugmentations.ICognitionAugmentation[M] extends (...args: any[]) => any ? M : never),
args: Parameters<Extract<BrainyAugmentations.ICognitionAugmentation[M], (...args: any[]) => any>>,
options: PipelineOptions = {}
): Promise<AugmentationResponse<R>[]> {
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options }
return this.executeTypedPipeline<BrainyAugmentations.ICognitionAugmentation, M, R>(
this.registry.cognition,
method,
args,
opts
)
}
/**
* Execute a memory pipeline
*
* @param method The method to execute on each memory augmentation
* @param args The arguments to pass to the method
* @param options The pipeline execution options
* @returns A promise that resolves with the results from all augmentations
*/
public async executeMemoryPipeline<
M extends keyof BrainyAugmentations.IMemoryAugmentation & string,
R extends BrainyAugmentations.IMemoryAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never
>(
method: M & (BrainyAugmentations.IMemoryAugmentation[M] extends (...args: any[]) => any ? M : never),
args: Parameters<Extract<BrainyAugmentations.IMemoryAugmentation[M], (...args: any[]) => any>>,
options: PipelineOptions = {}
): Promise<AugmentationResponse<R>[]> {
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options }
return this.executeTypedPipeline<BrainyAugmentations.IMemoryAugmentation, M, R>(
this.registry.memory,
method,
args,
opts
)
}
/**
* Execute a perception pipeline
*
* @param method The method to execute on each perception augmentation
* @param args The arguments to pass to the method
* @param options The pipeline execution options
* @returns A promise that resolves with the results from all augmentations
*/
public async executePerceptionPipeline<
M extends keyof BrainyAugmentations.IPerceptionAugmentation & string,
R extends BrainyAugmentations.IPerceptionAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never
>(
method: M & (BrainyAugmentations.IPerceptionAugmentation[M] extends (...args: any[]) => any ? M : never),
args: Parameters<Extract<BrainyAugmentations.IPerceptionAugmentation[M], (...args: any[]) => any>>,
options: PipelineOptions = {}
): Promise<AugmentationResponse<R>[]> {
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options }
return this.executeTypedPipeline<BrainyAugmentations.IPerceptionAugmentation, M, R>(
this.registry.perception,
method,
args,
opts
)
}
/**
* Execute a dialog pipeline
*
* @param method The method to execute on each dialog augmentation
* @param args The arguments to pass to the method
* @param options The pipeline execution options
* @returns A promise that resolves with the results from all augmentations
*/
public async executeDialogPipeline<
M extends keyof BrainyAugmentations.IDialogAugmentation & string,
R extends BrainyAugmentations.IDialogAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never
>(
method: M & (BrainyAugmentations.IDialogAugmentation[M] extends (...args: any[]) => any ? M : never),
args: Parameters<Extract<BrainyAugmentations.IDialogAugmentation[M], (...args: any[]) => any>>,
options: PipelineOptions = {}
): Promise<AugmentationResponse<R>[]> {
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options }
return this.executeTypedPipeline<BrainyAugmentations.IDialogAugmentation, M, R>(
this.registry.dialog,
method,
args,
opts
)
}
/**
* Execute an activation pipeline
*
* @param method The method to execute on each activation augmentation
* @param args The arguments to pass to the method
* @param options The pipeline execution options
* @returns A promise that resolves with the results from all augmentations
*/
public async executeActivationPipeline<
M extends keyof BrainyAugmentations.IActivationAugmentation & string,
R extends BrainyAugmentations.IActivationAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never
>(
method: M & (BrainyAugmentations.IActivationAugmentation[M] extends (...args: any[]) => any ? M : never),
args: Parameters<Extract<BrainyAugmentations.IActivationAugmentation[M], (...args: any[]) => any>>,
options: PipelineOptions = {}
): Promise<AugmentationResponse<R>[]> {
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options }
return this.executeTypedPipeline<BrainyAugmentations.IActivationAugmentation, M, R>(
this.registry.activation,
method,
args,
opts
)
}
/**
* Get all registered augmentations
*
* @returns An array of all registered augmentations
*/
private getAllAugmentations(): IAugmentation[] {
// Create a Set to avoid duplicates (an augmentation might be in multiple registries)
const allAugmentations = new Set<IAugmentation>([
...this.registry.sense,
...this.registry.conduit,
...this.registry.cognition,
...this.registry.memory,
...this.registry.perception,
...this.registry.dialog,
...this.registry.activation,
...this.registry.webSocket
])
// Convert back to array
return Array.from(allAugmentations)
}
/**
* Get all WebSocket-supporting augmentations
*
* @returns An array of all augmentations that support WebSocket connections
*/
public getWebSocketAugmentations(): IWebSocketSupport[] {
return [...this.registry.webSocket]
}
/**
* Check if an augmentation is of a specific type
*
* @param augmentation The augmentation to check
* @param methods The methods that should be present on the augmentation
* @returns True if the augmentation is of the specified type
*/
private isAugmentationType<T extends IAugmentation>(
augmentation: IAugmentation,
...methods: (keyof T)[]
): augmentation is T {
// First check that the augmentation has all the required base methods
const baseMethodsExist = [
'initialize',
'shutDown',
'getStatus'
].every(method => typeof (augmentation as any)[method] === 'function')
if (!baseMethodsExist) {
return false
}
// Then check that it has all the specific methods for this type
return methods.every(method => typeof (augmentation as any)[method] === 'function')
}
/**
* Execute a pipeline for a specific augmentation type
*
* @param augmentations The augmentations to execute
* @param method The method to execute on each augmentation
* @param args The arguments to pass to the method
* @param options The pipeline execution options
* @returns A promise that resolves with the results from all augmentations
*/
private async executeTypedPipeline<
T extends IAugmentation,
M extends keyof T & string,
R extends T[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never
>(
augmentations: T[],
method: M & (T[M] extends (...args: any[]) => any ? M : never),
args: Parameters<Extract<T[M], (...args: any[]) => any>>,
options: PipelineOptions
): Promise<Promise<{
success: boolean
data: R
error?: string
}>[]> {
if (augmentations.length === 0) {
return []
}
// Create a function to execute the method on an augmentation
const executeMethod = async (augmentation: T): Promise<{
success: boolean
data: R
error?: string
}> => {
try {
// Create a timeout promise if a timeout is specified
const timeoutPromise = options.timeout
? new Promise<{
success: boolean
data: R
error?: string
}>((_, reject) => {
setTimeout(() => {
reject(new Error(`Timeout executing ${String(method)} on ${augmentation.name}`))
}, options.timeout)
})
: null
// Execute the method on the augmentation
const methodPromise = (augmentation[method] as Function)(...args) as AugmentationResponse<R>
// Race the method promise against the timeout promise if a timeout is specified
const result = timeoutPromise
? await Promise.race([methodPromise, timeoutPromise])
: await methodPromise
return result
} catch (error) {
console.error(`Error executing ${String(method)} on ${augmentation.name}:`, error)
return {
success: false,
data: null as unknown as R,
error: error instanceof Error ? error.message : String(error)
}
}
}
// Execute the pipeline based on the specified mode
switch (options.mode) {
case ExecutionMode.PARALLEL:
// Execute all augmentations in parallel
return augmentations.map(executeMethod)
case ExecutionMode.FIRST_SUCCESS:
// Execute augmentations sequentially until one succeeds
for (const augmentation of augmentations) {
const resultPromise = executeMethod(augmentation)
const result = await resultPromise
if (result.success) {
return [resultPromise]
}
}
return []
case ExecutionMode.FIRST_RESULT:
// Execute augmentations sequentially until one returns a result
for (const augmentation of augmentations) {
const resultPromise = executeMethod(augmentation)
const result = await resultPromise
if (result.success && result.data) {
return [resultPromise]
}
}
return []
case ExecutionMode.SEQUENTIAL:
default:
// Execute augmentations sequentially
const results: Promise<{
success: boolean
data: R
error?: string
}>[] = []
for (const augmentation of augmentations) {
const resultPromise = executeMethod(augmentation)
results.push(resultPromise)
// Check if we need to stop on error
if (options.stopOnError) {
const result = await resultPromise
if (!result.success) {
break
}
}
}
return results
}
}
}
// Create and export a default instance of the pipeline
export const augmentationPipeline = new AugmentationPipeline()

266
src/examples/augmentationPipeline.ts Normal file
View file

@ -0,0 +1,266 @@
/**
* Augmentation Pipeline Example
*
* This example demonstrates how to use the augmentation pipeline to register
* and execute multiple augmentations of each type.
*/
import {
augmentationPipeline,
ExecutionMode,
PipelineOptions,
IAugmentation,
AugmentationResponse,
BrainyAugmentations
} from '../index.js'
/**
* Example Cognition Augmentation
*/
class SimpleCognitionAugmentation implements BrainyAugmentations.ICognitionAugmentation {
readonly name = 'simple-cognition'
readonly description = 'A simple cognition augmentation for demonstration'
async initialize(): Promise<void> {
console.log('Initializing SimpleCognitionAugmentation')
}
async shutDown(): Promise<void> {
console.log('Shutting down SimpleCognitionAugmentation')
}
async getStatus(): Promise<'active' | 'inactive' | 'error'> {
return 'active'
}
async reason(
query: string,
context?: Record<string, unknown>
): Promise<AugmentationResponse<{ inference: string; confidence: number }>> {
console.log(`SimpleCognitionAugmentation reasoning about: ${query}`)
console.log('Context:', context)
return {
success: true,
data: {
inference: `Simple inference about: ${query}`,
confidence: 0.7
}
}
}
async infer(
dataSubset: Record<string, unknown>
): Promise<AugmentationResponse<Record<string, unknown>>> {
return {
success: true,
data: {
result: `Inferred from data: ${JSON.stringify(dataSubset)}`
}
}
}
async executeLogic(
ruleId: string,
input: Record<string, unknown>
): Promise<AugmentationResponse<boolean>> {
return {
success: true,
data: true
}
}
}
/**
* Another Example Cognition Augmentation
*/
class AdvancedCognitionAugmentation implements BrainyAugmentations.ICognitionAugmentation {
readonly name = 'advanced-cognition'
readonly description = 'A more advanced cognition augmentation for demonstration'
async initialize(): Promise<void> {
console.log('Initializing AdvancedCognitionAugmentation')
}
async shutDown(): Promise<void> {
console.log('Shutting down AdvancedCognitionAugmentation')
}
async getStatus(): Promise<'active' | 'inactive' | 'error'> {
return 'active'
}
async reason(
query: string,
context?: Record<string, unknown>
): Promise<AugmentationResponse<{ inference: string; confidence: number }>> {
console.log(`AdvancedCognitionAugmentation reasoning about: ${query}`)
console.log('Context:', context)
return {
success: true,
data: {
inference: `Advanced inference about: ${query} with detailed analysis`,
confidence: 0.9
}
}
}
async infer(
dataSubset: Record<string, unknown>
): Promise<AugmentationResponse<Record<string, unknown>>> {
return {
success: true,
data: {
result: `Advanced inference from data: ${JSON.stringify(dataSubset)}`,
additionalInsights: ['insight1', 'insight2']
}
}
}
async executeLogic(
ruleId: string,
input: Record<string, unknown>
): Promise<AugmentationResponse<boolean>> {
return {
success: true,
data: true
}
}
}
/**
* Example Sense Augmentation
*/
class SimpleSenseAugmentation implements BrainyAugmentations.ISenseAugmentation {
readonly name = 'simple-sense'
readonly description = 'A simple sense augmentation for demonstration'
async initialize(): Promise<void> {
console.log('Initializing SimpleSenseAugmentation')
}
async shutDown(): Promise<void> {
console.log('Shutting down SimpleSenseAugmentation')
}
async getStatus(): Promise<'active' | 'inactive' | 'error'> {
return 'active'
}
async processRawData(
rawData: Buffer | string,
dataType: string
): Promise<AugmentationResponse<{ nouns: string[]; verbs: string[] }>> {
console.log(`SimpleSenseAugmentation processing ${dataType} data:`,
typeof rawData === 'string' ? rawData : 'Buffer data')
return {
success: true,
data: {
nouns: ['example', 'data', 'processing'],
verbs: ['process', 'analyze', 'extract']
}
}
}
async listenToFeed(
feedUrl: string,
callback: (data: { nouns: string[]; verbs: string[] }) => void
): Promise<void> {
console.log(`SimpleSenseAugmentation listening to feed: ${feedUrl}`)
// In a real implementation, this would set up a listener
}
}
/**
* Main function to demonstrate the augmentation pipeline
*/
async function main() {
try {
console.log('=== Augmentation Pipeline Example ===')
// Create augmentation instances
const simpleCognition = new SimpleCognitionAugmentation()
const advancedCognition = new AdvancedCognitionAugmentation()
const simpleSense = new SimpleSenseAugmentation()
// Register augmentations with the pipeline
augmentationPipeline
.register(simpleCognition)
.register(advancedCognition)
.register(simpleSense)
// Initialize all registered augmentations
console.log('\n=== Initializing Augmentations ===')
await augmentationPipeline.initialize()
// Execute a cognition pipeline in sequential mode (default)
console.log('\n=== Executing Cognition Pipeline (Sequential) ===')
const reasoningResults = await augmentationPipeline.executeCognitionPipeline(
'reason',
['What is the capital of France?', { additionalContext: 'geography' }]
)
console.log('\nReasoning Results:')
reasoningResults.forEach((result, index) => {
console.log(`Result ${index + 1}:`)
console.log(` Success: ${result.success}`)
if (result.success) {
console.log(` Inference: ${result.data.inference}`)
console.log(` Confidence: ${result.data.confidence}`)
} else {
console.log(` Error: ${result.error}`)
}
})
// Execute a cognition pipeline in parallel mode
console.log('\n=== Executing Cognition Pipeline (Parallel) ===')
const inferResults = await augmentationPipeline.executeCognitionPipeline(
'infer',
[{ topic: 'climate change', data: [1, 2, 3] }],
{ mode: ExecutionMode.PARALLEL }
)
console.log('\nInference Results:')
inferResults.forEach((result, index) => {
console.log(`Result ${index + 1}:`)
console.log(` Success: ${result.success}`)
if (result.success) {
console.log(` Data: ${JSON.stringify(result.data)}`)
} else {
console.log(` Error: ${result.error}`)
}
})
// Execute a sense pipeline
console.log('\n=== Executing Sense Pipeline ===')
const processingResults = await augmentationPipeline.executeSensePipeline(
'processRawData',
['This is some example text to process', 'text']
)
console.log('\nProcessing Results:')
processingResults.forEach((result, index) => {
console.log(`Result ${index + 1}:`)
console.log(` Success: ${result.success}`)
if (result.success) {
console.log(` Nouns: ${result.data.nouns.join(', ')}`)
console.log(` Verbs: ${result.data.verbs.join(', ')}`)
} else {
console.log(` Error: ${result.error}`)
}
})
// Shut down all registered augmentations
console.log('\n=== Shutting Down Augmentations ===')
await augmentationPipeline.shutDown()
console.log('\n=== Example Complete ===')
} catch (error) {
console.error('Error in augmentation pipeline example:', error)
}
}
// Run the example
main()

38
src/index.ts
View file

@ -34,6 +34,20 @@ export {
createStorage
}
// Export augmentation pipeline
import {
AugmentationPipeline,
augmentationPipeline,
ExecutionMode,
PipelineOptions
} from './augmentationPipeline.js'
export {
AugmentationPipeline,
augmentationPipeline,
ExecutionMode
}
export type { PipelineOptions }
// Export types
import type {
Vector,
@ -59,3 +73,27 @@ export type {
HNSWConfig,
StorageAdapter
}
// Export augmentation types
import type {
IAugmentation,
AugmentationResponse,
IWebSocketSupport
} from './types/augmentations.js'
export type {
IAugmentation,
AugmentationResponse,
IWebSocketSupport
}
export { BrainyAugmentations } from './types/augmentations.js'
// Export combined WebSocket augmentation interfaces
export type {
IWebSocketCognitionAugmentation,
IWebSocketSenseAugmentation,
IWebSocketPerceptionAugmentation,
IWebSocketActivationAugmentation,
IWebSocketDialogAugmentation,
IWebSocketConduitAugmentation,
IWebSocketMemoryAugmentation
} from './types/augmentations.js'

349
src/types/augmentations.ts
View file

@ -6,7 +6,7 @@ type WebSocketConnection = {
}
type DataCallback<T> = (data: T) => void
type AugmentationResponse<T> = Promise<{
export type AugmentationResponse<T> = Promise<{
success: boolean
data: T
error?: string
@ -16,7 +16,7 @@ type AugmentationResponse<T> = Promise<{
* Base interface for all Brainy augmentations.
* All augmentations must implement these core properties.
*/
interface IAugmentation {
export interface IAugmentation {
/** A unique identifier for the augmentation (e.g., "my-reasoner-v1") */
readonly name: string
/** A human-readable description of the augmentation's purpose */
@ -37,7 +37,7 @@ interface IAugmentation {
* Interface for WebSocket support.
* Augmentations that implement this interface can communicate via WebSockets.
*/
interface IWebSocketSupport {
export interface IWebSocketSupport extends IAugmentation {
/**
* Establishes a WebSocket connection.
* @param url The WebSocket server URL to connect to
@ -69,36 +69,7 @@ interface IWebSocketSupport {
closeWebSocket(connectionId: string, code?: number, reason?: string): Promise<void>
}
namespace BrainyAugmentations {
/**
* Interface for Cognitions augmentations.
* These augmentations enable advanced reasoning, inference, and logical operations.
*/
export interface ICognitionAugmentation extends IAugmentation {
/**
* Performs a reasoning operation based on current knowledge.
* @param query The specific reasoning task or question
* @param context Optional additional context for the reasoning
*/
reason(query: string, context?: Record<string, unknown>): AugmentationResponse<{
inference: string
confidence: number
}>
/**
* Infers relationships or new facts from existing data.
* @param dataSubset A subset of data to infer from
*/
infer(dataSubset: Record<string, unknown>): AugmentationResponse<Record<string, unknown>>
/**
* Executes a logical operation or rule set.
* @param ruleId The identifier of the rule or logic to apply
* @param input Data to apply the logic to
*/
executeLogic(ruleId: string, input: Record<string, unknown>): AugmentationResponse<boolean>
}
export namespace BrainyAugmentations {
/**
* Interface for Senses augmentations.
* These augmentations ingest and process raw, unstructured data into nouns and verbs.
@ -125,111 +96,6 @@ namespace BrainyAugmentations {
): Promise<void>
}
/**
* Interface for Perceptions augmentations.
* These augmentations interpret, contextualize, and visualize identified nouns and verbs.
*/
export interface IPerceptionAugmentation extends IAugmentation {
/**
* Interprets and contextualizes processed nouns and verbs.
* @param nouns The list of identified nouns
* @param verbs The list of identified verbs
* @param context Optional additional context for interpretation
*/
interpret(
nouns: string[],
verbs: string[],
context?: Record<string, unknown>
): AugmentationResponse<Record<string, unknown>>
/**
* Organizes and filters information.
* @param data The data to organize (e.g., interpreted perceptions)
* @param criteria Optional criteria for filtering/prioritization
*/
organize(
data: Record<string, unknown>,
criteria?: Record<string, unknown>
): AugmentationResponse<Record<string, unknown>>
/**
* Generates a visualization based on the provided data.
* @param data The data to visualize (e.g., interpreted patterns)
* @param visualizationType The desired type of visualization (e.g., 'graph', 'chart')
*/
generateVisualization(
data: Record<string, unknown>,
visualizationType: string
): AugmentationResponse<string | Buffer | Record<string, unknown>>
}
/**
* Interface for Activations augmentations.
* These augmentations dictate how Brainy initiates actions, responses, or data manipulations.
*/
export interface IActivationAugmentation extends IAugmentation {
/**
* Triggers an action based on a processed command or internal state.
* @param actionName The name of the action to trigger
* @param parameters Optional parameters for the action
*/
triggerAction(
actionName: string,
parameters?: Record<string, unknown>
): AugmentationResponse<unknown>
/**
* Generates an expressive output or response from Brainy.
* @param knowledgeId The identifier of the knowledge to express
* @param format The desired output format (e.g., 'text', 'json')
*/
generateOutput(knowledgeId: string, format: string): AugmentationResponse<string | Record<string, unknown>>
/**
* Interacts with an external system or API.
* @param systemId The identifier of the external system
* @param payload The data to send to the external system
*/
interactExternal(systemId: string, payload: Record<string, unknown>): AugmentationResponse<unknown>
}
/**
* Interface for Dialogs augmentations.
* These augmentations facilitate natural language understanding and generation for conversational interaction.
*/
export interface IDialogAugmentation extends IAugmentation {
/**
* Processes a user's natural language input (query).
* @param naturalLanguageQuery The raw text query from the user
* @param sessionId An optional session ID for conversational context
*/
processUserInput(naturalLanguageQuery: string, sessionId?: string): AugmentationResponse<{
intent: string
nouns: string[]
verbs: string[]
context: Record<string, unknown>
}>
/**
* Generates a natural language response based on Brainy's knowledge and interpreted input.
* @param interpretedInput The output from `processUserInput` or similar
* @param knowledgeContext Relevant knowledge retrieved from Brainy
* @param sessionId An optional session ID for conversational context
*/
generateResponse(
interpretedInput: Record<string, unknown>,
knowledgeContext: Record<string, unknown>,
sessionId?: string
): AugmentationResponse<string>
/**
* Manages and updates conversational context.
* @param sessionId The session ID
* @param contextUpdate The data to update the context with
*/
manageContext(sessionId: string, contextUpdate: Record<string, unknown>): Promise<void>
}
/**
* Interface for Conduits augmentations.
* These augmentations establish and manage high-bandwidth, dedicated channels for structured, programmatic two-way data exchange.
@ -272,12 +138,207 @@ namespace BrainyAugmentations {
*/
monitorStream(streamId: string, callback: DataCallback<unknown>): Promise<void>
}
/**
* Interface for Cognitions augmentations.
* These augmentations enable advanced reasoning, inference, and logical operations.
*/
export interface ICognitionAugmentation extends IAugmentation {
/**
* Performs a reasoning operation based on current knowledge.
* @param query The specific reasoning task or question
* @param context Optional additional context for the reasoning
*/
reason(query: string, context?: Record<string, unknown>): AugmentationResponse<{
inference: string
confidence: number
}>
/**
* Infers relationships or new facts from existing data.
* @param dataSubset A subset of data to infer from
*/
infer(dataSubset: Record<string, unknown>): AugmentationResponse<Record<string, unknown>>
/**
* Executes a logical operation or rule set.
* @param ruleId The identifier of the rule or logic to apply
* @param input Data to apply the logic to
*/
executeLogic(ruleId: string, input: Record<string, unknown>): AugmentationResponse<boolean>
}
/**
* Interface for Memory augmentations.
* These augmentations provide storage capabilities for data in different formats (e.g., fileSystem, in-memory, or firestore).
*/
export interface IMemoryAugmentation extends IAugmentation {
/**
* Stores data in the memory system.
* @param key The unique identifier for the data
* @param data The data to store
* @param options Optional storage options (e.g., expiration, format)
*/
storeData(
key: string,
data: unknown,
options?: Record<string, unknown>
): AugmentationResponse<boolean>
/**
* Retrieves data from the memory system.
* @param key The unique identifier for the data
* @param options Optional retrieval options (e.g., format, version)
*/
retrieveData(
key: string,
options?: Record<string, unknown>
): AugmentationResponse<unknown>
/**
* Updates existing data in the memory system.
* @param key The unique identifier for the data
* @param data The updated data
* @param options Optional update options (e.g., merge, overwrite)
*/
updateData(
key: string,
data: unknown,
options?: Record<string, unknown>
): AugmentationResponse<boolean>
/**
* Deletes data from the memory system.
* @param key The unique identifier for the data
* @param options Optional deletion options
*/
deleteData(
key: string,
options?: Record<string, unknown>
): AugmentationResponse<boolean>
/**
* Lists available data keys in the memory system.
* @param pattern Optional pattern to filter keys (e.g., prefix, regex)
* @param options Optional listing options (e.g., limit, offset)
*/
listDataKeys(
pattern?: string,
options?: Record<string, unknown>
): AugmentationResponse<string[]>
}
/**
* Interface for Perceptions augmentations.
* These augmentations interpret, contextualize, and visualize identified nouns and verbs.
*/
export interface IPerceptionAugmentation extends IAugmentation {
/**
* Interprets and contextualizes processed nouns and verbs.
* @param nouns The list of identified nouns
* @param verbs The list of identified verbs
* @param context Optional additional context for interpretation
*/
interpret(
nouns: string[],
verbs: string[],
context?: Record<string, unknown>
): AugmentationResponse<Record<string, unknown>>
/**
* Organizes and filters information.
* @param data The data to organize (e.g., interpreted perceptions)
* @param criteria Optional criteria for filtering/prioritization
*/
organize(
data: Record<string, unknown>,
criteria?: Record<string, unknown>
): AugmentationResponse<Record<string, unknown>>
/**
* Generates a visualization based on the provided data.
* @param data The data to visualize (e.g., interpreted patterns)
* @param visualizationType The desired type of visualization (e.g., 'graph', 'chart')
*/
generateVisualization(
data: Record<string, unknown>,
visualizationType: string
): AugmentationResponse<string | Buffer | Record<string, unknown>>
}
/**
* Interface for Dialogs augmentations.
* These augmentations facilitate natural language understanding and generation for conversational interaction.
*/
export interface IDialogAugmentation extends IAugmentation {
/**
* Processes a user's natural language input (query).
* @param naturalLanguageQuery The raw text query from the user
* @param sessionId An optional session ID for conversational context
*/
processUserInput(naturalLanguageQuery: string, sessionId?: string): AugmentationResponse<{
intent: string
nouns: string[]
verbs: string[]
context: Record<string, unknown>
}>
/**
* Generates a natural language response based on Brainy's knowledge and interpreted input.
* @param interpretedInput The output from `processUserInput` or similar
* @param knowledgeContext Relevant knowledge retrieved from Brainy
* @param sessionId An optional session ID for conversational context
*/
generateResponse(
interpretedInput: Record<string, unknown>,
knowledgeContext: Record<string, unknown>,
sessionId?: string
): AugmentationResponse<string>
/**
* Manages and updates conversational context.
* @param sessionId The session ID
* @param contextUpdate The data to update the context with
*/
manageContext(sessionId: string, contextUpdate: Record<string, unknown>): Promise<void>
}
/**
* Interface for Activations augmentations.
* These augmentations dictate how Brainy initiates actions, responses, or data manipulations.
*/
export interface IActivationAugmentation extends IAugmentation {
/**
* Triggers an action based on a processed command or internal state.
* @param actionName The name of the action to trigger
* @param parameters Optional parameters for the action
*/
triggerAction(
actionName: string,
parameters?: Record<string, unknown>
): AugmentationResponse<unknown>
/**
* Generates an expressive output or response from Brainy.
* @param knowledgeId The identifier of the knowledge to express
* @param format The desired output format (e.g., 'text', 'json')
*/
generateOutput(knowledgeId: string, format: string): AugmentationResponse<string | Record<string, unknown>>
/**
* Interacts with an external system or API.
* @param systemId The identifier of the external system
* @param payload The data to send to the external system
*/
interactExternal(systemId: string, payload: Record<string, unknown>): AugmentationResponse<unknown>
}
}
/** WebSocket-enabled augmentation interfaces */
type IWebSocketCognitionAugmentation = BrainyAugmentations.ICognitionAugmentation & IWebSocketSupport
type IWebSocketSenseAugmentation = BrainyAugmentations.ISenseAugmentation & IWebSocketSupport
type IWebSocketPerceptionAugmentation = BrainyAugmentations.IPerceptionAugmentation & IWebSocketSupport
type IWebSocketActivationAugmentation = BrainyAugmentations.IActivationAugmentation & IWebSocketSupport
type IWebSocketDialogAugmentation = BrainyAugmentations.IDialogAugmentation & IWebSocketSupport
type IWebSocketConduitAugmentation = BrainyAugmentations.IConduitAugmentation & IWebSocketSupport
export type IWebSocketSenseAugmentation = BrainyAugmentations.ISenseAugmentation & IWebSocketSupport
export type IWebSocketConduitAugmentation = BrainyAugmentations.IConduitAugmentation & IWebSocketSupport
export type IWebSocketCognitionAugmentation = BrainyAugmentations.ICognitionAugmentation & IWebSocketSupport
export type IWebSocketMemoryAugmentation = BrainyAugmentations.IMemoryAugmentation & IWebSocketSupport
export type IWebSocketPerceptionAugmentation = BrainyAugmentations.IPerceptionAugmentation & IWebSocketSupport
export type IWebSocketDialogAugmentation = BrainyAugmentations.IDialogAugmentation & IWebSocketSupport
export type IWebSocketActivationAugmentation = BrainyAugmentations.IActivationAugmentation & IWebSocketSupport