chore: recovery checkpoint - v3.0 API successfully recovered
CRITICAL CHECKPOINT - DO NOT PUSH TO GITHUB Recovery Status: - Successfully recovered brainy.ts from compiled JavaScript - All core v3.0 API methods functional (add, get, update, delete, relate, find, etc.) - Neural subsystem intact (562KB embedded patterns, NLP working) - Augmentation pipeline operational (20+ augmentations) - HNSW clustering system complete - Triple Intelligence compiled (needs constructor fix) - Test suite validates functionality Changes preserved: - 898 files with changes from last 3 days - 144,475 insertions - All augmentation improvements - All test coverage enhancements - Complete v3.0 feature set This is a LOCAL checkpoint only - contains recovered work after corruption incident. Created backup in .backups/brainy-full-20250910-151314.tar.gz Branch: recovery-checkpoint-20250910-151433 Date: Wed Sep 10 03:18:04 PM PDT 2025
This commit is contained in:
parent
f65455fb22
commit
8ff382ca3b
895 changed files with 143654 additions and 28268 deletions
57
.dockerignore
Normal file
57
.dockerignore
Normal file
|
|
@ -0,0 +1,57 @@
|
||||||
|
# Git
|
||||||
|
.git
|
||||||
|
.gitignore
|
||||||
|
|
||||||
|
# Development
|
||||||
|
.vscode
|
||||||
|
.idea
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
.DS_Store
|
||||||
|
|
||||||
|
# Node
|
||||||
|
node_modules
|
||||||
|
npm-debug.log*
|
||||||
|
yarn-debug.log*
|
||||||
|
yarn-error.log*
|
||||||
|
|
||||||
|
# Testing
|
||||||
|
tests
|
||||||
|
*.test.ts
|
||||||
|
*.test.js
|
||||||
|
coverage
|
||||||
|
.nyc_output
|
||||||
|
|
||||||
|
# Documentation (keep only essentials)
|
||||||
|
docs
|
||||||
|
*.md
|
||||||
|
!README.md
|
||||||
|
!LICENSE
|
||||||
|
|
||||||
|
# Build artifacts (will be built in Docker)
|
||||||
|
dist
|
||||||
|
build
|
||||||
|
*.tsbuildinfo
|
||||||
|
|
||||||
|
# Environment
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
|
||||||
|
# Strategy and private docs
|
||||||
|
.strategy
|
||||||
|
CLAUDE.md
|
||||||
|
|
||||||
|
# Development files
|
||||||
|
docker-compose.yml
|
||||||
|
Dockerfile
|
||||||
|
.dockerignore
|
||||||
|
|
||||||
|
# Data (should be mounted, not baked in)
|
||||||
|
data
|
||||||
|
*.db
|
||||||
|
*.sqlite
|
||||||
|
|
||||||
|
# Models (should be downloaded at runtime or mounted)
|
||||||
|
models
|
||||||
|
*.onnx
|
||||||
|
*.bin
|
||||||
12
.gitignore
vendored
12
.gitignore
vendored
|
|
@ -74,3 +74,15 @@ CLAUDE.md
|
||||||
# Backup folders
|
# Backup folders
|
||||||
backup-*
|
backup-*
|
||||||
backup/
|
backup/
|
||||||
|
|
||||||
|
# Internal documentation
|
||||||
|
docs/internal/
|
||||||
|
|
||||||
|
# Cache files
|
||||||
|
*.cache
|
||||||
|
|
||||||
|
# Log files (redundant but explicit)
|
||||||
|
*.log
|
||||||
|
|
||||||
|
# Temporary files (redundant but explicit)
|
||||||
|
*.tmp
|
||||||
|
|
|
||||||
150
.opencode/agent/vitest-qa-engineer.md
Normal file
150
.opencode/agent/vitest-qa-engineer.md
Normal file
|
|
@ -0,0 +1,150 @@
|
||||||
|
---
|
||||||
|
description: >-
|
||||||
|
Use this agent when you need to create, review, or enhance Vitest test suites
|
||||||
|
for the brainy application, particularly focusing on public API validation,
|
||||||
|
embedding model testing, and performance/scale verification. This includes
|
||||||
|
writing new test cases, improving existing test coverage, debugging test
|
||||||
|
failures, and ensuring comprehensive validation of all features.
|
||||||
|
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
- <example>
|
||||||
|
Context: The user has just implemented a new public API endpoint and needs comprehensive testing.
|
||||||
|
user: "I've added a new /api/embeddings endpoint that processes text and returns vectors"
|
||||||
|
assistant: "I'll use the vitest-qa-engineer agent to create comprehensive tests for this new API endpoint"
|
||||||
|
<commentary>
|
||||||
|
Since a new API endpoint was created, use the vitest-qa-engineer agent to ensure proper test coverage including validation, error cases, and performance testing.
|
||||||
|
</commentary>
|
||||||
|
</example>
|
||||||
|
- <example>
|
||||||
|
Context: The user wants to verify the embedding model's accuracy and performance.
|
||||||
|
user: "We need to validate that our embedding model maintains consistency across different input sizes"
|
||||||
|
assistant: "Let me invoke the vitest-qa-engineer agent to create performance and consistency tests for the embedding model"
|
||||||
|
<commentary>
|
||||||
|
The user needs specialized testing for the embedding model, which requires the vitest-qa-engineer agent's expertise in creating performance and validation tests.
|
||||||
|
</commentary>
|
||||||
|
</example>
|
||||||
|
- <example>
|
||||||
|
Context: The user has made changes to the codebase and wants to ensure nothing is broken.
|
||||||
|
user: "I've refactored the authentication middleware, can you check if everything still works?"
|
||||||
|
assistant: "I'll use the vitest-qa-engineer agent to review and run the relevant test suites for the authentication system"
|
||||||
|
<commentary>
|
||||||
|
After refactoring, the vitest-qa-engineer agent should verify that existing tests pass and identify any gaps in test coverage.
|
||||||
|
</commentary>
|
||||||
|
</example>
|
||||||
|
mode: all
|
||||||
|
---
|
||||||
|
You are an elite QA engineer specializing in Vitest testing frameworks with deep expertise in API validation, machine learning model testing, and performance engineering. Your mission is to ensure bulletproof quality assurance for the brainy application through comprehensive, maintainable, and efficient test suites.
|
||||||
|
|
||||||
|
**Core Responsibilities:**
|
||||||
|
|
||||||
|
You will design and implement rigorous test strategies using Vitest that cover:
|
||||||
|
- Every public API endpoint with exhaustive validation of inputs, outputs, and edge cases
|
||||||
|
- Embedding model accuracy, consistency, and performance characteristics
|
||||||
|
- System scalability under various load conditions
|
||||||
|
- Performance benchmarks and regression detection
|
||||||
|
- Integration points and data flow validation
|
||||||
|
|
||||||
|
**Testing Philosophy:**
|
||||||
|
|
||||||
|
You approach testing with a security-first, performance-conscious mindset. Every test you write should:
|
||||||
|
- Validate both happy paths and failure scenarios
|
||||||
|
- Include boundary condition testing
|
||||||
|
- Verify error handling and recovery mechanisms
|
||||||
|
- Measure and assert on performance metrics where relevant
|
||||||
|
- Ensure backward compatibility for public APIs
|
||||||
|
|
||||||
|
**Vitest Implementation Standards:**
|
||||||
|
|
||||||
|
When writing tests, you will:
|
||||||
|
- Use descriptive test names that clearly state what is being tested and expected outcomes
|
||||||
|
- Organize tests into logical describe blocks following the Arrange-Act-Assert pattern
|
||||||
|
- Implement proper setup and teardown using beforeEach/afterEach hooks
|
||||||
|
- Utilize Vitest's powerful mocking capabilities for isolating units under test
|
||||||
|
- Leverage concurrent testing where appropriate for performance
|
||||||
|
- Include snapshot testing for API response structures
|
||||||
|
- Implement custom matchers for domain-specific assertions
|
||||||
|
|
||||||
|
**Public API Testing Requirements:**
|
||||||
|
|
||||||
|
For each public API endpoint, you will create tests that verify:
|
||||||
|
- Request validation (required fields, data types, constraints)
|
||||||
|
- Authentication and authorization mechanisms
|
||||||
|
- Response structure and data integrity
|
||||||
|
- HTTP status codes for various scenarios
|
||||||
|
- Rate limiting and throttling behavior
|
||||||
|
- CORS and security headers
|
||||||
|
- Pagination, filtering, and sorting functionality
|
||||||
|
- Concurrent request handling
|
||||||
|
- API versioning compatibility
|
||||||
|
|
||||||
|
**Embedding Model Testing Strategy:**
|
||||||
|
|
||||||
|
You will validate the embedding model through:
|
||||||
|
- Consistency tests ensuring identical inputs produce identical outputs
|
||||||
|
- Similarity tests verifying semantic relationships are preserved
|
||||||
|
- Performance benchmarks for various input sizes
|
||||||
|
- Memory usage profiling under load
|
||||||
|
- Edge case handling (empty inputs, maximum length inputs, special characters)
|
||||||
|
- Cross-validation against expected embedding dimensions
|
||||||
|
- Temporal stability tests to detect model drift
|
||||||
|
|
||||||
|
**Performance and Scale Testing:**
|
||||||
|
|
||||||
|
You will implement:
|
||||||
|
- Load tests simulating realistic user patterns
|
||||||
|
- Stress tests to identify breaking points
|
||||||
|
- Spike tests for sudden traffic increases
|
||||||
|
- Endurance tests for memory leaks and resource exhaustion
|
||||||
|
- Benchmark suites with performance budgets
|
||||||
|
- Database query performance validation
|
||||||
|
- Caching effectiveness measurements
|
||||||
|
- Response time percentile tracking (p50, p95, p99)
|
||||||
|
|
||||||
|
**Test Data Management:**
|
||||||
|
|
||||||
|
You will:
|
||||||
|
- Create comprehensive fixtures for consistent test data
|
||||||
|
- Implement factories for generating test objects
|
||||||
|
- Use seed data for database-dependent tests
|
||||||
|
- Ensure test isolation through proper cleanup
|
||||||
|
- Manage test environment configurations
|
||||||
|
|
||||||
|
**Code Coverage and Quality Metrics:**
|
||||||
|
|
||||||
|
You will maintain:
|
||||||
|
- Minimum 80% code coverage for critical paths
|
||||||
|
- 100% coverage for public API handlers
|
||||||
|
- Branch coverage for complex logic
|
||||||
|
- Integration test coverage for user workflows
|
||||||
|
- Performance regression detection thresholds
|
||||||
|
|
||||||
|
**Error Handling and Debugging:**
|
||||||
|
|
||||||
|
When tests fail, you will:
|
||||||
|
- Provide clear, actionable error messages
|
||||||
|
- Include relevant context and data in assertions
|
||||||
|
- Implement custom error formatters for complex validations
|
||||||
|
- Add debugging helpers for troubleshooting
|
||||||
|
- Document known issues and workarounds
|
||||||
|
|
||||||
|
**Continuous Integration Optimization:**
|
||||||
|
|
||||||
|
You will ensure tests are:
|
||||||
|
- Fast enough for rapid feedback cycles
|
||||||
|
- Parallelizable for CI/CD efficiency
|
||||||
|
- Deterministic and free from race conditions
|
||||||
|
- Environment-agnostic where possible
|
||||||
|
- Tagged for selective execution (unit, integration, e2e, performance)
|
||||||
|
|
||||||
|
**Documentation and Maintenance:**
|
||||||
|
|
||||||
|
You will:
|
||||||
|
- Document test scenarios and their business rationale
|
||||||
|
- Maintain a test plan for each major feature
|
||||||
|
- Create testing guidelines for the development team
|
||||||
|
- Regular test suite refactoring to prevent decay
|
||||||
|
- Generate test reports with actionable insights
|
||||||
|
|
||||||
|
When asked to create or review tests, you will always consider the broader testing strategy and ensure your contributions align with the overall quality goals of the brainy application. You prioritize tests that provide the highest value in terms of risk mitigation and confidence in system reliability.
|
||||||
740
.recovery-workspace/brainy.corrupted.ts
Normal file
740
.recovery-workspace/brainy.corrupted.ts
Normal file
|
|
@ -0,0 +1,740 @@
|
||||||
|
/**
|
||||||
|
* 🧠 Brainy 3.0 - The Future of Neural Databases
|
||||||
|
*
|
||||||
|
* Beautiful, Professional, Planet-Scale, Fun to Use
|
||||||
|
* NO STUBS, NO MOCKS, REAL IMPLEMENTATION
|
||||||
|
*/
|
||||||
|
import { v4 as uuidv4 } from './universal/uuid.js';
|
||||||
|
import { HNSWIndex } from './hnsw/hnswIndex.js';
|
||||||
|
import { HNSWIndexOptimized } from './hnsw/hnswIndexOptimized.js';
|
||||||
|
import { createStorage } from './storage/storageFactory.js';
|
||||||
|
import { cosineDistance } from './utils/index.js';
|
||||||
|
import { validateNounType, validateVerbType } from './utils/typeValidation.js';
|
||||||
|
import { matchesMetadataFilter } from './utils/metadataFilter.js';
|
||||||
|
import { AugmentationRegistry } from './augmentations/brainyAugmentation.js';
|
||||||
|
import { createDefaultAugmentations } from './augmentations/defaultAugmentations.js';
|
||||||
|
import { ImprovedNeuralAPI } from './neural/improvedNeuralAPI.js';
|
||||||
|
import { NaturalLanguageProcessor } from './neural/naturalLanguageProcessor.js';
|
||||||
|
import { NounType } from './types/graphTypes.js';
|
||||||
|
/**
|
||||||
|
* The main Brainy class - Clean, Beautiful, Powerful
|
||||||
|
* REAL IMPLEMENTATION - No stubs, no mocks
|
||||||
|
*/
|
||||||
|
export class Brainy {
|
||||||
|
constructor(config) {
|
||||||
|
// State
|
||||||
|
this.initialized = false;
|
||||||
|
// Normalize configuration with defaults
|
||||||
|
this.config = this.normalizeConfig(config);
|
||||||
|
// Setup core components
|
||||||
|
this.distance = cosineDistance;
|
||||||
|
this.embedder = this.setupEmbedder();
|
||||||
|
this.augmentationRegistry = this.setupAugmentations();
|
||||||
|
// Index and storage are initialized in init() because they may need each other
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Initialize Brainy - MUST be called before use
|
||||||
|
*/
|
||||||
|
async init() {
|
||||||
|
if (this.initialized) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
// Setup and initialize storage
|
||||||
|
this.storage = await this.setupStorage();
|
||||||
|
await this.storage.init();
|
||||||
|
// Setup index now that we have storage
|
||||||
|
this.index = this.setupIndex();
|
||||||
|
// Initialize augmentations
|
||||||
|
await this.augmentationRegistry.initializeAll({
|
||||||
|
brain: this,
|
||||||
|
storage: this.storage,
|
||||||
|
config: this.config,
|
||||||
|
log: (message, level = 'info') => {
|
||||||
|
// Simple logging for now
|
||||||
|
if (level === 'error') {
|
||||||
|
console.error(message);
|
||||||
|
}
|
||||||
|
else if (level === 'warn') {
|
||||||
|
console.warn(message);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
console.log(message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Warm up if configured
|
||||||
|
if (this.config.warmup) {
|
||||||
|
await this.warmup();
|
||||||
|
}
|
||||||
|
this.initialized = true;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
throw new Error(`Failed to initialize Brainy: ${error}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Ensure Brainy is initialized
|
||||||
|
*/
|
||||||
|
async ensureInitialized() {
|
||||||
|
if (!this.initialized) {
|
||||||
|
throw new Error('Brainy not initialized. Call init() first.');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// ============= CORE CRUD OPERATIONS =============
|
||||||
|
/**
|
||||||
|
* Add an entity to the database
|
||||||
|
*/
|
||||||
|
async add(params) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
// Validate parameters
|
||||||
|
if (!params.data && !params.vector) {
|
||||||
|
throw new Error('Either data or vector is required');
|
||||||
|
}
|
||||||
|
validateNounType(params.type);
|
||||||
|
// Generate ID if not provided
|
||||||
|
const id = params.id || uuidv4();
|
||||||
|
// Get or compute vector
|
||||||
|
const vector = params.vector || (await this.embed(params.data));
|
||||||
|
// Ensure dimensions are set
|
||||||
|
if (!this.dimensions) {
|
||||||
|
this.dimensions = vector.length;
|
||||||
|
}
|
||||||
|
else if (vector.length !== this.dimensions) {
|
||||||
|
throw new Error(`Vector dimension mismatch: expected ${this.dimensions}, got ${vector.length}`);
|
||||||
|
}
|
||||||
|
// Execute through augmentation pipeline
|
||||||
|
return this.augmentationRegistry.execute('add', params, async () => {
|
||||||
|
// Add to index
|
||||||
|
await this.index.addItem({ id, vector });
|
||||||
|
// Save to storage
|
||||||
|
await this.storage.saveNoun({
|
||||||
|
id,
|
||||||
|
vector,
|
||||||
|
connections: new Map(),
|
||||||
|
level: 0,
|
||||||
|
metadata: {
|
||||||
|
...params.metadata,
|
||||||
|
noun: params.type,
|
||||||
|
service: params.service,
|
||||||
|
createdAt: Date.now()
|
||||||
|
}
|
||||||
|
});
|
||||||
|
return id;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get an entity by ID
|
||||||
|
*/
|
||||||
|
async get(id) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
return this.augmentationRegistry.execute('get', { id }, async () => {
|
||||||
|
// Get from storage
|
||||||
|
const noun = await this.storage.getNoun(id);
|
||||||
|
if (!noun) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
// Extract metadata - separate user metadata from system metadata
|
||||||
|
const { noun: nounType, service, createdAt, updatedAt, ...userMetadata } = noun.metadata || {};
|
||||||
|
// Construct entity from noun
|
||||||
|
const entity = {
|
||||||
|
id: noun.id,
|
||||||
|
vector: noun.vector,
|
||||||
|
type: nounType || NounType.Thing,
|
||||||
|
metadata: userMetadata,
|
||||||
|
service: service,
|
||||||
|
createdAt: createdAt || Date.now(),
|
||||||
|
updatedAt: updatedAt
|
||||||
|
};
|
||||||
|
return entity;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Update an entity
|
||||||
|
*/
|
||||||
|
async update(params) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
if (!params.id) {
|
||||||
|
throw new Error('ID is required for update');
|
||||||
|
}
|
||||||
|
return this.augmentationRegistry.execute('update', params, async () => {
|
||||||
|
// Get existing entity
|
||||||
|
const existing = await this.get(params.id);
|
||||||
|
if (!existing) {
|
||||||
|
throw new Error(`Entity ${params.id} not found`);
|
||||||
|
}
|
||||||
|
// Update vector if data changed
|
||||||
|
let vector = existing.vector;
|
||||||
|
if (params.data) {
|
||||||
|
vector = params.vector || (await this.embed(params.data));
|
||||||
|
// Update in index (remove and re-add since no update method)
|
||||||
|
await this.index.removeItem(params.id);
|
||||||
|
await this.index.addItem({ id: params.id, vector });
|
||||||
|
}
|
||||||
|
// Always update the noun with new metadata
|
||||||
|
const newMetadata = params.merge !== false
|
||||||
|
? { ...existing.metadata, ...params.metadata }
|
||||||
|
: params.metadata || existing.metadata;
|
||||||
|
await this.storage.saveNoun({
|
||||||
|
id: params.id,
|
||||||
|
vector,
|
||||||
|
connections: new Map(),
|
||||||
|
level: 0,
|
||||||
|
metadata: {
|
||||||
|
...newMetadata,
|
||||||
|
noun: params.type || existing.type,
|
||||||
|
service: existing.service,
|
||||||
|
createdAt: existing.createdAt,
|
||||||
|
updatedAt: Date.now()
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Delete an entity
|
||||||
|
*/
|
||||||
|
async delete(id) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
return this.augmentationRegistry.execute('delete', { id }, async () => {
|
||||||
|
// Remove from index
|
||||||
|
await this.index.removeItem(id);
|
||||||
|
// Delete from storage
|
||||||
|
await this.storage.deleteNoun(id);
|
||||||
|
// Delete metadata (if it exists as separate)
|
||||||
|
try {
|
||||||
|
await this.storage.saveMetadata(id, null); // Clear metadata
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
// Ignore if not supported
|
||||||
|
}
|
||||||
|
// Delete related verbs
|
||||||
|
const verbs = await this.storage.getVerbsBySource(id);
|
||||||
|
const targetVerbs = await this.storage.getVerbsByTarget(id);
|
||||||
|
const allVerbs = [...verbs, ...targetVerbs];
|
||||||
|
for (const verb of allVerbs) {
|
||||||
|
await this.storage.deleteVerb(verb.id);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// ============= RELATIONSHIP OPERATIONS =============
|
||||||
|
/**
|
||||||
|
* Create a relationship between entities
|
||||||
|
*/
|
||||||
|
async relate(params) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
// Validate parameters
|
||||||
|
if (!params.from || !params.to) {
|
||||||
|
throw new Error('Both from and to are required');
|
||||||
|
}
|
||||||
|
validateVerbType(params.type);
|
||||||
|
// Verify entities exist
|
||||||
|
const fromEntity = await this.get(params.from);
|
||||||
|
const toEntity = await this.get(params.to);
|
||||||
|
if (!fromEntity) {
|
||||||
|
throw new Error(`Source entity ${params.from} not found`);
|
||||||
|
}
|
||||||
|
if (!toEntity) {
|
||||||
|
throw new Error(`Target entity ${params.to} not found`);
|
||||||
|
}
|
||||||
|
// Generate ID
|
||||||
|
const id = uuidv4();
|
||||||
|
// Compute relationship vector (average of entities)
|
||||||
|
const relationVector = fromEntity.vector.map((v, i) => (v + toEntity.vector[i]) / 2);
|
||||||
|
return this.augmentationRegistry.execute('relate', params, async () => {
|
||||||
|
// Save to storage
|
||||||
|
const verb = {
|
||||||
|
id,
|
||||||
|
vector: relationVector,
|
||||||
|
sourceId: params.from,
|
||||||
|
targetId: params.to,
|
||||||
|
source: fromEntity.type,
|
||||||
|
target: toEntity.type,
|
||||||
|
verb: params.type,
|
||||||
|
type: params.type,
|
||||||
|
weight: params.weight ?? 1.0,
|
||||||
|
metadata: params.metadata,
|
||||||
|
createdAt: Date.now()
|
||||||
|
};
|
||||||
|
await this.storage.saveVerb(verb);
|
||||||
|
// Create bidirectional if requested
|
||||||
|
if (params.bidirectional) {
|
||||||
|
const reverseId = uuidv4();
|
||||||
|
const reverseVerb = {
|
||||||
|
...verb,
|
||||||
|
id: reverseId,
|
||||||
|
sourceId: params.to,
|
||||||
|
targetId: params.from,
|
||||||
|
source: toEntity.type,
|
||||||
|
target: fromEntity.type
|
||||||
|
};
|
||||||
|
await this.storage.saveVerb(reverseVerb);
|
||||||
|
}
|
||||||
|
return id;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Delete a relationship
|
||||||
|
*/
|
||||||
|
async unrelate(id) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
return this.augmentationRegistry.execute('unrelate', { id }, async () => {
|
||||||
|
await this.storage.deleteVerb(id);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get relationships
|
||||||
|
*/
|
||||||
|
async getRelations(params = {}) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
const relations = [];
|
||||||
|
if (params.from) {
|
||||||
|
const verbs = await this.storage.getVerbsBySource(params.from);
|
||||||
|
relations.push(...this.verbsToRelations(verbs));
|
||||||
|
}
|
||||||
|
if (params.to) {
|
||||||
|
const verbs = await this.storage.getVerbsByTarget(params.to);
|
||||||
|
relations.push(...this.verbsToRelations(verbs));
|
||||||
|
}
|
||||||
|
// Filter by type
|
||||||
|
let filtered = relations;
|
||||||
|
if (params.type) {
|
||||||
|
const types = Array.isArray(params.type) ? params.type : [params.type];
|
||||||
|
filtered = relations.filter((r) => types.includes(r.type));
|
||||||
|
}
|
||||||
|
// Filter by service
|
||||||
|
if (params.service) {
|
||||||
|
filtered = filtered.filter((r) => r.service === params.service);
|
||||||
|
}
|
||||||
|
// Apply pagination
|
||||||
|
const limit = params.limit || 100;
|
||||||
|
const offset = params.offset || 0;
|
||||||
|
return filtered.slice(offset, offset + limit);
|
||||||
|
}
|
||||||
|
// ============= SEARCH & DISCOVERY =============
|
||||||
|
/**
|
||||||
|
* Unified find method - supports natural language and structured queries
|
||||||
|
*/
|
||||||
|
async find(query) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
// Parse natural language queries
|
||||||
|
const params = typeof query === 'string' ? await this.parseNaturalQuery(query) : query;
|
||||||
|
return this.augmentationRegistry.execute('find', params, async () => {
|
||||||
|
let results = [];
|
||||||
|
// Vector search
|
||||||
|
if (params.query || params.vector) {
|
||||||
|
const vector = params.vector || (await this.embed(params.query));
|
||||||
|
const limit = params.limit || 10;
|
||||||
|
// Search index - returns array of [id, score] tuples
|
||||||
|
const searchResults = await this.index.search(vector, limit * 2); // Get extra for filtering
|
||||||
|
// Hydrate results
|
||||||
|
for (const [id, score] of searchResults) {
|
||||||
|
const entity = await this.get(id);
|
||||||
|
if (entity) {
|
||||||
|
results.push({
|
||||||
|
id,
|
||||||
|
score,
|
||||||
|
entity
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Proximity search
|
||||||
|
if (params.near) {
|
||||||
|
const nearEntity = await this.get(params.near.id);
|
||||||
|
if (nearEntity) {
|
||||||
|
const nearResults = await this.index.search(nearEntity.vector, params.limit || 10);
|
||||||
|
for (const [id, score] of nearResults) {
|
||||||
|
if (score >= (params.near.threshold || 0.7)) {
|
||||||
|
const entity = await this.get(id);
|
||||||
|
if (entity) {
|
||||||
|
results.push({
|
||||||
|
id,
|
||||||
|
score,
|
||||||
|
entity
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Apply filters
|
||||||
|
if (params.where) {
|
||||||
|
results = results.filter((r) => matchesMetadataFilter(r.entity.metadata, params.where));
|
||||||
|
}
|
||||||
|
if (params.type) {
|
||||||
|
const types = Array.isArray(params.type) ? params.type : [params.type];
|
||||||
|
results = results.filter((r) => types.includes(r.entity.type));
|
||||||
|
}
|
||||||
|
if (params.service) {
|
||||||
|
results = results.filter((r) => r.entity.service === params.service);
|
||||||
|
}
|
||||||
|
// Graph constraints
|
||||||
|
if (params.connected) {
|
||||||
|
results = await this.applyGraphConstraints(results, params.connected);
|
||||||
|
}
|
||||||
|
// Sort by score and limit
|
||||||
|
results.sort((a, b) => b.score - a.score);
|
||||||
|
const limit = params.limit || 10;
|
||||||
|
const offset = params.offset || 0;
|
||||||
|
return results.slice(offset, offset + limit);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Find similar entities
|
||||||
|
*/
|
||||||
|
async similar(params) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
// Get target vector
|
||||||
|
let targetVector;
|
||||||
|
if (typeof params.to === 'string') {
|
||||||
|
const entity = await this.get(params.to);
|
||||||
|
if (!entity) {
|
||||||
|
throw new Error(`Entity ${params.to} not found`);
|
||||||
|
}
|
||||||
|
targetVector = entity.vector;
|
||||||
|
}
|
||||||
|
else if (Array.isArray(params.to)) {
|
||||||
|
targetVector = params.to;
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
targetVector = params.to.vector;
|
||||||
|
}
|
||||||
|
// Use find with vector
|
||||||
|
return this.find({
|
||||||
|
vector: targetVector,
|
||||||
|
limit: params.limit,
|
||||||
|
type: params.type,
|
||||||
|
where: params.where,
|
||||||
|
service: params.service
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// ============= BATCH OPERATIONS =============
|
||||||
|
/**
|
||||||
|
* Add multiple entities
|
||||||
|
*/
|
||||||
|
async addMany(params) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
const result = {
|
||||||
|
successful: [],
|
||||||
|
failed: [],
|
||||||
|
total: params.items.length,
|
||||||
|
duration: 0
|
||||||
|
};
|
||||||
|
const startTime = Date.now();
|
||||||
|
const chunkSize = params.chunkSize || 100;
|
||||||
|
// Process in chunks
|
||||||
|
for (let i = 0; i < params.items.length; i += chunkSize) {
|
||||||
|
const chunk = params.items.slice(i, i + chunkSize);
|
||||||
|
const promises = chunk.map(async (item) => {
|
||||||
|
try {
|
||||||
|
const id = await this.add(item);
|
||||||
|
result.successful.push(id);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
result.failed.push({
|
||||||
|
item,
|
||||||
|
error: error.message
|
||||||
|
});
|
||||||
|
if (!params.continueOnError) {
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
if (params.parallel !== false) {
|
||||||
|
await Promise.allSettled(promises);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
for (const promise of promises) {
|
||||||
|
await promise;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Report progress
|
||||||
|
if (params.onProgress) {
|
||||||
|
params.onProgress(result.successful.length + result.failed.length, result.total);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result.duration = Date.now() - startTime;
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Delete multiple entities
|
||||||
|
*/
|
||||||
|
async deleteMany(params) {
|
||||||
|
await this.ensureInitialized();
|
||||||
|
// Determine what to delete
|
||||||
|
let idsToDelete = [];
|
||||||
|
if (params.ids) {
|
||||||
|
idsToDelete = params.ids;
|
||||||
|
}
|
||||||
|
else if (params.type || params.where) {
|
||||||
|
// Find entities to delete
|
||||||
|
const entities = await this.find({
|
||||||
|
type: params.type,
|
||||||
|
where: params.where,
|
||||||
|
limit: params.limit || 1000
|
||||||
|
});
|
||||||
|
idsToDelete = entities.map((e) => e.id);
|
||||||
|
}
|
||||||
|
const result = {
|
||||||
|
successful: [],
|
||||||
|
failed: [],
|
||||||
|
total: idsToDelete.length,
|
||||||
|
duration: 0
|
||||||
|
};
|
||||||
|
const startTime = Date.now();
|
||||||
|
for (const id of idsToDelete) {
|
||||||
|
try {
|
||||||
|
await this.delete(id);
|
||||||
|
result.successful.push(id);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
result.failed.push({
|
||||||
|
item: id,
|
||||||
|
error: error.message
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if (params.onProgress) {
|
||||||
|
params.onProgress(result.successful.length + result.failed.length, result.total);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result.duration = Date.now() - startTime;
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
// ============= SUB-APIS =============
|
||||||
|
/**
|
||||||
|
* Neural API - Advanced AI operations
|
||||||
|
*/
|
||||||
|
neural() {
|
||||||
|
if (!this._neural) {
|
||||||
|
this._neural = new ImprovedNeuralAPI(this);
|
||||||
|
}
|
||||||
|
return this._neural;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return this._neural;
|
||||||
|
/**
|
||||||
|
* Neural API - Advanced AI operations
|
||||||
|
*/
|
||||||
|
neural();
|
||||||
|
{
|
||||||
|
if (!this._neural) {
|
||||||
|
this._neural = new ImprovedNeuralAPI(this);
|
||||||
|
}
|
||||||
|
return this._neural;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Natural Language Processing API
|
||||||
|
*/
|
||||||
|
nlp();
|
||||||
|
{
|
||||||
|
if (!this._nlp) {
|
||||||
|
this._nlp = new NaturalLanguageProcessor(this);
|
||||||
|
}
|
||||||
|
return this._nlp;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Create a streaming pipeline
|
||||||
|
*/
|
||||||
|
stream();
|
||||||
|
{
|
||||||
|
const { Pipeline } = require('./streaming/pipeline.js');
|
||||||
|
return new Pipeline(this);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get insights about the data
|
||||||
|
*/
|
||||||
|
async;
|
||||||
|
insights();
|
||||||
|
Promise < {
|
||||||
|
entities: number,
|
||||||
|
relationships: number,
|
||||||
|
types: (Record),
|
||||||
|
services: string[],
|
||||||
|
density: number
|
||||||
|
} > {
|
||||||
|
await, this: .ensureInitialized()
|
||||||
|
// Get all entities count
|
||||||
|
,
|
||||||
|
// Get all entities count
|
||||||
|
const: allEntities = await this.storage.getAllNouns(),
|
||||||
|
const: entities = allEntities.length
|
||||||
|
// Get relationships count
|
||||||
|
,
|
||||||
|
// Get relationships count
|
||||||
|
const: allVerbs = await this.storage.getAllVerbs(),
|
||||||
|
const: relationships = allVerbs.length
|
||||||
|
// Count by type
|
||||||
|
,
|
||||||
|
// Count by type
|
||||||
|
const: types
|
||||||
|
};
|
||||||
|
{ }
|
||||||
|
for (const entity of allEntities) {
|
||||||
|
const type = entity.metadata?.noun || 'unknown';
|
||||||
|
types[type] = (types[type] || 0) + 1;
|
||||||
|
}
|
||||||
|
// Get unique services
|
||||||
|
const services = [...new Set(allEntities.map(e => e.metadata?.service).filter(Boolean))];
|
||||||
|
// Calculate density (relationships per entity)
|
||||||
|
const density = entities > 0 ? relationships / entities : 0;
|
||||||
|
return {
|
||||||
|
entities,
|
||||||
|
relationships,
|
||||||
|
types,
|
||||||
|
services,
|
||||||
|
density
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Augmentations API - Clean and simple
|
||||||
|
*/
|
||||||
|
get;
|
||||||
|
augmentations();
|
||||||
|
{
|
||||||
|
return {
|
||||||
|
list: () => this.augmentationRegistry.getAll().map(a => a.name),
|
||||||
|
get: (name) => this.augmentationRegistry.getAll().find(a => a.name === name),
|
||||||
|
has: (name) => this.augmentationRegistry.getAll().some(a => a.name === name)
|
||||||
|
};
|
||||||
|
}
|
||||||
|
async;
|
||||||
|
parseNaturalQuery(query, string);
|
||||||
|
Promise < FindParams < T >> {
|
||||||
|
: ._nlp
|
||||||
|
};
|
||||||
|
{
|
||||||
|
this._nlp = new NaturalLanguageProcessor(this);
|
||||||
|
}
|
||||||
|
const parsed = await this._nlp.processNaturalQuery(query);
|
||||||
|
return parsed;
|
||||||
|
async;
|
||||||
|
applyGraphConstraints(results, Result < T > [], constraints, any);
|
||||||
|
Promise < Result < T > [] > {
|
||||||
|
// Filter by graph connections
|
||||||
|
if(constraints) { }, : .to || constraints.from
|
||||||
|
};
|
||||||
|
{
|
||||||
|
const filtered = [];
|
||||||
|
for (const result of results) {
|
||||||
|
if (constraints.to) {
|
||||||
|
const verbs = await this.storage.getVerbsBySource(result.id);
|
||||||
|
const hasConnection = verbs.some(v => v.targetId === constraints.to);
|
||||||
|
if (hasConnection) {
|
||||||
|
filtered.push(result);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (constraints.from) {
|
||||||
|
const verbs = await this.storage.getVerbsByTarget(result.id);
|
||||||
|
const hasConnection = verbs.some(v => v.sourceId === constraints.from);
|
||||||
|
if (hasConnection) {
|
||||||
|
filtered.push(result);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return filtered;
|
||||||
|
}
|
||||||
|
return results;
|
||||||
|
verbsToRelations(verbs, GraphVerb[]);
|
||||||
|
Relation < T > [];
|
||||||
|
{
|
||||||
|
return verbs.map((v) => ({
|
||||||
|
id: v.id,
|
||||||
|
from: v.sourceId,
|
||||||
|
to: v.targetId,
|
||||||
|
type: (v.verb || v.type),
|
||||||
|
weight: v.weight,
|
||||||
|
metadata: v.metadata,
|
||||||
|
service: v.metadata?.service,
|
||||||
|
createdAt: typeof v.createdAt === 'number' ? v.createdAt : Date.now()
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
async;
|
||||||
|
embed(data, any);
|
||||||
|
Promise < Vector > {
|
||||||
|
return: this.embedder(data)
|
||||||
|
};
|
||||||
|
async;
|
||||||
|
warmup();
|
||||||
|
Promise < void > {
|
||||||
|
// Warm up embedder
|
||||||
|
await, this: .embed('warmup')
|
||||||
|
};
|
||||||
|
setupEmbedder();
|
||||||
|
EmbeddingFunction;
|
||||||
|
{
|
||||||
|
if (this.config.model?.type === 'custom' && this.config.model.name) {
|
||||||
|
// TODO: Load custom model
|
||||||
|
return defaultEmbeddingFunction;
|
||||||
|
}
|
||||||
|
return defaultEmbeddingFunction;
|
||||||
|
}
|
||||||
|
async;
|
||||||
|
setupStorage();
|
||||||
|
Promise < StorageAdapter > {
|
||||||
|
const: storage = await createStorage({
|
||||||
|
type: this.config.storage?.type || 'memory',
|
||||||
|
...this.config.storage?.options
|
||||||
|
}),
|
||||||
|
return: storage
|
||||||
|
};
|
||||||
|
setupIndex();
|
||||||
|
HNSWIndex | HNSWIndexOptimized;
|
||||||
|
{
|
||||||
|
const indexConfig = {
|
||||||
|
...this.config.index,
|
||||||
|
distanceFunction: this.distance
|
||||||
|
};
|
||||||
|
// Use optimized index for larger datasets
|
||||||
|
if (this.config.storage?.type !== 'memory') {
|
||||||
|
return new HNSWIndexOptimized(indexConfig, this.distance, this.storage);
|
||||||
|
}
|
||||||
|
return new HNSWIndex(indexConfig);
|
||||||
|
}
|
||||||
|
setupAugmentations();
|
||||||
|
AugmentationRegistry;
|
||||||
|
{
|
||||||
|
const registry = new AugmentationRegistry();
|
||||||
|
// Register default augmentations
|
||||||
|
const defaults = createDefaultAugmentations(this.config.augmentations);
|
||||||
|
for (const aug of defaults) {
|
||||||
|
registry.register(aug);
|
||||||
|
}
|
||||||
|
return registry;
|
||||||
|
}
|
||||||
|
normalizeConfig(config ? : BrainyConfig);
|
||||||
|
Required < BrainyConfig > {
|
||||||
|
return: {
|
||||||
|
storage: config?.storage || { type: 'memory' },
|
||||||
|
model: config?.model || { type: 'fast' },
|
||||||
|
index: config?.index || {},
|
||||||
|
cache: config?.cache ?? true,
|
||||||
|
augmentations: config?.augmentations || {},
|
||||||
|
warmup: config?.warmup ?? false,
|
||||||
|
realtime: config?.realtime ?? false,
|
||||||
|
multiTenancy: config?.multiTenancy ?? false,
|
||||||
|
telemetry: config?.telemetry ?? false
|
||||||
|
}
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Close and cleanup
|
||||||
|
*/
|
||||||
|
async;
|
||||||
|
close();
|
||||||
|
Promise < void > {
|
||||||
|
// Shutdown augmentations
|
||||||
|
const: augs = this.augmentationRegistry.getAll(),
|
||||||
|
for(, aug, of, augs) {
|
||||||
|
if ('shutdown' in aug && typeof aug.shutdown === 'function') {
|
||||||
|
await aug.shutdown();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Storage doesn't have close in current interface
|
||||||
|
// We'll just mark as not initialized
|
||||||
|
,
|
||||||
|
// Storage doesn't have close in current interface
|
||||||
|
// We'll just mark as not initialized
|
||||||
|
this: .initialized = false
|
||||||
|
};
|
||||||
|
// Re-export types for convenience
|
||||||
|
export * from './types/brainy.types.js';
|
||||||
|
export { NounType, VerbType } from './types/graphTypes.js';
|
||||||
|
//# sourceMappingURL=brainy.js.map
|
||||||
927
.recovery-workspace/brainy.recovered-20250910-142457.ts
Normal file
927
.recovery-workspace/brainy.recovered-20250910-142457.ts
Normal file
|
|
@ -0,0 +1,927 @@
|
||||||
|
/**
|
||||||
|
* 🧠 Brainy 3.0 - The Future of Neural Databases
|
||||||
|
*
|
||||||
|
* Beautiful, Professional, Planet-Scale, Fun to Use
|
||||||
|
* NO STUBS, NO MOCKS, REAL IMPLEMENTATION
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { v4 as uuidv4 } from './universal/uuid.js'
|
||||||
|
import { HNSWIndex } from './hnsw/hnswIndex.js'
|
||||||
|
import { HNSWIndexOptimized } from './hnsw/hnswIndexOptimized.js'
|
||||||
|
import { createStorage } from './storage/storageFactory.js'
|
||||||
|
import { StorageAdapter, Vector, DistanceFunction, EmbeddingFunction, GraphVerb } from './coreTypes.js'
|
||||||
|
import {
|
||||||
|
defaultEmbeddingFunction,
|
||||||
|
cosineDistance
|
||||||
|
} from './utils/index.js'
|
||||||
|
import { validateNounType, validateVerbType } from './utils/typeValidation.js'
|
||||||
|
import { matchesMetadataFilter } from './utils/metadataFilter.js'
|
||||||
|
import { AugmentationRegistry, AugmentationContext } from './augmentations/brainyAugmentation.js'
|
||||||
|
import { createDefaultAugmentations } from './augmentations/defaultAugmentations.js'
|
||||||
|
import { ImprovedNeuralAPI } from './neural/improvedNeuralAPI.js'
|
||||||
|
import { NaturalLanguageProcessor } from './neural/naturalLanguageProcessor.js'
|
||||||
|
import { TripleIntelligenceSystem } from './triple/TripleIntelligenceSystem.js'
|
||||||
|
import {
|
||||||
|
Entity,
|
||||||
|
Relation,
|
||||||
|
Result,
|
||||||
|
AddParams,
|
||||||
|
UpdateParams,
|
||||||
|
RelateParams,
|
||||||
|
FindParams,
|
||||||
|
SimilarParams,
|
||||||
|
GetRelationsParams,
|
||||||
|
AddManyParams,
|
||||||
|
DeleteManyParams,
|
||||||
|
BatchResult,
|
||||||
|
BrainyConfig
|
||||||
|
} from './types/brainy.types.js'
|
||||||
|
import { NounType, VerbType } from './types/graphTypes.js'
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The main Brainy class - Clean, Beautiful, Powerful
|
||||||
|
* REAL IMPLEMENTATION - No stubs, no mocks
|
||||||
|
*/
|
||||||
|
export class Brainy<T = any> {
|
||||||
|
// Core components
|
||||||
|
private index!: HNSWIndex | HNSWIndexOptimized
|
||||||
|
private storage!: StorageAdapter
|
||||||
|
private embedder: EmbeddingFunction
|
||||||
|
private distance: DistanceFunction
|
||||||
|
private augmentationRegistry: AugmentationRegistry
|
||||||
|
private config: Required<BrainyConfig>
|
||||||
|
|
||||||
|
// Sub-APIs (lazy-loaded)
|
||||||
|
private _neural?: ImprovedNeuralAPI
|
||||||
|
private _nlp?: NaturalLanguageProcessor
|
||||||
|
private _tripleIntelligence?: TripleIntelligenceSystem
|
||||||
|
|
||||||
|
// State
|
||||||
|
private initialized = false
|
||||||
|
private dimensions?: number
|
||||||
|
|
||||||
|
constructor(config?: BrainyConfig) {
|
||||||
|
// Normalize configuration with defaults
|
||||||
|
this.config = this.normalizeConfig(config)
|
||||||
|
|
||||||
|
// Setup core components
|
||||||
|
this.distance = cosineDistance
|
||||||
|
this.embedder = this.setupEmbedder()
|
||||||
|
this.augmentationRegistry = this.setupAugmentations()
|
||||||
|
|
||||||
|
// Index and storage are initialized in init() because they may need each other
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Initialize Brainy - MUST be called before use
|
||||||
|
* @param overrides Optional configuration overrides for init
|
||||||
|
*/
|
||||||
|
async init(overrides?: Partial<BrainyConfig & { dimensions?: number }>): Promise<void> {
|
||||||
|
if (this.initialized) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply any init-time configuration overrides
|
||||||
|
if (overrides) {
|
||||||
|
const { dimensions, ...configOverrides } = overrides
|
||||||
|
this.config = {
|
||||||
|
...this.config,
|
||||||
|
...configOverrides,
|
||||||
|
storage: { ...this.config.storage, ...configOverrides.storage },
|
||||||
|
model: { ...this.config.model, ...configOverrides.model },
|
||||||
|
index: { ...this.config.index, ...configOverrides.index },
|
||||||
|
augmentations: { ...this.config.augmentations, ...configOverrides.augmentations }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set dimensions if provided
|
||||||
|
if (dimensions) {
|
||||||
|
this.dimensions = dimensions
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Setup and initialize storage
|
||||||
|
this.storage = await this.setupStorage()
|
||||||
|
await this.storage.init()
|
||||||
|
|
||||||
|
// Setup index now that we have storage
|
||||||
|
this.index = this.setupIndex()
|
||||||
|
|
||||||
|
// Initialize augmentations
|
||||||
|
await this.augmentationRegistry.initializeAll({
|
||||||
|
brain: this,
|
||||||
|
storage: this.storage,
|
||||||
|
config: this.config,
|
||||||
|
log: (message: string, level = 'info') => {
|
||||||
|
// Simple logging for now
|
||||||
|
if (level === 'error') {
|
||||||
|
console.error(message)
|
||||||
|
} else if (level === 'warn') {
|
||||||
|
console.warn(message)
|
||||||
|
} else {
|
||||||
|
console.log(message)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// Warm up if configured
|
||||||
|
if (this.config.warmup) {
|
||||||
|
await this.warmup()
|
||||||
|
}
|
||||||
|
|
||||||
|
this.initialized = true
|
||||||
|
} catch (error) {
|
||||||
|
throw new Error(`Failed to initialize Brainy: ${error}`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ensure Brainy is initialized
|
||||||
|
*/
|
||||||
|
private async ensureInitialized(): Promise<void> {
|
||||||
|
if (!this.initialized) {
|
||||||
|
throw new Error('Brainy not initialized. Call init() first.')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============= CORE CRUD OPERATIONS =============
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Add an entity to the database
|
||||||
|
*/
|
||||||
|
async add(params: AddParams<T>): Promise<string> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
// Validate parameters
|
||||||
|
if (!params.data && !params.vector) {
|
||||||
|
throw new Error('Either data or vector is required')
|
||||||
|
}
|
||||||
|
validateNounType(params.type)
|
||||||
|
|
||||||
|
// Generate ID if not provided
|
||||||
|
const id = params.id || uuidv4()
|
||||||
|
|
||||||
|
// Get or compute vector
|
||||||
|
const vector = params.vector || (await this.embed(params.data))
|
||||||
|
|
||||||
|
// Ensure dimensions are set
|
||||||
|
if (!this.dimensions) {
|
||||||
|
this.dimensions = vector.length
|
||||||
|
} else if (vector.length !== this.dimensions) {
|
||||||
|
throw new Error(
|
||||||
|
`Vector dimension mismatch: expected ${this.dimensions}, got ${vector.length}`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Execute through augmentation pipeline
|
||||||
|
return this.augmentationRegistry.execute('add', params, async () => {
|
||||||
|
// Add to index
|
||||||
|
await this.index.addItem({ id, vector })
|
||||||
|
|
||||||
|
// Save to storage
|
||||||
|
await this.storage.saveNoun({
|
||||||
|
id,
|
||||||
|
vector,
|
||||||
|
connections: new Map(),
|
||||||
|
level: 0,
|
||||||
|
metadata: {
|
||||||
|
...params.metadata,
|
||||||
|
noun: params.type,
|
||||||
|
service: params.service,
|
||||||
|
createdAt: Date.now()
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
return id
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get an entity by ID
|
||||||
|
*/
|
||||||
|
async get(id: string): Promise<Entity<T> | null> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
return this.augmentationRegistry.execute('get', { id }, async () => {
|
||||||
|
// Get from storage
|
||||||
|
const noun = await this.storage.getNoun(id)
|
||||||
|
if (!noun) {
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract metadata - separate user metadata from system metadata
|
||||||
|
const { noun: nounType, service, createdAt, updatedAt, ...userMetadata } = noun.metadata || {}
|
||||||
|
|
||||||
|
// Construct entity from noun
|
||||||
|
const entity: Entity<T> = {
|
||||||
|
id: noun.id,
|
||||||
|
vector: noun.vector,
|
||||||
|
type: (nounType as NounType) || NounType.Thing,
|
||||||
|
metadata: userMetadata as T,
|
||||||
|
service: service as string,
|
||||||
|
createdAt: (createdAt as number) || Date.now(),
|
||||||
|
updatedAt: updatedAt as number
|
||||||
|
}
|
||||||
|
|
||||||
|
return entity
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Update an entity
|
||||||
|
*/
|
||||||
|
async update(params: UpdateParams<T>): Promise<void> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
if (!params.id) {
|
||||||
|
throw new Error('ID is required for update')
|
||||||
|
}
|
||||||
|
|
||||||
|
return this.augmentationRegistry.execute('update', params, async () => {
|
||||||
|
// Get existing entity
|
||||||
|
const existing = await this.get(params.id)
|
||||||
|
if (!existing) {
|
||||||
|
throw new Error(`Entity ${params.id} not found`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update vector if data changed
|
||||||
|
let vector = existing.vector
|
||||||
|
if (params.data) {
|
||||||
|
vector = params.vector || (await this.embed(params.data))
|
||||||
|
// Update in index (remove and re-add since no update method)
|
||||||
|
await this.index.removeItem(params.id)
|
||||||
|
await this.index.addItem({ id: params.id, vector })
|
||||||
|
}
|
||||||
|
|
||||||
|
// Always update the noun with new metadata
|
||||||
|
const newMetadata = params.merge !== false
|
||||||
|
? { ...existing.metadata, ...params.metadata }
|
||||||
|
: params.metadata || existing.metadata
|
||||||
|
|
||||||
|
await this.storage.saveNoun({
|
||||||
|
id: params.id,
|
||||||
|
vector,
|
||||||
|
connections: new Map(),
|
||||||
|
level: 0,
|
||||||
|
metadata: {
|
||||||
|
...newMetadata,
|
||||||
|
noun: params.type || existing.type,
|
||||||
|
service: existing.service,
|
||||||
|
createdAt: existing.createdAt,
|
||||||
|
updatedAt: Date.now()
|
||||||
|
}
|
||||||
|
})
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Delete an entity
|
||||||
|
*/
|
||||||
|
async delete(id: string): Promise<void> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
return this.augmentationRegistry.execute('delete', { id }, async () => {
|
||||||
|
// Remove from index
|
||||||
|
await this.index.removeItem(id)
|
||||||
|
|
||||||
|
// Delete from storage
|
||||||
|
await this.storage.deleteNoun(id)
|
||||||
|
|
||||||
|
// Delete metadata (if it exists as separate)
|
||||||
|
try {
|
||||||
|
await this.storage.saveMetadata(id, null as any) // Clear metadata
|
||||||
|
} catch {
|
||||||
|
// Ignore if not supported
|
||||||
|
}
|
||||||
|
|
||||||
|
// Delete related verbs
|
||||||
|
const verbs = await this.storage.getVerbsBySource(id)
|
||||||
|
const targetVerbs = await this.storage.getVerbsByTarget(id)
|
||||||
|
const allVerbs = [...verbs, ...targetVerbs]
|
||||||
|
|
||||||
|
for (const verb of allVerbs) {
|
||||||
|
await this.storage.deleteVerb(verb.id)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============= RELATIONSHIP OPERATIONS =============
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Create a relationship between entities
|
||||||
|
*/
|
||||||
|
async relate(params: RelateParams<T>): Promise<string> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
// Validate parameters
|
||||||
|
if (!params.from || !params.to) {
|
||||||
|
throw new Error('Both from and to are required')
|
||||||
|
}
|
||||||
|
validateVerbType(params.type)
|
||||||
|
|
||||||
|
// Verify entities exist
|
||||||
|
const fromEntity = await this.get(params.from)
|
||||||
|
const toEntity = await this.get(params.to)
|
||||||
|
|
||||||
|
if (!fromEntity) {
|
||||||
|
throw new Error(`Source entity ${params.from} not found`)
|
||||||
|
}
|
||||||
|
if (!toEntity) {
|
||||||
|
throw new Error(`Target entity ${params.to} not found`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Generate ID
|
||||||
|
const id = uuidv4()
|
||||||
|
|
||||||
|
// Compute relationship vector (average of entities)
|
||||||
|
const relationVector = fromEntity.vector.map(
|
||||||
|
(v, i) => (v + toEntity.vector[i]) / 2
|
||||||
|
)
|
||||||
|
|
||||||
|
return this.augmentationRegistry.execute('relate', params, async () => {
|
||||||
|
// Save to storage
|
||||||
|
const verb: GraphVerb = {
|
||||||
|
id,
|
||||||
|
vector: relationVector,
|
||||||
|
sourceId: params.from,
|
||||||
|
targetId: params.to,
|
||||||
|
source: fromEntity.type,
|
||||||
|
target: toEntity.type,
|
||||||
|
verb: params.type,
|
||||||
|
type: params.type,
|
||||||
|
weight: params.weight ?? 1.0,
|
||||||
|
metadata: params.metadata as any,
|
||||||
|
createdAt: Date.now()
|
||||||
|
} as any
|
||||||
|
|
||||||
|
await this.storage.saveVerb(verb)
|
||||||
|
|
||||||
|
// Create bidirectional if requested
|
||||||
|
if (params.bidirectional) {
|
||||||
|
const reverseId = uuidv4()
|
||||||
|
const reverseVerb: GraphVerb = {
|
||||||
|
...verb,
|
||||||
|
id: reverseId,
|
||||||
|
sourceId: params.to,
|
||||||
|
targetId: params.from,
|
||||||
|
source: toEntity.type,
|
||||||
|
target: fromEntity.type
|
||||||
|
} as any
|
||||||
|
|
||||||
|
await this.storage.saveVerb(reverseVerb)
|
||||||
|
}
|
||||||
|
|
||||||
|
return id
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Delete a relationship
|
||||||
|
*/
|
||||||
|
async unrelate(id: string): Promise<void> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
return this.augmentationRegistry.execute('unrelate', { id }, async () => {
|
||||||
|
await this.storage.deleteVerb(id)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get relationships
|
||||||
|
*/
|
||||||
|
async getRelations(
|
||||||
|
params: GetRelationsParams = {}
|
||||||
|
): Promise<Relation<T>[]> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
const relations: Relation<T>[] = []
|
||||||
|
|
||||||
|
if (params.from) {
|
||||||
|
const verbs = await this.storage.getVerbsBySource(params.from)
|
||||||
|
relations.push(...this.verbsToRelations(verbs))
|
||||||
|
}
|
||||||
|
|
||||||
|
if (params.to) {
|
||||||
|
const verbs = await this.storage.getVerbsByTarget(params.to)
|
||||||
|
relations.push(...this.verbsToRelations(verbs))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Filter by type
|
||||||
|
let filtered = relations
|
||||||
|
if (params.type) {
|
||||||
|
const types = Array.isArray(params.type) ? params.type : [params.type]
|
||||||
|
filtered = relations.filter((r) => types.includes(r.type))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Filter by service
|
||||||
|
if (params.service) {
|
||||||
|
filtered = filtered.filter((r) => r.service === params.service)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply pagination
|
||||||
|
const limit = params.limit || 100
|
||||||
|
const offset = params.offset || 0
|
||||||
|
return filtered.slice(offset, offset + limit)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============= SEARCH & DISCOVERY =============
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unified find method - supports natural language and structured queries
|
||||||
|
*/
|
||||||
|
async find(query: string | FindParams<T>): Promise<Result<T>[]> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
// Parse natural language queries
|
||||||
|
const params: FindParams<T> =
|
||||||
|
typeof query === 'string' ? await this.parseNaturalQuery(query) : query
|
||||||
|
|
||||||
|
return this.augmentationRegistry.execute('find', params, async () => {
|
||||||
|
let results: Result<T>[] = []
|
||||||
|
|
||||||
|
// Vector search
|
||||||
|
if (params.query || params.vector) {
|
||||||
|
const vector = params.vector || (await this.embed(params.query!))
|
||||||
|
const limit = params.limit || 10
|
||||||
|
|
||||||
|
// Search index - returns array of [id, score] tuples
|
||||||
|
const searchResults = await this.index.search(vector, limit * 2) // Get extra for filtering
|
||||||
|
|
||||||
|
// Hydrate results
|
||||||
|
for (const [id, score] of searchResults) {
|
||||||
|
const entity = await this.get(id)
|
||||||
|
if (entity) {
|
||||||
|
results.push({
|
||||||
|
id,
|
||||||
|
score,
|
||||||
|
entity
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Proximity search
|
||||||
|
if (params.near) {
|
||||||
|
const nearEntity = await this.get(params.near.id)
|
||||||
|
if (nearEntity) {
|
||||||
|
const nearResults = await this.index.search(
|
||||||
|
nearEntity.vector,
|
||||||
|
params.limit || 10
|
||||||
|
)
|
||||||
|
|
||||||
|
for (const [id, score] of nearResults) {
|
||||||
|
if (score >= (params.near.threshold || 0.7)) {
|
||||||
|
const entity = await this.get(id)
|
||||||
|
if (entity) {
|
||||||
|
results.push({
|
||||||
|
id,
|
||||||
|
score,
|
||||||
|
entity
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply filters
|
||||||
|
if (params.where) {
|
||||||
|
results = results.filter((r) =>
|
||||||
|
matchesMetadataFilter(r.entity.metadata, params.where!)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if (params.type) {
|
||||||
|
const types = Array.isArray(params.type) ? params.type : [params.type]
|
||||||
|
results = results.filter((r) => types.includes(r.entity.type))
|
||||||
|
}
|
||||||
|
|
||||||
|
if (params.service) {
|
||||||
|
results = results.filter((r) => r.entity.service === params.service)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Graph constraints
|
||||||
|
if (params.connected) {
|
||||||
|
results = await this.applyGraphConstraints(results, params.connected)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sort by score and limit
|
||||||
|
results.sort((a, b) => b.score - a.score)
|
||||||
|
const limit = params.limit || 10
|
||||||
|
const offset = params.offset || 0
|
||||||
|
|
||||||
|
return results.slice(offset, offset + limit)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Find similar entities
|
||||||
|
*/
|
||||||
|
async similar(params: SimilarParams<T>): Promise<Result<T>[]> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
// Get target vector
|
||||||
|
let targetVector: Vector
|
||||||
|
|
||||||
|
if (typeof params.to === 'string') {
|
||||||
|
const entity = await this.get(params.to)
|
||||||
|
if (!entity) {
|
||||||
|
throw new Error(`Entity ${params.to} not found`)
|
||||||
|
}
|
||||||
|
targetVector = entity.vector
|
||||||
|
} else if (Array.isArray(params.to)) {
|
||||||
|
targetVector = params.to as Vector
|
||||||
|
} else {
|
||||||
|
targetVector = (params.to as Entity<T>).vector
|
||||||
|
}
|
||||||
|
|
||||||
|
// Use find with vector
|
||||||
|
return this.find({
|
||||||
|
vector: targetVector,
|
||||||
|
limit: params.limit,
|
||||||
|
type: params.type,
|
||||||
|
where: params.where,
|
||||||
|
service: params.service
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============= BATCH OPERATIONS =============
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Add multiple entities
|
||||||
|
*/
|
||||||
|
async addMany(params: AddManyParams<T>): Promise<BatchResult<string>> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
const result: BatchResult<string> = {
|
||||||
|
successful: [],
|
||||||
|
failed: [],
|
||||||
|
total: params.items.length,
|
||||||
|
duration: 0
|
||||||
|
}
|
||||||
|
|
||||||
|
const startTime = Date.now()
|
||||||
|
const chunkSize = params.chunkSize || 100
|
||||||
|
|
||||||
|
// Process in chunks
|
||||||
|
for (let i = 0; i < params.items.length; i += chunkSize) {
|
||||||
|
const chunk = params.items.slice(i, i + chunkSize)
|
||||||
|
|
||||||
|
const promises = chunk.map(async (item) => {
|
||||||
|
try {
|
||||||
|
const id = await this.add(item)
|
||||||
|
result.successful.push(id)
|
||||||
|
} catch (error) {
|
||||||
|
result.failed.push({
|
||||||
|
item,
|
||||||
|
error: (error as Error).message
|
||||||
|
})
|
||||||
|
if (!params.continueOnError) {
|
||||||
|
throw error
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
if (params.parallel !== false) {
|
||||||
|
await Promise.allSettled(promises)
|
||||||
|
} else {
|
||||||
|
for (const promise of promises) {
|
||||||
|
await promise
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Report progress
|
||||||
|
if (params.onProgress) {
|
||||||
|
params.onProgress(
|
||||||
|
result.successful.length + result.failed.length,
|
||||||
|
result.total
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
result.duration = Date.now() - startTime
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Delete multiple entities
|
||||||
|
*/
|
||||||
|
async deleteMany(params: DeleteManyParams): Promise<BatchResult<string>> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
// Determine what to delete
|
||||||
|
let idsToDelete: string[] = []
|
||||||
|
|
||||||
|
if (params.ids) {
|
||||||
|
idsToDelete = params.ids
|
||||||
|
} else if (params.type || params.where) {
|
||||||
|
// Find entities to delete
|
||||||
|
const entities = await this.find({
|
||||||
|
type: params.type,
|
||||||
|
where: params.where,
|
||||||
|
limit: params.limit || 1000
|
||||||
|
})
|
||||||
|
idsToDelete = entities.map((e) => e.id)
|
||||||
|
}
|
||||||
|
|
||||||
|
const result: BatchResult<string> = {
|
||||||
|
successful: [],
|
||||||
|
failed: [],
|
||||||
|
total: idsToDelete.length,
|
||||||
|
duration: 0
|
||||||
|
}
|
||||||
|
|
||||||
|
const startTime = Date.now()
|
||||||
|
|
||||||
|
for (const id of idsToDelete) {
|
||||||
|
try {
|
||||||
|
await this.delete(id)
|
||||||
|
result.successful.push(id)
|
||||||
|
} catch (error) {
|
||||||
|
result.failed.push({
|
||||||
|
item: id,
|
||||||
|
error: (error as Error).message
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
if (params.onProgress) {
|
||||||
|
params.onProgress(
|
||||||
|
result.successful.length + result.failed.length,
|
||||||
|
result.total
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
result.duration = Date.now() - startTime
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============= SUB-APIS =============
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Neural API - Advanced AI operations
|
||||||
|
*/
|
||||||
|
neural(): ImprovedNeuralAPI {
|
||||||
|
if (!this._neural) {
|
||||||
|
this._neural = new ImprovedNeuralAPI(this as any)
|
||||||
|
}
|
||||||
|
return this._neural
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Natural Language Processing API
|
||||||
|
*/
|
||||||
|
nlp(): NaturalLanguageProcessor {
|
||||||
|
if (!this._nlp) {
|
||||||
|
this._nlp = new NaturalLanguageProcessor(this)
|
||||||
|
}
|
||||||
|
return this._nlp
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get Triple Intelligence System
|
||||||
|
* Advanced pattern recognition and relationship analysis
|
||||||
|
*/
|
||||||
|
getTripleIntelligence(): TripleIntelligenceSystem {
|
||||||
|
if (!this._tripleIntelligence) {
|
||||||
|
this._tripleIntelligence = new TripleIntelligenceSystem(this as any)
|
||||||
|
}
|
||||||
|
return this._tripleIntelligence
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Create a streaming pipeline
|
||||||
|
*/
|
||||||
|
stream() {
|
||||||
|
const { Pipeline } = require('./streaming/pipeline.js')
|
||||||
|
return new Pipeline(this)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get insights about the data
|
||||||
|
*/
|
||||||
|
async insights(): Promise<{
|
||||||
|
entities: number
|
||||||
|
relationships: number
|
||||||
|
types: Record<string, number>
|
||||||
|
services: string[]
|
||||||
|
density: number
|
||||||
|
}> {
|
||||||
|
await this.ensureInitialized()
|
||||||
|
|
||||||
|
// Get all entities count - use getNouns with high limit
|
||||||
|
const entitiesResult = await this.storage.getNouns({
|
||||||
|
pagination: { limit: 10000 }
|
||||||
|
})
|
||||||
|
const entities = entitiesResult.totalCount || entitiesResult.items.length
|
||||||
|
|
||||||
|
// Get relationships count - use getVerbs with high limit
|
||||||
|
const verbsResult = await this.storage.getVerbs({
|
||||||
|
pagination: { limit: 10000 }
|
||||||
|
})
|
||||||
|
const relationships = verbsResult.totalCount || verbsResult.items.length
|
||||||
|
|
||||||
|
// Count by type
|
||||||
|
const types: Record<string, number> = {}
|
||||||
|
for (const entity of entitiesResult.items) {
|
||||||
|
const type = (entity.metadata?.noun as string) || 'unknown'
|
||||||
|
types[type] = (types[type] || 0) + 1
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get unique services
|
||||||
|
const services = [...new Set(entitiesResult.items.map((e: any) => e.metadata?.service).filter(Boolean))] as string[]
|
||||||
|
|
||||||
|
// Calculate density (relationships per entity)
|
||||||
|
const density = entities > 0 ? relationships / entities : 0
|
||||||
|
|
||||||
|
return {
|
||||||
|
entities,
|
||||||
|
relationships,
|
||||||
|
types,
|
||||||
|
services,
|
||||||
|
density
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Augmentations API - Clean and simple
|
||||||
|
*/
|
||||||
|
get augmentations() {
|
||||||
|
return {
|
||||||
|
list: () => this.augmentationRegistry.getAll().map(a => a.name),
|
||||||
|
get: (name: string) => this.augmentationRegistry.getAll().find(a => a.name === name),
|
||||||
|
has: (name: string) => this.augmentationRegistry.getAll().some(a => a.name === name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ============= HELPER METHODS =============
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse natural language query
|
||||||
|
*/
|
||||||
|
private async parseNaturalQuery(query: string): Promise<FindParams<T>> {
|
||||||
|
if (!this._nlp) {
|
||||||
|
this._nlp = new NaturalLanguageProcessor(this as any)
|
||||||
|
}
|
||||||
|
const parsed = await this._nlp.processNaturalQuery(query)
|
||||||
|
return parsed as FindParams<T>
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Apply graph constraints to results
|
||||||
|
*/
|
||||||
|
private async applyGraphConstraints(
|
||||||
|
results: Result<T>[],
|
||||||
|
constraints: any
|
||||||
|
): Promise<Result<T>[]> {
|
||||||
|
// Filter by graph connections
|
||||||
|
if (constraints.to || constraints.from) {
|
||||||
|
const filtered: Result<T>[] = []
|
||||||
|
|
||||||
|
for (const result of results) {
|
||||||
|
if (constraints.to) {
|
||||||
|
const verbs = await this.storage.getVerbsBySource(result.id)
|
||||||
|
const hasConnection = verbs.some(v => v.targetId === constraints.to)
|
||||||
|
if (hasConnection) {
|
||||||
|
filtered.push(result)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (constraints.from) {
|
||||||
|
const verbs = await this.storage.getVerbsByTarget(result.id)
|
||||||
|
const hasConnection = verbs.some(v => v.sourceId === constraints.from)
|
||||||
|
if (hasConnection) {
|
||||||
|
filtered.push(result)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return filtered
|
||||||
|
}
|
||||||
|
|
||||||
|
return results
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Convert verbs to relations
|
||||||
|
*/
|
||||||
|
private verbsToRelations(verbs: GraphVerb[]): Relation<T>[] {
|
||||||
|
return verbs.map((v) => ({
|
||||||
|
id: v.id,
|
||||||
|
from: v.sourceId,
|
||||||
|
to: v.targetId,
|
||||||
|
type: (v.verb || v.type) as VerbType,
|
||||||
|
weight: v.weight,
|
||||||
|
metadata: v.metadata,
|
||||||
|
service: v.metadata?.service as string,
|
||||||
|
createdAt: typeof v.createdAt === 'number' ? v.createdAt : Date.now()
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Embed data into vector
|
||||||
|
*/
|
||||||
|
private async embed(data: any): Promise<Vector> {
|
||||||
|
return this.embedder(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Warm up the system
|
||||||
|
*/
|
||||||
|
private async warmup(): Promise<void> {
|
||||||
|
// Warm up embedder
|
||||||
|
await this.embed('warmup')
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Setup embedder
|
||||||
|
*/
|
||||||
|
private setupEmbedder(): EmbeddingFunction {
|
||||||
|
if (this.config.model?.type === 'custom' && this.config.model.name) {
|
||||||
|
// TODO: Load custom model
|
||||||
|
return defaultEmbeddingFunction
|
||||||
|
}
|
||||||
|
return defaultEmbeddingFunction
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Setup storage
|
||||||
|
*/
|
||||||
|
private async setupStorage(): Promise<StorageAdapter> {
|
||||||
|
const storage = await createStorage({
|
||||||
|
type: this.config.storage?.type || 'memory',
|
||||||
|
...this.config.storage?.options
|
||||||
|
})
|
||||||
|
return storage
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Setup index
|
||||||
|
*/
|
||||||
|
private setupIndex(): HNSWIndex | HNSWIndexOptimized {
|
||||||
|
const indexConfig = {
|
||||||
|
...this.config.index,
|
||||||
|
distanceFunction: this.distance
|
||||||
|
}
|
||||||
|
|
||||||
|
// Use optimized index for larger datasets
|
||||||
|
if (this.config.storage?.type !== 'memory') {
|
||||||
|
return new HNSWIndexOptimized(indexConfig, this.distance, this.storage)
|
||||||
|
}
|
||||||
|
|
||||||
|
return new HNSWIndex(indexConfig as any)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Setup augmentations
|
||||||
|
*/
|
||||||
|
private setupAugmentations(): AugmentationRegistry {
|
||||||
|
const registry = new AugmentationRegistry()
|
||||||
|
|
||||||
|
// Register default augmentations
|
||||||
|
const defaults = createDefaultAugmentations(this.config.augmentations)
|
||||||
|
for (const aug of defaults) {
|
||||||
|
registry.register(aug)
|
||||||
|
}
|
||||||
|
|
||||||
|
return registry
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Normalize configuration
|
||||||
|
*/
|
||||||
|
private normalizeConfig(config?: BrainyConfig): Required<BrainyConfig> {
|
||||||
|
return {
|
||||||
|
storage: config?.storage || { type: 'memory' },
|
||||||
|
model: config?.model || { type: 'fast' },
|
||||||
|
index: config?.index || {},
|
||||||
|
cache: config?.cache ?? true,
|
||||||
|
augmentations: config?.augmentations || {},
|
||||||
|
warmup: config?.warmup ?? false,
|
||||||
|
realtime: config?.realtime ?? false,
|
||||||
|
multiTenancy: config?.multiTenancy ?? false,
|
||||||
|
telemetry: config?.telemetry ?? false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Close and cleanup
|
||||||
|
*/
|
||||||
|
async close(): Promise<void> {
|
||||||
|
// Shutdown augmentations
|
||||||
|
const augs = this.augmentationRegistry.getAll()
|
||||||
|
for (const aug of augs) {
|
||||||
|
if ('shutdown' in aug && typeof aug.shutdown === 'function') {
|
||||||
|
await aug.shutdown()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Storage doesn't have close in current interface
|
||||||
|
// We'll just mark as not initialized
|
||||||
|
this.initialized = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Re-export types for convenience
|
||||||
|
export * from './types/brainy.types.js'
|
||||||
|
export { NounType, VerbType } from './types/graphTypes.js'
|
||||||
67
.recovery-workspace/dist-backup-20250910-141917/api/ConfigAPI.d.ts
vendored
Normal file
67
.recovery-workspace/dist-backup-20250910-141917/api/ConfigAPI.d.ts
vendored
Normal file
|
|
@ -0,0 +1,67 @@
|
||||||
|
/**
|
||||||
|
* Configuration API for Brainy 3.0
|
||||||
|
* Provides configuration storage with optional encryption
|
||||||
|
*/
|
||||||
|
import { StorageAdapter } from '../coreTypes.js';
|
||||||
|
export interface ConfigOptions {
|
||||||
|
encrypt?: boolean;
|
||||||
|
decrypt?: boolean;
|
||||||
|
}
|
||||||
|
export interface ConfigEntry {
|
||||||
|
key: string;
|
||||||
|
value: any;
|
||||||
|
encrypted: boolean;
|
||||||
|
createdAt: number;
|
||||||
|
updatedAt: number;
|
||||||
|
}
|
||||||
|
export declare class ConfigAPI {
|
||||||
|
private storage;
|
||||||
|
private security;
|
||||||
|
private configCache;
|
||||||
|
private CONFIG_NOUN_PREFIX;
|
||||||
|
constructor(storage: StorageAdapter);
|
||||||
|
/**
|
||||||
|
* Set a configuration value with optional encryption
|
||||||
|
*/
|
||||||
|
set(params: {
|
||||||
|
key: string;
|
||||||
|
value: any;
|
||||||
|
encrypt?: boolean;
|
||||||
|
}): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Get a configuration value with optional decryption
|
||||||
|
*/
|
||||||
|
get(params: {
|
||||||
|
key: string;
|
||||||
|
decrypt?: boolean;
|
||||||
|
defaultValue?: any;
|
||||||
|
}): Promise<any>;
|
||||||
|
/**
|
||||||
|
* Delete a configuration value
|
||||||
|
*/
|
||||||
|
delete(key: string): Promise<void>;
|
||||||
|
/**
|
||||||
|
* List all configuration keys
|
||||||
|
*/
|
||||||
|
list(): Promise<string[]>;
|
||||||
|
/**
|
||||||
|
* Check if a configuration key exists
|
||||||
|
*/
|
||||||
|
has(key: string): Promise<boolean>;
|
||||||
|
/**
|
||||||
|
* Clear all configuration
|
||||||
|
*/
|
||||||
|
clear(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Export all configuration
|
||||||
|
*/
|
||||||
|
export(): Promise<Record<string, ConfigEntry>>;
|
||||||
|
/**
|
||||||
|
* Import configuration
|
||||||
|
*/
|
||||||
|
import(config: Record<string, ConfigEntry>): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Get raw config entry (without decryption)
|
||||||
|
*/
|
||||||
|
private getEntry;
|
||||||
|
}
|
||||||
166
.recovery-workspace/dist-backup-20250910-141917/api/ConfigAPI.js
Normal file
166
.recovery-workspace/dist-backup-20250910-141917/api/ConfigAPI.js
Normal file
|
|
@ -0,0 +1,166 @@
|
||||||
|
/**
|
||||||
|
* Configuration API for Brainy 3.0
|
||||||
|
* Provides configuration storage with optional encryption
|
||||||
|
*/
|
||||||
|
import { SecurityAPI } from './SecurityAPI.js';
|
||||||
|
export class ConfigAPI {
|
||||||
|
constructor(storage) {
|
||||||
|
this.storage = storage;
|
||||||
|
this.configCache = new Map();
|
||||||
|
this.CONFIG_NOUN_PREFIX = '_config_';
|
||||||
|
this.security = new SecurityAPI();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Set a configuration value with optional encryption
|
||||||
|
*/
|
||||||
|
async set(params) {
|
||||||
|
const { key, value, encrypt = false } = params;
|
||||||
|
// Serialize and optionally encrypt the value
|
||||||
|
let storedValue = value;
|
||||||
|
if (typeof value !== 'string') {
|
||||||
|
storedValue = JSON.stringify(value);
|
||||||
|
}
|
||||||
|
if (encrypt) {
|
||||||
|
storedValue = await this.security.encrypt(storedValue);
|
||||||
|
}
|
||||||
|
// Create config entry
|
||||||
|
const entry = {
|
||||||
|
key,
|
||||||
|
value: storedValue,
|
||||||
|
encrypted: encrypt,
|
||||||
|
createdAt: this.configCache.get(key)?.createdAt || Date.now(),
|
||||||
|
updatedAt: Date.now()
|
||||||
|
};
|
||||||
|
// Store in cache
|
||||||
|
this.configCache.set(key, entry);
|
||||||
|
// Persist to storage
|
||||||
|
const configId = this.CONFIG_NOUN_PREFIX + key;
|
||||||
|
await this.storage.saveMetadata(configId, entry);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get a configuration value with optional decryption
|
||||||
|
*/
|
||||||
|
async get(params) {
|
||||||
|
const { key, decrypt, defaultValue } = params;
|
||||||
|
// Check cache first
|
||||||
|
let entry = this.configCache.get(key);
|
||||||
|
// If not in cache, load from storage
|
||||||
|
if (!entry) {
|
||||||
|
const configId = this.CONFIG_NOUN_PREFIX + key;
|
||||||
|
const metadata = await this.storage.getMetadata(configId);
|
||||||
|
if (!metadata) {
|
||||||
|
return defaultValue;
|
||||||
|
}
|
||||||
|
entry = metadata;
|
||||||
|
this.configCache.set(key, entry);
|
||||||
|
}
|
||||||
|
let value = entry.value;
|
||||||
|
// Decrypt if needed
|
||||||
|
const shouldDecrypt = decrypt !== undefined ? decrypt : entry.encrypted;
|
||||||
|
if (shouldDecrypt && entry.encrypted && typeof value === 'string') {
|
||||||
|
value = await this.security.decrypt(value);
|
||||||
|
}
|
||||||
|
// Try to parse JSON if it looks like JSON
|
||||||
|
if (typeof value === 'string' && (value.startsWith('{') || value.startsWith('['))) {
|
||||||
|
try {
|
||||||
|
value = JSON.parse(value);
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
// Not JSON, return as string
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Delete a configuration value
|
||||||
|
*/
|
||||||
|
async delete(key) {
|
||||||
|
// Remove from cache
|
||||||
|
this.configCache.delete(key);
|
||||||
|
// Remove from storage
|
||||||
|
const configId = this.CONFIG_NOUN_PREFIX + key;
|
||||||
|
await this.storage.saveMetadata(configId, null);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* List all configuration keys
|
||||||
|
*/
|
||||||
|
async list() {
|
||||||
|
// Get all metadata keys from storage
|
||||||
|
const allMetadata = await this.storage.getMetadata('');
|
||||||
|
if (!allMetadata || typeof allMetadata !== 'object') {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
// Filter for config keys
|
||||||
|
const configKeys = [];
|
||||||
|
for (const key of Object.keys(allMetadata)) {
|
||||||
|
if (key.startsWith(this.CONFIG_NOUN_PREFIX)) {
|
||||||
|
configKeys.push(key.substring(this.CONFIG_NOUN_PREFIX.length));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return configKeys;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if a configuration key exists
|
||||||
|
*/
|
||||||
|
async has(key) {
|
||||||
|
if (this.configCache.has(key)) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
const configId = this.CONFIG_NOUN_PREFIX + key;
|
||||||
|
const metadata = await this.storage.getMetadata(configId);
|
||||||
|
return metadata !== null && metadata !== undefined;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear all configuration
|
||||||
|
*/
|
||||||
|
async clear() {
|
||||||
|
// Clear cache
|
||||||
|
this.configCache.clear();
|
||||||
|
// Clear from storage
|
||||||
|
const keys = await this.list();
|
||||||
|
for (const key of keys) {
|
||||||
|
await this.delete(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Export all configuration
|
||||||
|
*/
|
||||||
|
async export() {
|
||||||
|
const keys = await this.list();
|
||||||
|
const config = {};
|
||||||
|
for (const key of keys) {
|
||||||
|
const entry = await this.getEntry(key);
|
||||||
|
if (entry) {
|
||||||
|
config[key] = entry;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return config;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Import configuration
|
||||||
|
*/
|
||||||
|
async import(config) {
|
||||||
|
for (const [key, entry] of Object.entries(config)) {
|
||||||
|
this.configCache.set(key, entry);
|
||||||
|
const configId = this.CONFIG_NOUN_PREFIX + key;
|
||||||
|
await this.storage.saveMetadata(configId, entry);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get raw config entry (without decryption)
|
||||||
|
*/
|
||||||
|
async getEntry(key) {
|
||||||
|
if (this.configCache.has(key)) {
|
||||||
|
return this.configCache.get(key);
|
||||||
|
}
|
||||||
|
const configId = this.CONFIG_NOUN_PREFIX + key;
|
||||||
|
const metadata = await this.storage.getMetadata(configId);
|
||||||
|
if (!metadata) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const entry = metadata;
|
||||||
|
this.configCache.set(key, entry);
|
||||||
|
return entry;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=ConfigAPI.js.map
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
{"version":3,"file":"ConfigAPI.js","sourceRoot":"","sources":["../../src/api/ConfigAPI.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,EAAE,WAAW,EAAE,MAAM,kBAAkB,CAAA;AAgB9C,MAAM,OAAO,SAAS;IAKpB,YAAoB,OAAuB;QAAvB,YAAO,GAAP,OAAO,CAAgB;QAHnC,gBAAW,GAA6B,IAAI,GAAG,EAAE,CAAA;QACjD,uBAAkB,GAAG,UAAU,CAAA;QAGrC,IAAI,CAAC,QAAQ,GAAG,IAAI,WAAW,EAAE,CAAA;IACnC,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,GAAG,CAAC,MAIT;QACC,MAAM,EAAE,GAAG,EAAE,KAAK,EAAE,OAAO,GAAG,KAAK,EAAE,GAAG,MAAM,CAAA;QAE9C,6CAA6C;QAC7C,IAAI,WAAW,GAAQ,KAAK,CAAA;QAC5B,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;YAC9B,WAAW,GAAG,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,CAAA;QACrC,CAAC;QAED,IAAI,OAAO,EAAE,CAAC;YACZ,WAAW,GAAG,MAAM,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,WAAW,CAAC,CAAA;QACxD,CAAC;QAED,sBAAsB;QACtB,MAAM,KAAK,GAAgB;YACzB,GAAG;YACH,KAAK,EAAE,WAAW;YAClB,SAAS,EAAE,OAAO;YAClB,SAAS,EAAE,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,SAAS,IAAI,IAAI,CAAC,GAAG,EAAE;YAC7D,SAAS,EAAE,IAAI,CAAC,GAAG,EAAE;SACtB,CAAA;QAED,iBAAiB;QACjB,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAA;QAEhC,qBAAqB;QACrB,MAAM,QAAQ,GAAG,IAAI,CAAC,kBAAkB,GAAG,GAAG,CAAA;QAC9C,MAAM,IAAI,CAAC,OAAO,CAAC,YAAY,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAA;IAClD,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,GAAG,CAAC,MAIT;QACC,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,YAAY,EAAE,GAAG,MAAM,CAAA;QAE7C,oBAAoB;QACpB,IAAI,KAAK,GAAG,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,GAAG,CAAC,CAAA;QAErC,qCAAqC;QACrC,IAAI,CAAC,KAAK,EAAE,CAAC;YACX,MAAM,QAAQ,GAAG,IAAI,CAAC,kBAAkB,GAAG,GAAG,CAAA;YAC9C,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,WAAW,CAAC,QAAQ,CAAC,CAAA;YAEzD,IAAI,CAAC,QAAQ,EAAE,CAAC;gBACd,OAAO,YAAY,CAAA;YACrB,CAAC;YAED,KAAK,GAAG,QAAuB,CAAA;YAC/B,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAA;QAClC,CAAC;QAED,IAAI,KAAK,GAAG,KAAK,CAAC,KAAK,CAAA;QAEvB,oBAAoB;QACpB,MAAM,aAAa,GAAG,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,SAAS,CAAA;QACvE,IAAI,aAAa,IAAI,KAAK,CAAC,SAAS,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;YAClE,KAAK,GAAG,MAAM,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,CAAA;QAC5C,CAAC;QAED,0CAA0C;QAC1C,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,KAAK,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YAClF,IAAI,CAAC;gBACH,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,CAAA;YAC3B,CAAC;YAAC,MAAM,CAAC;gBACP,6BAA6B;YAC/B,CAAC;QACH,CAAC;QAED,OAAO,KAAK,CAAA;IACd,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,MAAM,CAAC,GAAW;QACtB,oBAAoB;QACpB,IAAI,CAAC,WAAW,CAAC,MAAM,CAAC,GAAG,CAAC,CAAA;QAE5B,sBAAsB;QACtB,MAAM,QAAQ,GAAG,IAAI,CAAC,kBAAkB,GAAG,GAAG,CAAA;QAC9C,MAAM,IAAI,CAAC,OAAO,CAAC,YAAY,CAAC,QAAQ,EAAE,IAAW,CAAC,CAAA;IACxD,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,IAAI;QACR,qCAAqC;QACrC,MAAM,WAAW,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,WAAW,CAAC,EAAE,CAAC,CAAA;QAEtD,IAAI,CAAC,WAAW,IAAI,OAAO,WAAW,KAAK,QAAQ,EAAE,CAAC;YACpD,OAAO,EAAE,CAAA;QACX,CAAC;QAED,yBAAyB;QACzB,MAAM,UAAU,GAAa,EAAE,CAAA;QAC/B,KAAK,MAAM,GAAG,IAAI,MAAM,CAAC,IAAI,CAAC,WAAW,CAAC,EAAE,CAAC;YAC3C,IAAI,GAAG,CAAC,UAAU,CAAC,IAAI,CAAC,kBAAkB,CAAC,EAAE,CAAC;gBAC5C,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,SAAS,CAAC,IAAI,CAAC,kBAAkB,CAAC,MAAM,CAAC,CAAC,CAAA;YAChE,CAAC;QACH,CAAC;QAED,OAAO,UAAU,CAAA;IACnB,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,GAAG,CAAC,GAAW;QACnB,IAAI,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,CAAC;YAC9B,OAAO,IAAI,CAAA;QACb,CAAC;QAED,MAAM,QAAQ,GAAG,IAAI,CAAC,kBAAkB,GAAG,GAAG,CAAA;QAC9C,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,WAAW,CAAC,QAAQ,CAAC,CAAA;QACzD,OAAO,QAAQ,KAAK,IAAI,IAAI,QAAQ,KAAK,SAAS,CAAA;IACpD,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,KAAK;QACT,cAAc;QACd,IAAI,CAAC,WAAW,CAAC,KAAK,EAAE,CAAA;QAExB,qBAAqB;QACrB,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,IAAI,EAAE,CAAA;QAC9B,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;YACvB,MAAM,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,CAAA;QACxB,CAAC;IACH,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,MAAM;QACV,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,IAAI,EAAE,CAAA;QAC9B,MAAM,MAAM,GAAgC,EAAE,CAAA;QAE9C,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;YACvB,MAAM,KAAK,GAAG,MAAM,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAA;YACtC,IAAI,KAAK,EAAE,CAAC;gBACV,MAAM,CAAC,GAAG,CAAC,GAAG,KAAK,CAAA;YACrB,CAAC;QACH,CAAC;QAED,OAAO,MAAM,CAAA;IACf,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,MAAM,CAAC,MAAmC;QAC9C,KAAK,MAAM,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;YAClD,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAA;YAChC,MAAM,QAAQ,GAAG,IAAI,CAAC,kBAAkB,GAAG,GAAG,CAAA;YAC9C,MAAM,IAAI,CAAC,OAAO,CAAC,YAAY,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAA;QAClD,CAAC;IACH,CAAC;IAED;;OAEG;IACK,KAAK,CAAC,QAAQ,CAAC,GAAW;QAChC,IAAI,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,CAAC;YAC9B,OAAO,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,GAAG,CAAE,CAAA;QACnC,CAAC;QAED,MAAM,QAAQ,GAAG,IAAI,CAAC,kBAAkB,GAAG,GAAG,CAAA;QAC9C,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,WAAW,CAAC,QAAQ,CAAC,CAAA;QAEzD,IAAI,CAAC,QAAQ,EAAE,CAAC;YACd,OAAO,IAAI,CAAA;QACb,CAAC;QAED,MAAM,KAAK,GAAG,QAAuB,CAAA;QACrC,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAA;QAChC,OAAO,KAAK,CAAA;IACd,CAAC;CACF"}
|
||||||
118
.recovery-workspace/dist-backup-20250910-141917/api/DataAPI.d.ts
vendored
Normal file
118
.recovery-workspace/dist-backup-20250910-141917/api/DataAPI.d.ts
vendored
Normal file
|
|
@ -0,0 +1,118 @@
|
||||||
|
/**
|
||||||
|
* Data Management API for Brainy 3.0
|
||||||
|
* Provides backup, restore, import, export, and data management
|
||||||
|
*/
|
||||||
|
import { StorageAdapter } from '../coreTypes.js';
|
||||||
|
import { Entity, Relation } from '../types/brainy.types.js';
|
||||||
|
import { NounType } from '../types/graphTypes.js';
|
||||||
|
export interface BackupOptions {
|
||||||
|
includeVectors?: boolean;
|
||||||
|
compress?: boolean;
|
||||||
|
format?: 'json' | 'binary';
|
||||||
|
}
|
||||||
|
export interface RestoreOptions {
|
||||||
|
merge?: boolean;
|
||||||
|
overwrite?: boolean;
|
||||||
|
validate?: boolean;
|
||||||
|
}
|
||||||
|
export interface ImportOptions {
|
||||||
|
format: 'json' | 'csv' | 'parquet';
|
||||||
|
mapping?: Record<string, string>;
|
||||||
|
batchSize?: number;
|
||||||
|
validate?: boolean;
|
||||||
|
}
|
||||||
|
export interface ExportOptions {
|
||||||
|
format?: 'json' | 'csv' | 'parquet';
|
||||||
|
filter?: {
|
||||||
|
type?: NounType | NounType[];
|
||||||
|
where?: Record<string, any>;
|
||||||
|
service?: string;
|
||||||
|
};
|
||||||
|
includeVectors?: boolean;
|
||||||
|
}
|
||||||
|
export interface BackupData {
|
||||||
|
version: string;
|
||||||
|
timestamp: number;
|
||||||
|
entities: Array<{
|
||||||
|
id: string;
|
||||||
|
vector?: number[];
|
||||||
|
type: string;
|
||||||
|
metadata: any;
|
||||||
|
service?: string;
|
||||||
|
}>;
|
||||||
|
relations: Array<{
|
||||||
|
id: string;
|
||||||
|
from: string;
|
||||||
|
to: string;
|
||||||
|
type: string;
|
||||||
|
weight: number;
|
||||||
|
metadata?: any;
|
||||||
|
}>;
|
||||||
|
config?: Record<string, any>;
|
||||||
|
stats: {
|
||||||
|
entityCount: number;
|
||||||
|
relationCount: number;
|
||||||
|
vectorDimensions?: number;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
export interface ImportResult {
|
||||||
|
successful: number;
|
||||||
|
failed: number;
|
||||||
|
errors: Array<{
|
||||||
|
item: any;
|
||||||
|
error: string;
|
||||||
|
}>;
|
||||||
|
duration: number;
|
||||||
|
}
|
||||||
|
export declare class DataAPI {
|
||||||
|
private storage;
|
||||||
|
private getEntity;
|
||||||
|
private getRelation?;
|
||||||
|
private brain;
|
||||||
|
constructor(storage: StorageAdapter, getEntity: (id: string) => Promise<Entity | null>, getRelation?: ((id: string) => Promise<Relation | null>) | undefined, brain?: any);
|
||||||
|
/**
|
||||||
|
* Create a backup of all data
|
||||||
|
*/
|
||||||
|
backup(options?: BackupOptions): Promise<BackupData>;
|
||||||
|
/**
|
||||||
|
* Restore data from a backup
|
||||||
|
*/
|
||||||
|
restore(params: {
|
||||||
|
backup: BackupData;
|
||||||
|
merge?: boolean;
|
||||||
|
overwrite?: boolean;
|
||||||
|
validate?: boolean;
|
||||||
|
}): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Clear data
|
||||||
|
*/
|
||||||
|
clear(params?: {
|
||||||
|
entities?: boolean;
|
||||||
|
relations?: boolean;
|
||||||
|
config?: boolean;
|
||||||
|
}): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Import data from various formats
|
||||||
|
*/
|
||||||
|
import(params: ImportOptions & {
|
||||||
|
data: any;
|
||||||
|
}): Promise<ImportResult>;
|
||||||
|
/**
|
||||||
|
* Export data to various formats
|
||||||
|
*/
|
||||||
|
export(params?: ExportOptions): Promise<any>;
|
||||||
|
/**
|
||||||
|
* Get storage statistics
|
||||||
|
*/
|
||||||
|
getStats(): Promise<{
|
||||||
|
entities: number;
|
||||||
|
relations: number;
|
||||||
|
storageSize?: number;
|
||||||
|
vectorDimensions?: number;
|
||||||
|
}>;
|
||||||
|
private applyMapping;
|
||||||
|
private validateImportItem;
|
||||||
|
private matchesFilter;
|
||||||
|
private convertToCSV;
|
||||||
|
private generateId;
|
||||||
|
}
|
||||||
386
.recovery-workspace/dist-backup-20250910-141917/api/DataAPI.js
Normal file
386
.recovery-workspace/dist-backup-20250910-141917/api/DataAPI.js
Normal file
|
|
@ -0,0 +1,386 @@
|
||||||
|
/**
|
||||||
|
* Data Management API for Brainy 3.0
|
||||||
|
* Provides backup, restore, import, export, and data management
|
||||||
|
*/
|
||||||
|
import { NounType } from '../types/graphTypes.js';
|
||||||
|
export class DataAPI {
|
||||||
|
constructor(storage, getEntity, getRelation, brain) {
|
||||||
|
this.storage = storage;
|
||||||
|
this.getEntity = getEntity;
|
||||||
|
this.getRelation = getRelation;
|
||||||
|
this.brain = brain;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Create a backup of all data
|
||||||
|
*/
|
||||||
|
async backup(options = {}) {
|
||||||
|
const { includeVectors = true, compress = false, format = 'json' } = options;
|
||||||
|
const startTime = Date.now();
|
||||||
|
// Get all entities
|
||||||
|
const nounsResult = await this.storage.getNouns({
|
||||||
|
pagination: { limit: 1000000 }
|
||||||
|
});
|
||||||
|
const entities = [];
|
||||||
|
for (const noun of nounsResult.items) {
|
||||||
|
const entity = {
|
||||||
|
id: noun.id,
|
||||||
|
vector: includeVectors ? noun.vector : undefined,
|
||||||
|
type: noun.metadata?.noun || NounType.Thing,
|
||||||
|
metadata: noun.metadata,
|
||||||
|
service: noun.metadata?.service
|
||||||
|
};
|
||||||
|
entities.push(entity);
|
||||||
|
}
|
||||||
|
// Get all relations
|
||||||
|
const verbsResult = await this.storage.getVerbs({
|
||||||
|
pagination: { limit: 1000000 }
|
||||||
|
});
|
||||||
|
const relations = [];
|
||||||
|
for (const verb of verbsResult.items) {
|
||||||
|
relations.push({
|
||||||
|
id: verb.id,
|
||||||
|
from: verb.sourceId,
|
||||||
|
to: verb.targetId,
|
||||||
|
type: (verb.verb || verb.type),
|
||||||
|
weight: verb.weight || 1.0,
|
||||||
|
metadata: verb.metadata
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// Create backup data
|
||||||
|
const backupData = {
|
||||||
|
version: '3.0.0',
|
||||||
|
timestamp: Date.now(),
|
||||||
|
entities,
|
||||||
|
relations,
|
||||||
|
stats: {
|
||||||
|
entityCount: entities.length,
|
||||||
|
relationCount: relations.length,
|
||||||
|
vectorDimensions: entities[0]?.vector?.length
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// Compress if requested (simplified - just stringify for now)
|
||||||
|
if (compress) {
|
||||||
|
// In production, use proper compression like gzip
|
||||||
|
// For now, just return the data
|
||||||
|
}
|
||||||
|
return backupData;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Restore data from a backup
|
||||||
|
*/
|
||||||
|
async restore(params) {
|
||||||
|
const { backup, merge = false, overwrite = false, validate = true } = params;
|
||||||
|
// Validate backup format
|
||||||
|
if (validate) {
|
||||||
|
if (!backup.version || !backup.entities || !backup.relations) {
|
||||||
|
throw new Error('Invalid backup format');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Clear existing data if not merging
|
||||||
|
if (!merge && overwrite) {
|
||||||
|
await this.clear({ entities: true, relations: true });
|
||||||
|
}
|
||||||
|
// Restore entities
|
||||||
|
for (const entity of backup.entities) {
|
||||||
|
try {
|
||||||
|
const noun = {
|
||||||
|
id: entity.id,
|
||||||
|
vector: entity.vector || new Array(384).fill(0), // Default vector if missing
|
||||||
|
connections: new Map(),
|
||||||
|
level: 0,
|
||||||
|
metadata: {
|
||||||
|
...entity.metadata,
|
||||||
|
noun: entity.type,
|
||||||
|
service: entity.service
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// Check if entity exists when merging
|
||||||
|
if (merge) {
|
||||||
|
const existing = await this.storage.getNoun(entity.id);
|
||||||
|
if (existing && !overwrite) {
|
||||||
|
continue; // Skip existing entities unless overwriting
|
||||||
|
}
|
||||||
|
}
|
||||||
|
await this.storage.saveNoun(noun);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.error(`Failed to restore entity ${entity.id}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Restore relations
|
||||||
|
for (const relation of backup.relations) {
|
||||||
|
try {
|
||||||
|
// Get source and target entities to compute relation vector
|
||||||
|
const sourceNoun = await this.storage.getNoun(relation.from);
|
||||||
|
const targetNoun = await this.storage.getNoun(relation.to);
|
||||||
|
if (!sourceNoun || !targetNoun) {
|
||||||
|
console.warn(`Skipping relation ${relation.id}: missing entities`);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Compute relation vector as average of source and target
|
||||||
|
const relationVector = sourceNoun.vector.map((v, i) => (v + targetNoun.vector[i]) / 2);
|
||||||
|
const verb = {
|
||||||
|
id: relation.id,
|
||||||
|
vector: relationVector,
|
||||||
|
sourceId: relation.from,
|
||||||
|
targetId: relation.to,
|
||||||
|
source: sourceNoun.metadata?.noun || NounType.Thing,
|
||||||
|
target: targetNoun.metadata?.noun || NounType.Thing,
|
||||||
|
verb: relation.type,
|
||||||
|
type: relation.type,
|
||||||
|
weight: relation.weight,
|
||||||
|
metadata: relation.metadata,
|
||||||
|
createdAt: Date.now()
|
||||||
|
};
|
||||||
|
// Check if relation exists when merging
|
||||||
|
if (merge) {
|
||||||
|
const existing = await this.storage.getVerb(relation.id);
|
||||||
|
if (existing && !overwrite) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
await this.storage.saveVerb(verb);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.error(`Failed to restore relation ${relation.id}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear data
|
||||||
|
*/
|
||||||
|
async clear(params = {}) {
|
||||||
|
const { entities = true, relations = true, config = false } = params;
|
||||||
|
if (entities) {
|
||||||
|
// Clear all entities
|
||||||
|
const nounsResult = await this.storage.getNouns({
|
||||||
|
pagination: { limit: 1000000 }
|
||||||
|
});
|
||||||
|
for (const noun of nounsResult.items) {
|
||||||
|
await this.storage.deleteNoun(noun.id);
|
||||||
|
}
|
||||||
|
// Also clear the HNSW index if available
|
||||||
|
if (this.brain?.index?.clear) {
|
||||||
|
this.brain.index.clear();
|
||||||
|
}
|
||||||
|
// Clear metadata index if available
|
||||||
|
if (this.brain?.metadataIndex) {
|
||||||
|
await this.brain.metadataIndex.rebuild(); // Rebuild empty index
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (relations) {
|
||||||
|
// Clear all relations
|
||||||
|
const verbsResult = await this.storage.getVerbs({
|
||||||
|
pagination: { limit: 1000000 }
|
||||||
|
});
|
||||||
|
for (const verb of verbsResult.items) {
|
||||||
|
await this.storage.deleteVerb(verb.id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (config) {
|
||||||
|
// Clear configuration would be handled by ConfigAPI
|
||||||
|
// For now, skip this
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Import data from various formats
|
||||||
|
*/
|
||||||
|
async import(params) {
|
||||||
|
const { data, format, mapping = {}, batchSize = 100, validate = true } = params;
|
||||||
|
const result = {
|
||||||
|
successful: 0,
|
||||||
|
failed: 0,
|
||||||
|
errors: [],
|
||||||
|
duration: 0
|
||||||
|
};
|
||||||
|
const startTime = Date.now();
|
||||||
|
try {
|
||||||
|
// ALWAYS use neural import for proper type matching
|
||||||
|
const { UniversalImportAPI } = await import('./UniversalImportAPI.js');
|
||||||
|
const universalImport = new UniversalImportAPI(this.brain);
|
||||||
|
await universalImport.init();
|
||||||
|
// Convert to ImportSource format
|
||||||
|
const neuralResult = await universalImport.import({
|
||||||
|
type: 'object',
|
||||||
|
data,
|
||||||
|
format: format || 'json',
|
||||||
|
metadata: { mapping, batchSize, validate }
|
||||||
|
});
|
||||||
|
// Convert neural result to ImportResult format
|
||||||
|
result.successful = neuralResult.stats.entitiesCreated;
|
||||||
|
result.failed = 0; // Neural import always succeeds with best match
|
||||||
|
result.duration = neuralResult.stats.processingTimeMs;
|
||||||
|
// Log relationships created
|
||||||
|
if (neuralResult.stats.relationshipsCreated > 0) {
|
||||||
|
console.log(`Neural import also created ${neuralResult.stats.relationshipsCreated} relationships`);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
// Fallback to legacy import ONLY if neural import fails to load
|
||||||
|
console.warn('Neural import failed, using legacy import:', error);
|
||||||
|
let items = [];
|
||||||
|
// Parse data based on format
|
||||||
|
switch (format) {
|
||||||
|
case 'json':
|
||||||
|
items = Array.isArray(data) ? data : [data];
|
||||||
|
break;
|
||||||
|
case 'csv':
|
||||||
|
// CSV parsing would go here
|
||||||
|
// For now, assume data is already parsed
|
||||||
|
items = data;
|
||||||
|
break;
|
||||||
|
case 'parquet':
|
||||||
|
// Parquet parsing would go here
|
||||||
|
throw new Error('Parquet format not yet implemented');
|
||||||
|
default:
|
||||||
|
throw new Error(`Unsupported format: ${format}`);
|
||||||
|
}
|
||||||
|
// Process items in batches
|
||||||
|
for (let i = 0; i < items.length; i += batchSize) {
|
||||||
|
const batch = items.slice(i, i + batchSize);
|
||||||
|
for (const item of batch) {
|
||||||
|
try {
|
||||||
|
// Apply field mapping
|
||||||
|
const mapped = this.applyMapping(item, mapping);
|
||||||
|
// Validate if requested
|
||||||
|
if (validate) {
|
||||||
|
this.validateImportItem(mapped);
|
||||||
|
}
|
||||||
|
// Save as entity
|
||||||
|
const noun = {
|
||||||
|
id: mapped.id || this.generateId(),
|
||||||
|
vector: mapped.vector || new Array(384).fill(0),
|
||||||
|
connections: new Map(),
|
||||||
|
level: 0,
|
||||||
|
metadata: mapped
|
||||||
|
};
|
||||||
|
await this.storage.saveNoun(noun);
|
||||||
|
result.successful++;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
result.failed++;
|
||||||
|
result.errors.push({
|
||||||
|
item,
|
||||||
|
error: error.message
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result.duration = Date.now() - startTime;
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Export data to various formats
|
||||||
|
*/
|
||||||
|
async export(params = {}) {
|
||||||
|
const { format = 'json', filter = {}, includeVectors = false } = params;
|
||||||
|
// Get filtered entities
|
||||||
|
const nounsResult = await this.storage.getNouns({
|
||||||
|
pagination: { limit: 1000000 }
|
||||||
|
});
|
||||||
|
let entities = nounsResult.items;
|
||||||
|
// Apply filters
|
||||||
|
if (filter.type) {
|
||||||
|
const types = Array.isArray(filter.type) ? filter.type : [filter.type];
|
||||||
|
entities = entities.filter(e => types.includes(e.metadata?.noun));
|
||||||
|
}
|
||||||
|
if (filter.service) {
|
||||||
|
entities = entities.filter(e => e.metadata?.service === filter.service);
|
||||||
|
}
|
||||||
|
if (filter.where) {
|
||||||
|
entities = entities.filter(e => this.matchesFilter(e.metadata, filter.where));
|
||||||
|
}
|
||||||
|
// Format data based on export format
|
||||||
|
switch (format) {
|
||||||
|
case 'json':
|
||||||
|
return entities.map(e => ({
|
||||||
|
id: e.id,
|
||||||
|
vector: includeVectors ? e.vector : undefined,
|
||||||
|
...e.metadata
|
||||||
|
}));
|
||||||
|
case 'csv':
|
||||||
|
// Convert to CSV format
|
||||||
|
// For now, return simplified format
|
||||||
|
return this.convertToCSV(entities);
|
||||||
|
case 'parquet':
|
||||||
|
throw new Error('Parquet export not yet implemented');
|
||||||
|
default:
|
||||||
|
throw new Error(`Unsupported export format: ${format}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get storage statistics
|
||||||
|
*/
|
||||||
|
async getStats() {
|
||||||
|
const nounsResult = await this.storage.getNouns({
|
||||||
|
pagination: { limit: 1 }
|
||||||
|
});
|
||||||
|
const verbsResult = await this.storage.getVerbs({
|
||||||
|
pagination: { limit: 1 }
|
||||||
|
});
|
||||||
|
const firstNoun = nounsResult.items[0];
|
||||||
|
return {
|
||||||
|
entities: nounsResult.totalCount || nounsResult.items.length,
|
||||||
|
relations: verbsResult.totalCount || verbsResult.items.length,
|
||||||
|
vectorDimensions: firstNoun?.vector?.length
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// Helper methods
|
||||||
|
applyMapping(item, mapping) {
|
||||||
|
const mapped = {};
|
||||||
|
for (const [key, value] of Object.entries(item)) {
|
||||||
|
const mappedKey = mapping[key] || key;
|
||||||
|
mapped[mappedKey] = value;
|
||||||
|
}
|
||||||
|
return mapped;
|
||||||
|
}
|
||||||
|
validateImportItem(item) {
|
||||||
|
// Basic validation
|
||||||
|
if (!item || typeof item !== 'object') {
|
||||||
|
throw new Error('Invalid item: must be an object');
|
||||||
|
}
|
||||||
|
// Could add more validation here
|
||||||
|
}
|
||||||
|
matchesFilter(metadata, filter) {
|
||||||
|
for (const [key, value] of Object.entries(filter)) {
|
||||||
|
if (metadata[key] !== value) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
convertToCSV(entities) {
|
||||||
|
if (entities.length === 0)
|
||||||
|
return '';
|
||||||
|
// Get all unique keys from metadata
|
||||||
|
const keys = new Set();
|
||||||
|
for (const entity of entities) {
|
||||||
|
if (entity.metadata) {
|
||||||
|
Object.keys(entity.metadata).forEach(k => keys.add(k));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Create CSV header
|
||||||
|
const headers = ['id', ...Array.from(keys)];
|
||||||
|
const rows = [headers.join(',')];
|
||||||
|
// Add data rows
|
||||||
|
for (const entity of entities) {
|
||||||
|
const row = [entity.id];
|
||||||
|
for (const key of keys) {
|
||||||
|
const value = entity.metadata?.[key] || '';
|
||||||
|
// Escape values that contain commas
|
||||||
|
const escaped = String(value).includes(',')
|
||||||
|
? `"${String(value).replace(/"/g, '""')}"`
|
||||||
|
: String(value);
|
||||||
|
row.push(escaped);
|
||||||
|
}
|
||||||
|
rows.push(row.join(','));
|
||||||
|
}
|
||||||
|
return rows.join('\n');
|
||||||
|
}
|
||||||
|
generateId() {
|
||||||
|
return `import_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=DataAPI.js.map
|
||||||
File diff suppressed because one or more lines are too long
50
.recovery-workspace/dist-backup-20250910-141917/api/SecurityAPI.d.ts
vendored
Normal file
50
.recovery-workspace/dist-backup-20250910-141917/api/SecurityAPI.d.ts
vendored
Normal file
|
|
@ -0,0 +1,50 @@
|
||||||
|
/**
|
||||||
|
* Security API for Brainy 3.0
|
||||||
|
* Provides encryption, decryption, hashing, and secure storage
|
||||||
|
*/
|
||||||
|
export declare class SecurityAPI {
|
||||||
|
private config?;
|
||||||
|
private encryptionKey?;
|
||||||
|
constructor(config?: {
|
||||||
|
encryptionKey?: string;
|
||||||
|
} | undefined);
|
||||||
|
/**
|
||||||
|
* Encrypt data using AES-256-CBC
|
||||||
|
*/
|
||||||
|
encrypt(data: string): Promise<string>;
|
||||||
|
/**
|
||||||
|
* Decrypt data encrypted with encrypt()
|
||||||
|
*/
|
||||||
|
decrypt(encryptedData: string): Promise<string>;
|
||||||
|
/**
|
||||||
|
* Create a one-way hash of data (for passwords, etc)
|
||||||
|
*/
|
||||||
|
hash(data: string, algorithm?: 'sha256' | 'sha512'): Promise<string>;
|
||||||
|
/**
|
||||||
|
* Compare data with a hash (for password verification)
|
||||||
|
*/
|
||||||
|
compare(data: string, hash: string, algorithm?: 'sha256' | 'sha512'): Promise<boolean>;
|
||||||
|
/**
|
||||||
|
* Generate a secure random token
|
||||||
|
*/
|
||||||
|
generateToken(bytes?: number): Promise<string>;
|
||||||
|
/**
|
||||||
|
* Derive a key from a password using PBKDF2
|
||||||
|
* Note: Simplified version using hash instead of PBKDF2 which may not be available
|
||||||
|
*/
|
||||||
|
deriveKey(password: string, salt?: string, iterations?: number): Promise<{
|
||||||
|
key: string;
|
||||||
|
salt: string;
|
||||||
|
}>;
|
||||||
|
/**
|
||||||
|
* Sign data with HMAC
|
||||||
|
*/
|
||||||
|
sign(data: string, secret?: string): Promise<string>;
|
||||||
|
/**
|
||||||
|
* Verify HMAC signature
|
||||||
|
*/
|
||||||
|
verify(data: string, signature: string, secret: string): Promise<boolean>;
|
||||||
|
private hexToBytes;
|
||||||
|
private bytesToHex;
|
||||||
|
private constantTimeCompare;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,139 @@
|
||||||
|
/**
|
||||||
|
* Security API for Brainy 3.0
|
||||||
|
* Provides encryption, decryption, hashing, and secure storage
|
||||||
|
*/
|
||||||
|
export class SecurityAPI {
|
||||||
|
constructor(config) {
|
||||||
|
this.config = config;
|
||||||
|
if (config?.encryptionKey) {
|
||||||
|
// Use provided key (must be 32 bytes hex string)
|
||||||
|
this.encryptionKey = this.hexToBytes(config.encryptionKey);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Encrypt data using AES-256-CBC
|
||||||
|
*/
|
||||||
|
async encrypt(data) {
|
||||||
|
const crypto = await import('../universal/crypto.js');
|
||||||
|
// Generate or use existing key
|
||||||
|
const key = this.encryptionKey || crypto.randomBytes(32);
|
||||||
|
const iv = crypto.randomBytes(16);
|
||||||
|
const cipher = crypto.createCipheriv('aes-256-cbc', key, iv);
|
||||||
|
let encrypted = cipher.update(data, 'utf8', 'hex');
|
||||||
|
encrypted += cipher.final('hex');
|
||||||
|
// Package encrypted data with metadata
|
||||||
|
// In production, store keys separately in a key management service
|
||||||
|
return JSON.stringify({
|
||||||
|
encrypted,
|
||||||
|
key: this.bytesToHex(key),
|
||||||
|
iv: this.bytesToHex(iv),
|
||||||
|
algorithm: 'aes-256-cbc',
|
||||||
|
timestamp: Date.now()
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Decrypt data encrypted with encrypt()
|
||||||
|
*/
|
||||||
|
async decrypt(encryptedData) {
|
||||||
|
const crypto = await import('../universal/crypto.js');
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(encryptedData);
|
||||||
|
const { encrypted, key: keyHex, iv: ivHex, algorithm } = parsed;
|
||||||
|
if (algorithm && algorithm !== 'aes-256-cbc') {
|
||||||
|
throw new Error(`Unsupported encryption algorithm: ${algorithm}`);
|
||||||
|
}
|
||||||
|
const key = this.hexToBytes(keyHex);
|
||||||
|
const iv = this.hexToBytes(ivHex);
|
||||||
|
const decipher = crypto.createDecipheriv('aes-256-cbc', key, iv);
|
||||||
|
let decrypted = decipher.update(encrypted, 'hex', 'utf8');
|
||||||
|
decrypted += decipher.final('utf8');
|
||||||
|
return decrypted;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
throw new Error(`Decryption failed: ${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Create a one-way hash of data (for passwords, etc)
|
||||||
|
*/
|
||||||
|
async hash(data, algorithm = 'sha256') {
|
||||||
|
const crypto = await import('../universal/crypto.js');
|
||||||
|
const hash = crypto.createHash(algorithm);
|
||||||
|
hash.update(data);
|
||||||
|
return hash.digest('hex');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Compare data with a hash (for password verification)
|
||||||
|
*/
|
||||||
|
async compare(data, hash, algorithm = 'sha256') {
|
||||||
|
const dataHash = await this.hash(data, algorithm);
|
||||||
|
return this.constantTimeCompare(dataHash, hash);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Generate a secure random token
|
||||||
|
*/
|
||||||
|
async generateToken(bytes = 32) {
|
||||||
|
const crypto = await import('../universal/crypto.js');
|
||||||
|
const buffer = crypto.randomBytes(bytes);
|
||||||
|
return this.bytesToHex(buffer);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Derive a key from a password using PBKDF2
|
||||||
|
* Note: Simplified version using hash instead of PBKDF2 which may not be available
|
||||||
|
*/
|
||||||
|
async deriveKey(password, salt, iterations = 100000) {
|
||||||
|
const crypto = await import('../universal/crypto.js');
|
||||||
|
const actualSalt = salt || this.bytesToHex(crypto.randomBytes(32));
|
||||||
|
// Simplified key derivation using repeated hashing
|
||||||
|
// In production, use a proper PBKDF2 implementation
|
||||||
|
let derived = password + actualSalt;
|
||||||
|
for (let i = 0; i < Math.min(iterations, 1000); i++) {
|
||||||
|
const hash = crypto.createHash('sha256');
|
||||||
|
hash.update(derived);
|
||||||
|
derived = hash.digest('hex');
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
key: derived,
|
||||||
|
salt: actualSalt
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Sign data with HMAC
|
||||||
|
*/
|
||||||
|
async sign(data, secret) {
|
||||||
|
const crypto = await import('../universal/crypto.js');
|
||||||
|
const actualSecret = secret || (await this.generateToken());
|
||||||
|
const hmac = crypto.createHmac('sha256', actualSecret);
|
||||||
|
hmac.update(data);
|
||||||
|
return hmac.digest('hex');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Verify HMAC signature
|
||||||
|
*/
|
||||||
|
async verify(data, signature, secret) {
|
||||||
|
const expectedSignature = await this.sign(data, secret);
|
||||||
|
return this.constantTimeCompare(signature, expectedSignature);
|
||||||
|
}
|
||||||
|
// Helper methods
|
||||||
|
hexToBytes(hex) {
|
||||||
|
const matches = hex.match(/.{1,2}/g);
|
||||||
|
if (!matches)
|
||||||
|
throw new Error('Invalid hex string');
|
||||||
|
return new Uint8Array(matches.map(byte => parseInt(byte, 16)));
|
||||||
|
}
|
||||||
|
bytesToHex(bytes) {
|
||||||
|
return Array.from(bytes)
|
||||||
|
.map(b => b.toString(16).padStart(2, '0'))
|
||||||
|
.join('');
|
||||||
|
}
|
||||||
|
constantTimeCompare(a, b) {
|
||||||
|
if (a.length !== b.length)
|
||||||
|
return false;
|
||||||
|
let result = 0;
|
||||||
|
for (let i = 0; i < a.length; i++) {
|
||||||
|
result |= a.charCodeAt(i) ^ b.charCodeAt(i);
|
||||||
|
}
|
||||||
|
return result === 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=SecurityAPI.js.map
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
{"version":3,"file":"SecurityAPI.js","sourceRoot":"","sources":["../../src/api/SecurityAPI.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,MAAM,OAAO,WAAW;IAGtB,YAAoB,MAAmC;QAAnC,WAAM,GAAN,MAAM,CAA6B;QACrD,IAAI,MAAM,EAAE,aAAa,EAAE,CAAC;YAC1B,iDAAiD;YACjD,IAAI,CAAC,aAAa,GAAG,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,aAAa,CAAC,CAAA;QAC5D,CAAC;IACH,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,OAAO,CAAC,IAAY;QACxB,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,wBAAwB,CAAC,CAAA;QAErD,+BAA+B;QAC/B,MAAM,GAAG,GAAG,IAAI,CAAC,aAAa,IAAI,MAAM,CAAC,WAAW,CAAC,EAAE,CAAC,CAAA;QACxD,MAAM,EAAE,GAAG,MAAM,CAAC,WAAW,CAAC,EAAE,CAAC,CAAA;QAEjC,MAAM,MAAM,GAAG,MAAM,CAAC,cAAc,CAAC,aAAa,EAAE,GAAG,EAAE,EAAE,CAAC,CAAA;QAC5D,IAAI,SAAS,GAAG,MAAM,CAAC,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,KAAK,CAAC,CAAA;QAClD,SAAS,IAAI,MAAM,CAAC,KAAK,CAAC,KAAK,CAAC,CAAA;QAEhC,uCAAuC;QACvC,mEAAmE;QACnE,OAAO,IAAI,CAAC,SAAS,CAAC;YACpB,SAAS;YACT,GAAG,EAAE,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC;YACzB,EAAE,EAAE,IAAI,CAAC,UAAU,CAAC,EAAE,CAAC;YACvB,SAAS,EAAE,aAAa;YACxB,SAAS,EAAE,IAAI,CAAC,GAAG,EAAE;SACtB,CAAC,CAAA;IACJ,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,OAAO,CAAC,aAAqB;QACjC,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,wBAAwB,CAAC,CAAA;QAErD,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,aAAa,CAAC,CAAA;YACxC,MAAM,EAAE,SAAS,EAAE,GAAG,EAAE,MAAM,EAAE,EAAE,EAAE,KAAK,EAAE,SAAS,EAAE,GAAG,MAAM,CAAA;YAE/D,IAAI,SAAS,IAAI,SAAS,KAAK,aAAa,EAAE,CAAC;gBAC7C,MAAM,IAAI,KAAK,CAAC,qCAAqC,SAAS,EAAE,CAAC,CAAA;YACnE,CAAC;YAED,MAAM,GAAG,GAAG,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,CAAA;YACnC,MAAM,EAAE,GAAG,IAAI,CAAC,UAAU,CAAC,KAAK,CAAC,CAAA;YAEjC,MAAM,QAAQ,GAAG,MAAM,CAAC,gBAAgB,CAAC,aAAa,EAAE,GAAG,EAAE,EAAE,CAAC,CAAA;YAChE,IAAI,SAAS,GAAG,QAAQ,CAAC,MAAM,CAAC,SAAS,EAAE,KAAK,EAAE,MAAM,CAAC,CAAA;YACzD,SAAS,IAAI,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,CAAA;YAEnC,OAAO,SAAS,CAAA;QAClB,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,IAAI,KAAK,CAAC,sBAAuB,KAAe,CAAC,OAAO,EAAE,CAAC,CAAA;QACnE,CAAC;IACH,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,IAAI,CAAC,IAAY,EAAE,YAAiC,QAAQ;QAChE,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,wBAAwB,CAAC,CAAA;QACrD,MAAM,IAAI,GAAG,MAAM,CAAC,UAAU,CAAC,SAAS,CAAC,CAAA;QACzC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAA;QACjB,OAAO,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;IAC3B,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,OAAO,CAAC,IAAY,EAAE,IAAY,EAAE,YAAiC,QAAQ;QACjF,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,SAAS,CAAC,CAAA;QACjD,OAAO,IAAI,CAAC,mBAAmB,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAA;IACjD,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,aAAa,CAAC,QAAgB,EAAE;QACpC,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,wBAAwB,CAAC,CAAA;QACrD,MAAM,MAAM,GAAG,MAAM,CAAC,WAAW,CAAC,KAAK,CAAC,CAAA;QACxC,OAAO,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,CAAA;IAChC,CAAC;IAED;;;OAGG;IACH,KAAK,CAAC,SAAS,CAAC,QAAgB,EAAE,IAAa,EAAE,aAAqB,MAAM;QAI1E,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,wBAAwB,CAAC,CAAA;QACrD,MAAM,UAAU,GAAG,IAAI,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,WAAW,CAAC,EAAE,CAAC,CAAC,CAAA;QAElE,mDAAmD;QACnD,oDAAoD;QACpD,IAAI,OAAO,GAAG,QAAQ,GAAG,UAAU,CAAA;QACnC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,UAAU,EAAE,IAAI,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC;YACpD,MAAM,IAAI,GAAG,MAAM,CAAC,UAAU,CAAC,QAAQ,CAAC,CAAA;YACxC,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,CAAA;YACpB,OAAO,GAAG,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;QAC9B,CAAC;QAED,OAAO;YACL,GAAG,EAAE,OAAO;YACZ,IAAI,EAAE,UAAU;SACjB,CAAA;IACH,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,IAAI,CAAC,IAAY,EAAE,MAAe;QACtC,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,wBAAwB,CAAC,CAAA;QACrD,MAAM,YAAY,GAAG,MAAM,IAAI,CAAC,MAAM,IAAI,CAAC,aAAa,EAAE,CAAC,CAAA;QAC3D,MAAM,IAAI,GAAG,MAAM,CAAC,UAAU,CAAC,QAAQ,EAAE,YAAY,CAAC,CAAA;QACtD,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAA;QACjB,OAAO,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;IAC3B,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,MAAM,CAAC,IAAY,EAAE,SAAiB,EAAE,MAAc;QAC1D,MAAM,iBAAiB,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,MAAM,CAAC,CAAA;QACvD,OAAO,IAAI,CAAC,mBAAmB,CAAC,SAAS,EAAE,iBAAiB,CAAC,CAAA;IAC/D,CAAC;IAED,iBAAiB;IAET,UAAU,CAAC,GAAW;QAC5B,MAAM,OAAO,GAAG,GAAG,CAAC,KAAK,CAAC,SAAS,CAAC,CAAA;QACpC,IAAI,CAAC,OAAO;YAAE,MAAM,IAAI,KAAK,CAAC,oBAAoB,CAAC,CAAA;QACnD,OAAO,IAAI,UAAU,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,QAAQ,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,CAAC,CAAA;IAChE,CAAC;IAEO,UAAU,CAAC,KAA0B;QAC3C,OAAO,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC;aACrB,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;aACzC,IAAI,CAAC,EAAE,CAAC,CAAA;IACb,CAAC;IAEO,mBAAmB,CAAC,CAAS,EAAE,CAAS;QAC9C,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC,CAAC,MAAM;YAAE,OAAO,KAAK,CAAA;QAEvC,IAAI,MAAM,GAAG,CAAC,CAAA;QACd,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YAClC,MAAM,IAAI,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,CAAA;QAC7C,CAAC;QACD,OAAO,MAAM,KAAK,CAAC,CAAA;IACrB,CAAC;CACF"}
|
||||||
134
.recovery-workspace/dist-backup-20250910-141917/api/UniversalImportAPI.d.ts
vendored
Normal file
134
.recovery-workspace/dist-backup-20250910-141917/api/UniversalImportAPI.d.ts
vendored
Normal file
|
|
@ -0,0 +1,134 @@
|
||||||
|
/**
|
||||||
|
* Universal Neural Import API
|
||||||
|
*
|
||||||
|
* ALWAYS uses neural matching to map ANY data to our strict NounTypes and VerbTypes
|
||||||
|
* Never falls back to rules - neural matching is MANDATORY
|
||||||
|
*
|
||||||
|
* Handles:
|
||||||
|
* - Strings (text, JSON, CSV, YAML, Markdown)
|
||||||
|
* - Files (local paths, any format)
|
||||||
|
* - URLs (web pages, APIs, documents)
|
||||||
|
* - Objects (structured data)
|
||||||
|
* - Binary data (images, PDFs via extraction)
|
||||||
|
*/
|
||||||
|
import { NounType, VerbType } from '../types/graphTypes.js';
|
||||||
|
import { Vector } from '../coreTypes.js';
|
||||||
|
import type { Brainy } from '../brainy.js';
|
||||||
|
export interface ImportSource {
|
||||||
|
type: 'string' | 'file' | 'url' | 'object' | 'binary';
|
||||||
|
data: any;
|
||||||
|
format?: string;
|
||||||
|
metadata?: any;
|
||||||
|
}
|
||||||
|
export interface NeuralImportResult {
|
||||||
|
entities: Array<{
|
||||||
|
id: string;
|
||||||
|
type: NounType;
|
||||||
|
data: any;
|
||||||
|
vector: Vector;
|
||||||
|
confidence: number;
|
||||||
|
metadata: any;
|
||||||
|
}>;
|
||||||
|
relationships: Array<{
|
||||||
|
id: string;
|
||||||
|
from: string;
|
||||||
|
to: string;
|
||||||
|
type: VerbType;
|
||||||
|
weight: number;
|
||||||
|
confidence: number;
|
||||||
|
metadata?: any;
|
||||||
|
}>;
|
||||||
|
stats: {
|
||||||
|
totalProcessed: number;
|
||||||
|
entitiesCreated: number;
|
||||||
|
relationshipsCreated: number;
|
||||||
|
averageConfidence: number;
|
||||||
|
processingTimeMs: number;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
export declare class UniversalImportAPI {
|
||||||
|
private brain;
|
||||||
|
private typeMatcher;
|
||||||
|
private neuralImport;
|
||||||
|
private embedCache;
|
||||||
|
constructor(brain: Brainy<any>);
|
||||||
|
/**
|
||||||
|
* Initialize the neural import system
|
||||||
|
*/
|
||||||
|
init(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Universal import - handles ANY data source
|
||||||
|
* ALWAYS uses neural matching, NEVER falls back
|
||||||
|
*/
|
||||||
|
import(source: ImportSource | string | any): Promise<NeuralImportResult>;
|
||||||
|
/**
|
||||||
|
* Import from URL - fetches and processes
|
||||||
|
*/
|
||||||
|
importFromURL(url: string): Promise<NeuralImportResult>;
|
||||||
|
/**
|
||||||
|
* Import from file - reads and processes
|
||||||
|
* Note: In browser environment, use File API instead
|
||||||
|
*/
|
||||||
|
importFromFile(filePath: string): Promise<NeuralImportResult>;
|
||||||
|
/**
|
||||||
|
* Normalize any input to ImportSource
|
||||||
|
*/
|
||||||
|
private normalizeSource;
|
||||||
|
/**
|
||||||
|
* Extract structured data from source
|
||||||
|
*/
|
||||||
|
private extractData;
|
||||||
|
/**
|
||||||
|
* Extract data from URL
|
||||||
|
*/
|
||||||
|
private extractFromURL;
|
||||||
|
/**
|
||||||
|
* Extract data from file
|
||||||
|
*/
|
||||||
|
private extractFromFile;
|
||||||
|
/**
|
||||||
|
* Extract data from string based on format
|
||||||
|
*/
|
||||||
|
private extractFromString;
|
||||||
|
/**
|
||||||
|
* Extract from binary data (images, PDFs, etc)
|
||||||
|
*/
|
||||||
|
private extractFromBinary;
|
||||||
|
/**
|
||||||
|
* Extract entities from plain text
|
||||||
|
*/
|
||||||
|
private extractFromText;
|
||||||
|
/**
|
||||||
|
* Neural processing - CORE of the system
|
||||||
|
* ALWAYS uses embeddings and neural matching
|
||||||
|
*/
|
||||||
|
private neuralProcess;
|
||||||
|
/**
|
||||||
|
* Generate embedding for any data
|
||||||
|
*/
|
||||||
|
private generateEmbedding;
|
||||||
|
/**
|
||||||
|
* Convert any data to text for embedding
|
||||||
|
*/
|
||||||
|
private dataToText;
|
||||||
|
/**
|
||||||
|
* Detect relationships using neural matching
|
||||||
|
*/
|
||||||
|
private detectNeuralRelationships;
|
||||||
|
/**
|
||||||
|
* Check if a field looks like a reference
|
||||||
|
*/
|
||||||
|
private looksLikeReference;
|
||||||
|
/**
|
||||||
|
* Store processed data in brain
|
||||||
|
*/
|
||||||
|
private storeInBrain;
|
||||||
|
private detectFormat;
|
||||||
|
private parseCSV;
|
||||||
|
private parseYAML;
|
||||||
|
private parseMarkdown;
|
||||||
|
private parseHTML;
|
||||||
|
private generateId;
|
||||||
|
private simpleHash;
|
||||||
|
private hashBinary;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,610 @@
|
||||||
|
/**
|
||||||
|
* Universal Neural Import API
|
||||||
|
*
|
||||||
|
* ALWAYS uses neural matching to map ANY data to our strict NounTypes and VerbTypes
|
||||||
|
* Never falls back to rules - neural matching is MANDATORY
|
||||||
|
*
|
||||||
|
* Handles:
|
||||||
|
* - Strings (text, JSON, CSV, YAML, Markdown)
|
||||||
|
* - Files (local paths, any format)
|
||||||
|
* - URLs (web pages, APIs, documents)
|
||||||
|
* - Objects (structured data)
|
||||||
|
* - Binary data (images, PDFs via extraction)
|
||||||
|
*/
|
||||||
|
import { getBrainyTypes } from '../augmentations/typeMatching/brainyTypes.js';
|
||||||
|
import { NeuralImportAugmentation } from '../augmentations/neuralImport.js';
|
||||||
|
export class UniversalImportAPI {
|
||||||
|
constructor(brain) {
|
||||||
|
this.embedCache = new Map();
|
||||||
|
this.brain = brain;
|
||||||
|
this.neuralImport = new NeuralImportAugmentation({
|
||||||
|
confidenceThreshold: 0.0, // Accept ALL confidence levels - never reject
|
||||||
|
enableWeights: true,
|
||||||
|
skipDuplicates: false // Process everything
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Initialize the neural import system
|
||||||
|
*/
|
||||||
|
async init() {
|
||||||
|
this.typeMatcher = await getBrainyTypes();
|
||||||
|
// Neural import initializes itself
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Universal import - handles ANY data source
|
||||||
|
* ALWAYS uses neural matching, NEVER falls back
|
||||||
|
*/
|
||||||
|
async import(source) {
|
||||||
|
const startTime = Date.now();
|
||||||
|
// Normalize source
|
||||||
|
const normalizedSource = this.normalizeSource(source);
|
||||||
|
// Extract data based on source type
|
||||||
|
const extractedData = await this.extractData(normalizedSource);
|
||||||
|
// Neural processing - MANDATORY
|
||||||
|
const neuralResults = await this.neuralProcess(extractedData);
|
||||||
|
// Store in brain
|
||||||
|
const result = await this.storeInBrain(neuralResults);
|
||||||
|
result.stats.processingTimeMs = Date.now() - startTime;
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Import from URL - fetches and processes
|
||||||
|
*/
|
||||||
|
async importFromURL(url) {
|
||||||
|
const response = await fetch(url);
|
||||||
|
const contentType = response.headers.get('content-type') || 'text/plain';
|
||||||
|
let data;
|
||||||
|
if (contentType.includes('json')) {
|
||||||
|
data = await response.json();
|
||||||
|
}
|
||||||
|
else if (contentType.includes('text') || contentType.includes('html')) {
|
||||||
|
data = await response.text();
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
// Binary data
|
||||||
|
const buffer = await response.arrayBuffer();
|
||||||
|
data = new Uint8Array(buffer);
|
||||||
|
}
|
||||||
|
return this.import({
|
||||||
|
type: 'url',
|
||||||
|
data,
|
||||||
|
format: contentType,
|
||||||
|
metadata: { url, fetchedAt: Date.now() }
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Import from file - reads and processes
|
||||||
|
* Note: In browser environment, use File API instead
|
||||||
|
*/
|
||||||
|
async importFromFile(filePath) {
|
||||||
|
// For file imports, the caller should read the file and pass content
|
||||||
|
// This is a placeholder that treats the path as a reference
|
||||||
|
const ext = filePath.split('.').pop()?.toLowerCase() || 'txt';
|
||||||
|
return this.import({
|
||||||
|
type: 'file',
|
||||||
|
data: filePath, // Path as reference
|
||||||
|
format: ext,
|
||||||
|
metadata: {
|
||||||
|
path: filePath,
|
||||||
|
importedAt: Date.now()
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Normalize any input to ImportSource
|
||||||
|
*/
|
||||||
|
normalizeSource(source) {
|
||||||
|
// Already normalized
|
||||||
|
if (source && typeof source === 'object' && 'type' in source && 'data' in source) {
|
||||||
|
return source;
|
||||||
|
}
|
||||||
|
// String input
|
||||||
|
if (typeof source === 'string') {
|
||||||
|
// Check if it's a URL
|
||||||
|
if (source.startsWith('http://') || source.startsWith('https://')) {
|
||||||
|
return { type: 'url', data: source };
|
||||||
|
}
|
||||||
|
// Check if it looks like a file path
|
||||||
|
if (source.includes('/') || source.includes('\\') || source.includes('.')) {
|
||||||
|
// Assume it's a file path reference
|
||||||
|
return { type: 'file', data: source };
|
||||||
|
}
|
||||||
|
// Treat as raw string data
|
||||||
|
return { type: 'string', data: source };
|
||||||
|
}
|
||||||
|
// Object/Array input
|
||||||
|
if (typeof source === 'object') {
|
||||||
|
return { type: 'object', data: source };
|
||||||
|
}
|
||||||
|
// Default to string
|
||||||
|
return { type: 'string', data: String(source) };
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Extract structured data from source
|
||||||
|
*/
|
||||||
|
async extractData(source) {
|
||||||
|
switch (source.type) {
|
||||||
|
case 'url':
|
||||||
|
// URL is in data field, need to fetch
|
||||||
|
return this.extractFromURL(source.data);
|
||||||
|
case 'file':
|
||||||
|
// File path is in data field, need to read
|
||||||
|
return this.extractFromFile(source.data);
|
||||||
|
case 'string':
|
||||||
|
return this.extractFromString(source.data, source.format);
|
||||||
|
case 'object':
|
||||||
|
return Array.isArray(source.data) ? source.data : [source.data];
|
||||||
|
case 'binary':
|
||||||
|
return this.extractFromBinary(source.data, source.format);
|
||||||
|
default:
|
||||||
|
// Unknown type, treat as object
|
||||||
|
return [source.data];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Extract data from URL
|
||||||
|
*/
|
||||||
|
async extractFromURL(url) {
|
||||||
|
const result = await this.importFromURL(url);
|
||||||
|
return result.entities.map(e => e.data);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Extract data from file
|
||||||
|
*/
|
||||||
|
async extractFromFile(filePath) {
|
||||||
|
const result = await this.importFromFile(filePath);
|
||||||
|
return result.entities.map(e => e.data);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Extract data from string based on format
|
||||||
|
*/
|
||||||
|
extractFromString(data, format) {
|
||||||
|
// Try to detect format if not provided
|
||||||
|
const detectedFormat = format || this.detectFormat(data);
|
||||||
|
switch (detectedFormat) {
|
||||||
|
case 'json':
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(data);
|
||||||
|
return Array.isArray(parsed) ? parsed : [parsed];
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
// Not valid JSON, treat as text
|
||||||
|
return this.extractFromText(data);
|
||||||
|
}
|
||||||
|
case 'csv':
|
||||||
|
return this.parseCSV(data);
|
||||||
|
case 'yaml':
|
||||||
|
case 'yml':
|
||||||
|
return this.parseYAML(data);
|
||||||
|
case 'markdown':
|
||||||
|
case 'md':
|
||||||
|
return this.parseMarkdown(data);
|
||||||
|
case 'xml':
|
||||||
|
case 'html':
|
||||||
|
return this.parseHTML(data);
|
||||||
|
default:
|
||||||
|
return this.extractFromText(data);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Extract from binary data (images, PDFs, etc)
|
||||||
|
*/
|
||||||
|
async extractFromBinary(data, format) {
|
||||||
|
// For now, create a single entity representing the binary data
|
||||||
|
// In production, would use OCR, image recognition, PDF extraction, etc.
|
||||||
|
return [{
|
||||||
|
type: 'binary',
|
||||||
|
format: format || 'unknown',
|
||||||
|
size: data.length,
|
||||||
|
hash: await this.hashBinary(data),
|
||||||
|
extractedAt: Date.now()
|
||||||
|
}];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Extract entities from plain text
|
||||||
|
*/
|
||||||
|
extractFromText(text) {
|
||||||
|
// Split into meaningful chunks
|
||||||
|
const chunks = [];
|
||||||
|
// Split by paragraphs
|
||||||
|
const paragraphs = text.split(/\n\n+/);
|
||||||
|
for (const para of paragraphs) {
|
||||||
|
if (para.trim()) {
|
||||||
|
chunks.push({
|
||||||
|
text: para.trim(),
|
||||||
|
type: 'paragraph',
|
||||||
|
length: para.length
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// If no paragraphs, split by sentences
|
||||||
|
if (chunks.length === 0) {
|
||||||
|
const sentences = text.match(/[^.!?]+[.!?]+/g) || [text];
|
||||||
|
for (const sentence of sentences) {
|
||||||
|
if (sentence.trim()) {
|
||||||
|
chunks.push({
|
||||||
|
text: sentence.trim(),
|
||||||
|
type: 'sentence',
|
||||||
|
length: sentence.length
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return chunks;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Neural processing - CORE of the system
|
||||||
|
* ALWAYS uses embeddings and neural matching
|
||||||
|
*/
|
||||||
|
async neuralProcess(data) {
|
||||||
|
const entities = new Map();
|
||||||
|
const relationships = new Map();
|
||||||
|
for (const item of data) {
|
||||||
|
// Generate embedding for the item
|
||||||
|
const embedding = await this.generateEmbedding(item);
|
||||||
|
// Neural type matching - MANDATORY
|
||||||
|
const nounMatch = await this.typeMatcher.matchNounType(item);
|
||||||
|
// Never reject based on confidence - we ALWAYS accept the best match
|
||||||
|
const entityId = this.generateId(item);
|
||||||
|
entities.set(entityId, {
|
||||||
|
id: entityId,
|
||||||
|
type: nounMatch.type, // Always use the neural match
|
||||||
|
data: item,
|
||||||
|
vector: embedding,
|
||||||
|
confidence: nounMatch.confidence,
|
||||||
|
metadata: {
|
||||||
|
...item,
|
||||||
|
_neuralMatch: nounMatch,
|
||||||
|
_importedAt: Date.now()
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Detect relationships using neural matching
|
||||||
|
await this.detectNeuralRelationships(item, entityId, entities, relationships);
|
||||||
|
}
|
||||||
|
return { entities, relationships };
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Generate embedding for any data
|
||||||
|
*/
|
||||||
|
async generateEmbedding(data) {
|
||||||
|
// Convert to string for embedding
|
||||||
|
const text = this.dataToText(data);
|
||||||
|
// Check cache
|
||||||
|
if (this.embedCache.has(text)) {
|
||||||
|
return this.embedCache.get(text);
|
||||||
|
}
|
||||||
|
// Generate new embedding
|
||||||
|
const embedding = await this.brain.embed(text);
|
||||||
|
// Cache it
|
||||||
|
this.embedCache.set(text, embedding);
|
||||||
|
return embedding;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Convert any data to text for embedding
|
||||||
|
*/
|
||||||
|
dataToText(data) {
|
||||||
|
if (typeof data === 'string')
|
||||||
|
return data;
|
||||||
|
if (typeof data === 'object') {
|
||||||
|
// Extract meaningful text from object
|
||||||
|
const parts = [];
|
||||||
|
// Priority fields
|
||||||
|
const priorityFields = ['name', 'title', 'description', 'text', 'content', 'label', 'value'];
|
||||||
|
for (const field of priorityFields) {
|
||||||
|
if (data[field]) {
|
||||||
|
parts.push(String(data[field]));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Add other fields
|
||||||
|
for (const [key, value] of Object.entries(data)) {
|
||||||
|
if (!priorityFields.includes(key) && value) {
|
||||||
|
if (typeof value === 'string' || typeof value === 'number') {
|
||||||
|
parts.push(`${key}: ${value}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return parts.join(' ');
|
||||||
|
}
|
||||||
|
return JSON.stringify(data);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Detect relationships using neural matching
|
||||||
|
*/
|
||||||
|
async detectNeuralRelationships(item, sourceId, entities, relationships) {
|
||||||
|
if (typeof item !== 'object')
|
||||||
|
return;
|
||||||
|
// Look for references to other entities
|
||||||
|
for (const [key, value] of Object.entries(item)) {
|
||||||
|
// Check if this looks like a reference
|
||||||
|
if (this.looksLikeReference(key, value)) {
|
||||||
|
// Find or predict target entity
|
||||||
|
const targetId = String(value);
|
||||||
|
// Neural verb type matching
|
||||||
|
const verbMatch = await this.typeMatcher.matchVerbType(item, // source object
|
||||||
|
{ id: targetId }, // target (we may not have full data)
|
||||||
|
key // field name as context
|
||||||
|
);
|
||||||
|
// Always create relationship with neural match
|
||||||
|
const relationId = `${sourceId}_${verbMatch.type}_${targetId}`;
|
||||||
|
relationships.set(relationId, {
|
||||||
|
id: relationId,
|
||||||
|
from: sourceId,
|
||||||
|
to: targetId,
|
||||||
|
type: verbMatch.type,
|
||||||
|
weight: verbMatch.confidence, // Use confidence as weight
|
||||||
|
confidence: verbMatch.confidence,
|
||||||
|
metadata: {
|
||||||
|
field: key,
|
||||||
|
_neuralMatch: verbMatch,
|
||||||
|
_importedAt: Date.now()
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// Handle arrays of references
|
||||||
|
if (Array.isArray(value)) {
|
||||||
|
for (const item of value) {
|
||||||
|
if (this.looksLikeReference(key, item)) {
|
||||||
|
const targetId = String(item);
|
||||||
|
const verbMatch = await this.typeMatcher.matchVerbType(item, { id: targetId }, key);
|
||||||
|
const relationId = `${sourceId}_${verbMatch.type}_${targetId}`;
|
||||||
|
relationships.set(relationId, {
|
||||||
|
id: relationId,
|
||||||
|
from: sourceId,
|
||||||
|
to: targetId,
|
||||||
|
type: verbMatch.type,
|
||||||
|
weight: verbMatch.confidence,
|
||||||
|
confidence: verbMatch.confidence,
|
||||||
|
metadata: {
|
||||||
|
field: key,
|
||||||
|
array: true,
|
||||||
|
_neuralMatch: verbMatch,
|
||||||
|
_importedAt: Date.now()
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if a field looks like a reference
|
||||||
|
*/
|
||||||
|
looksLikeReference(key, value) {
|
||||||
|
// Field name patterns that suggest references
|
||||||
|
const refPatterns = [
|
||||||
|
/[Ii]d$/, // ends with Id or id
|
||||||
|
/_id$/, // ends with _id
|
||||||
|
/^parent/i, // starts with parent
|
||||||
|
/^child/i, // starts with child
|
||||||
|
/^related/i, // starts with related
|
||||||
|
/^ref/i, // starts with ref
|
||||||
|
/^link/i, // starts with link
|
||||||
|
/^target/i, // starts with target
|
||||||
|
/^source/i, // starts with source
|
||||||
|
];
|
||||||
|
// Check if field name matches patterns
|
||||||
|
const fieldLooksLikeRef = refPatterns.some(pattern => pattern.test(key));
|
||||||
|
// Check if value looks like an ID
|
||||||
|
const valueLooksLikeId = (typeof value === 'string' ||
|
||||||
|
typeof value === 'number') && String(value).length > 0;
|
||||||
|
return fieldLooksLikeRef && valueLooksLikeId;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Store processed data in brain
|
||||||
|
*/
|
||||||
|
async storeInBrain(neuralResults) {
|
||||||
|
const result = {
|
||||||
|
entities: [],
|
||||||
|
relationships: [],
|
||||||
|
stats: {
|
||||||
|
totalProcessed: neuralResults.entities.size + neuralResults.relationships.size,
|
||||||
|
entitiesCreated: 0,
|
||||||
|
relationshipsCreated: 0,
|
||||||
|
averageConfidence: 0,
|
||||||
|
processingTimeMs: 0
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let totalConfidence = 0;
|
||||||
|
// Store entities
|
||||||
|
for (const entity of neuralResults.entities.values()) {
|
||||||
|
const id = await this.brain.add({
|
||||||
|
data: entity.data,
|
||||||
|
type: entity.type,
|
||||||
|
metadata: entity.metadata,
|
||||||
|
vector: entity.vector,
|
||||||
|
writeOnly: true // Fast mode since we already have vectors
|
||||||
|
});
|
||||||
|
// Update entity ID for relationship mapping
|
||||||
|
entity.id = id;
|
||||||
|
result.entities.push({
|
||||||
|
...entity,
|
||||||
|
id
|
||||||
|
});
|
||||||
|
result.stats.entitiesCreated++;
|
||||||
|
totalConfidence += entity.confidence;
|
||||||
|
}
|
||||||
|
// Store relationships
|
||||||
|
for (const relation of neuralResults.relationships.values()) {
|
||||||
|
// Map to actual entity IDs
|
||||||
|
const sourceEntity = Array.from(neuralResults.entities.values())
|
||||||
|
.find(e => e.id === relation.from);
|
||||||
|
const targetEntity = Array.from(neuralResults.entities.values())
|
||||||
|
.find(e => e.id === relation.to);
|
||||||
|
if (sourceEntity && targetEntity) {
|
||||||
|
const id = await this.brain.relate({
|
||||||
|
from: sourceEntity.id,
|
||||||
|
to: targetEntity.id,
|
||||||
|
type: relation.type,
|
||||||
|
weight: relation.weight,
|
||||||
|
metadata: relation.metadata,
|
||||||
|
writeOnly: true
|
||||||
|
});
|
||||||
|
result.relationships.push({
|
||||||
|
...relation,
|
||||||
|
id,
|
||||||
|
from: sourceEntity.id,
|
||||||
|
to: targetEntity.id
|
||||||
|
});
|
||||||
|
result.stats.relationshipsCreated++;
|
||||||
|
totalConfidence += relation.confidence;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Calculate average confidence
|
||||||
|
const totalItems = result.stats.entitiesCreated + result.stats.relationshipsCreated;
|
||||||
|
result.stats.averageConfidence = totalItems > 0 ? totalConfidence / totalItems : 0;
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
// Helper methods for parsing different formats
|
||||||
|
detectFormat(data) {
|
||||||
|
const trimmed = data.trim();
|
||||||
|
// JSON
|
||||||
|
if ((trimmed.startsWith('{') && trimmed.endsWith('}')) ||
|
||||||
|
(trimmed.startsWith('[') && trimmed.endsWith(']'))) {
|
||||||
|
return 'json';
|
||||||
|
}
|
||||||
|
// CSV (has commas and newlines)
|
||||||
|
if (trimmed.includes(',') && trimmed.includes('\n')) {
|
||||||
|
return 'csv';
|
||||||
|
}
|
||||||
|
// YAML (has colons and indentation)
|
||||||
|
if (trimmed.includes(':') && (trimmed.includes('\n ') || trimmed.includes('\n\t'))) {
|
||||||
|
return 'yaml';
|
||||||
|
}
|
||||||
|
// Markdown (has headers)
|
||||||
|
if (trimmed.includes('#') || trimmed.includes('```')) {
|
||||||
|
return 'markdown';
|
||||||
|
}
|
||||||
|
// HTML/XML
|
||||||
|
if (trimmed.includes('<') && trimmed.includes('>')) {
|
||||||
|
return trimmed.toLowerCase().includes('<!doctype html') ? 'html' : 'xml';
|
||||||
|
}
|
||||||
|
return 'text';
|
||||||
|
}
|
||||||
|
parseCSV(data) {
|
||||||
|
// Reuse the CSV parser from neural import
|
||||||
|
const lines = data.split('\n').filter(l => l.trim());
|
||||||
|
if (lines.length === 0)
|
||||||
|
return [];
|
||||||
|
const headers = lines[0].split(',').map(h => h.trim());
|
||||||
|
const results = [];
|
||||||
|
for (let i = 1; i < lines.length; i++) {
|
||||||
|
const values = lines[i].split(',').map(v => v.trim());
|
||||||
|
const obj = {};
|
||||||
|
headers.forEach((header, index) => {
|
||||||
|
obj[header] = values[index] || '';
|
||||||
|
});
|
||||||
|
results.push(obj);
|
||||||
|
}
|
||||||
|
return results;
|
||||||
|
}
|
||||||
|
parseYAML(data) {
|
||||||
|
// Simple YAML parser
|
||||||
|
const results = [];
|
||||||
|
const lines = data.split('\n');
|
||||||
|
let current = null;
|
||||||
|
for (const line of lines) {
|
||||||
|
const trimmed = line.trim();
|
||||||
|
if (!trimmed || trimmed.startsWith('#'))
|
||||||
|
continue;
|
||||||
|
if (trimmed.startsWith('- ')) {
|
||||||
|
// Array item
|
||||||
|
const value = trimmed.substring(2);
|
||||||
|
if (!current) {
|
||||||
|
results.push(value);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
if (!current._items)
|
||||||
|
current._items = [];
|
||||||
|
current._items.push(value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else if (trimmed.includes(':')) {
|
||||||
|
// Key-value
|
||||||
|
const [key, ...valueParts] = trimmed.split(':');
|
||||||
|
const value = valueParts.join(':').trim();
|
||||||
|
if (!current) {
|
||||||
|
current = {};
|
||||||
|
results.push(current);
|
||||||
|
}
|
||||||
|
current[key.trim()] = value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return results.length > 0 ? results : [{ text: data }];
|
||||||
|
}
|
||||||
|
parseMarkdown(data) {
|
||||||
|
const results = [];
|
||||||
|
const lines = data.split('\n');
|
||||||
|
let current = null;
|
||||||
|
let inCodeBlock = false;
|
||||||
|
for (const line of lines) {
|
||||||
|
if (line.startsWith('```')) {
|
||||||
|
inCodeBlock = !inCodeBlock;
|
||||||
|
if (inCodeBlock && current) {
|
||||||
|
current.code = '';
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (inCodeBlock && current) {
|
||||||
|
current.code += line + '\n';
|
||||||
|
}
|
||||||
|
else if (line.startsWith('#')) {
|
||||||
|
// Header
|
||||||
|
const level = line.match(/^#+/)?.[0].length || 1;
|
||||||
|
const text = line.replace(/^#+\s*/, '');
|
||||||
|
current = {
|
||||||
|
type: 'heading',
|
||||||
|
level,
|
||||||
|
text
|
||||||
|
};
|
||||||
|
results.push(current);
|
||||||
|
}
|
||||||
|
else if (line.trim()) {
|
||||||
|
// Paragraph
|
||||||
|
if (!current || current.type !== 'paragraph') {
|
||||||
|
current = {
|
||||||
|
type: 'paragraph',
|
||||||
|
text: ''
|
||||||
|
};
|
||||||
|
results.push(current);
|
||||||
|
}
|
||||||
|
current.text += line + ' ';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return results;
|
||||||
|
}
|
||||||
|
parseHTML(data) {
|
||||||
|
// Simple HTML text extraction
|
||||||
|
const text = data
|
||||||
|
.replace(/<script\b[^<]*(?:(?!<\/script>)<[^<]*)*<\/script>/gi, '') // Remove scripts
|
||||||
|
.replace(/<style\b[^<]*(?:(?!<\/style>)<[^<]*)*<\/style>/gi, '') // Remove styles
|
||||||
|
.replace(/<[^>]+>/g, ' ') // Remove tags
|
||||||
|
.replace(/\s+/g, ' ') // Normalize whitespace
|
||||||
|
.trim();
|
||||||
|
return this.extractFromText(text);
|
||||||
|
}
|
||||||
|
generateId(data) {
|
||||||
|
// Generate deterministic ID based on content
|
||||||
|
const text = this.dataToText(data);
|
||||||
|
const hash = this.simpleHash(text);
|
||||||
|
return `import_${hash}_${Date.now()}`;
|
||||||
|
}
|
||||||
|
simpleHash(text) {
|
||||||
|
let hash = 0;
|
||||||
|
for (let i = 0; i < text.length; i++) {
|
||||||
|
const char = text.charCodeAt(i);
|
||||||
|
hash = ((hash << 5) - hash) + char;
|
||||||
|
hash = hash & hash;
|
||||||
|
}
|
||||||
|
return Math.abs(hash).toString(36);
|
||||||
|
}
|
||||||
|
async hashBinary(data) {
|
||||||
|
// Simple binary hash
|
||||||
|
let hash = 0;
|
||||||
|
for (let i = 0; i < Math.min(data.length, 1000); i++) {
|
||||||
|
hash = ((hash << 5) - hash) + data[i];
|
||||||
|
hash = hash & hash;
|
||||||
|
}
|
||||||
|
return Math.abs(hash).toString(36);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=UniversalImportAPI.js.map
|
||||||
File diff suppressed because one or more lines are too long
86
.recovery-workspace/dist-backup-20250910-141917/augmentationFactory.d.ts
vendored
Normal file
86
.recovery-workspace/dist-backup-20250910-141917/augmentationFactory.d.ts
vendored
Normal file
|
|
@ -0,0 +1,86 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Factory
|
||||||
|
*
|
||||||
|
* This module provides a simplified factory for creating augmentations with minimal boilerplate.
|
||||||
|
* It reduces the complexity of creating and using augmentations by providing a fluent API
|
||||||
|
* and handling common patterns automatically.
|
||||||
|
*/
|
||||||
|
import { IAugmentation, AugmentationResponse, ISenseAugmentation, IConduitAugmentation, IMemoryAugmentation, IWebSocketSupport, WebSocketConnection } from './types/augmentations.js';
|
||||||
|
/**
|
||||||
|
* Options for creating an augmentation
|
||||||
|
*/
|
||||||
|
export interface AugmentationOptions {
|
||||||
|
name: string;
|
||||||
|
description?: string;
|
||||||
|
enabled?: boolean;
|
||||||
|
autoRegister?: boolean;
|
||||||
|
autoInitialize?: boolean;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Factory for creating sense augmentations
|
||||||
|
*/
|
||||||
|
export declare function createSenseAugmentation(options: AugmentationOptions & {
|
||||||
|
processRawData?: (rawData: Buffer | string, dataType: string) => Promise<AugmentationResponse<{
|
||||||
|
nouns: string[];
|
||||||
|
verbs: string[];
|
||||||
|
}>> | AugmentationResponse<{
|
||||||
|
nouns: string[];
|
||||||
|
verbs: string[];
|
||||||
|
}>;
|
||||||
|
listenToFeed?: (feedUrl: string, callback: (data: {
|
||||||
|
nouns: string[];
|
||||||
|
verbs: string[];
|
||||||
|
}) => void) => Promise<void>;
|
||||||
|
}): ISenseAugmentation;
|
||||||
|
/**
|
||||||
|
* Factory for creating conduit augmentations
|
||||||
|
*/
|
||||||
|
export declare function createConduitAugmentation(options: AugmentationOptions & {
|
||||||
|
establishConnection?: (targetSystemId: string, config: Record<string, unknown>) => Promise<AugmentationResponse<WebSocketConnection>> | AugmentationResponse<WebSocketConnection>;
|
||||||
|
readData?: (query: Record<string, unknown>, options?: Record<string, unknown>) => Promise<AugmentationResponse<unknown>> | AugmentationResponse<unknown>;
|
||||||
|
writeData?: (data: Record<string, unknown>, options?: Record<string, unknown>) => Promise<AugmentationResponse<unknown>> | AugmentationResponse<unknown>;
|
||||||
|
monitorStream?: (streamId: string, callback: (data: unknown) => void) => Promise<void>;
|
||||||
|
}): IConduitAugmentation;
|
||||||
|
/**
|
||||||
|
* Factory for creating memory augmentations
|
||||||
|
*/
|
||||||
|
export declare function createMemoryAugmentation(options: AugmentationOptions & {
|
||||||
|
storeData?: (key: string, data: unknown, options?: Record<string, unknown>) => Promise<AugmentationResponse<boolean>> | AugmentationResponse<boolean>;
|
||||||
|
retrieveData?: (key: string, options?: Record<string, unknown>) => Promise<AugmentationResponse<unknown>> | AugmentationResponse<unknown>;
|
||||||
|
updateData?: (key: string, data: unknown, options?: Record<string, unknown>) => Promise<AugmentationResponse<boolean>> | AugmentationResponse<boolean>;
|
||||||
|
deleteData?: (key: string, options?: Record<string, unknown>) => Promise<AugmentationResponse<boolean>> | AugmentationResponse<boolean>;
|
||||||
|
listDataKeys?: (pattern?: string, options?: Record<string, unknown>) => Promise<AugmentationResponse<string[]>> | AugmentationResponse<string[]>;
|
||||||
|
search?: (query: unknown, k?: number, options?: Record<string, unknown>) => Promise<AugmentationResponse<Array<{
|
||||||
|
id: string;
|
||||||
|
score: number;
|
||||||
|
data: unknown;
|
||||||
|
}>>> | AugmentationResponse<Array<{
|
||||||
|
id: string;
|
||||||
|
score: number;
|
||||||
|
data: unknown;
|
||||||
|
}>>;
|
||||||
|
}): IMemoryAugmentation;
|
||||||
|
/**
|
||||||
|
* Factory for creating WebSocket-enabled augmentations
|
||||||
|
* This can be combined with other augmentation factories to create WebSocket-enabled versions
|
||||||
|
*/
|
||||||
|
export declare function addWebSocketSupport<T extends IAugmentation>(augmentation: T, options: {
|
||||||
|
connectWebSocket?: (url: string, protocols?: string | string[]) => Promise<WebSocketConnection>;
|
||||||
|
sendWebSocketMessage?: (connectionId: string, data: unknown) => Promise<void>;
|
||||||
|
onWebSocketMessage?: (connectionId: string, callback: (data: unknown) => void) => Promise<void>;
|
||||||
|
offWebSocketMessage?: (connectionId: string, callback: (data: unknown) => void) => Promise<void>;
|
||||||
|
closeWebSocket?: (connectionId: string, code?: number, reason?: string) => Promise<void>;
|
||||||
|
}): T & IWebSocketSupport;
|
||||||
|
/**
|
||||||
|
* Simplified function to execute an augmentation method with automatic error handling
|
||||||
|
* This provides a more concise way to execute augmentation methods compared to the full pipeline
|
||||||
|
*/
|
||||||
|
export declare function executeAugmentation<T, R>(augmentation: IAugmentation, method: string, ...args: any[]): Promise<AugmentationResponse<R>>;
|
||||||
|
/**
|
||||||
|
* Dynamically load augmentations from a module at runtime
|
||||||
|
* This allows for lazy-loading augmentations when needed instead of at build time
|
||||||
|
*/
|
||||||
|
export declare function loadAugmentationModule(modulePromise: Promise<any>, options?: {
|
||||||
|
autoRegister?: boolean;
|
||||||
|
autoInitialize?: boolean;
|
||||||
|
}): Promise<IAugmentation[]>;
|
||||||
|
|
@ -0,0 +1,342 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Factory
|
||||||
|
*
|
||||||
|
* This module provides a simplified factory for creating augmentations with minimal boilerplate.
|
||||||
|
* It reduces the complexity of creating and using augmentations by providing a fluent API
|
||||||
|
* and handling common patterns automatically.
|
||||||
|
*/
|
||||||
|
import { registerAugmentation } from './augmentationRegistry.js';
|
||||||
|
/**
|
||||||
|
* Base class for all augmentations created with the factory
|
||||||
|
* Handles common functionality like initialization, shutdown, and status
|
||||||
|
*/
|
||||||
|
class BaseAugmentation {
|
||||||
|
constructor(options) {
|
||||||
|
this.enabled = true;
|
||||||
|
this.isInitialized = false;
|
||||||
|
this.name = options.name;
|
||||||
|
this.description = options.description || `${options.name} augmentation`;
|
||||||
|
this.enabled = options.enabled !== false;
|
||||||
|
}
|
||||||
|
async initialize() {
|
||||||
|
if (this.isInitialized)
|
||||||
|
return;
|
||||||
|
this.isInitialized = true;
|
||||||
|
}
|
||||||
|
async shutDown() {
|
||||||
|
this.isInitialized = false;
|
||||||
|
}
|
||||||
|
async getStatus() {
|
||||||
|
return this.isInitialized ? 'active' : 'inactive';
|
||||||
|
}
|
||||||
|
async ensureInitialized() {
|
||||||
|
if (!this.isInitialized) {
|
||||||
|
await this.initialize();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Factory for creating sense augmentations
|
||||||
|
*/
|
||||||
|
export function createSenseAugmentation(options) {
|
||||||
|
const augmentation = new BaseAugmentation(options);
|
||||||
|
// Implement the sense augmentation methods
|
||||||
|
augmentation.processRawData = async (rawData, dataType) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.processRawData) {
|
||||||
|
const result = options.processRawData(rawData, dataType);
|
||||||
|
return result instanceof Promise ? await result : result;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: { nouns: [], verbs: [] },
|
||||||
|
error: 'processRawData not implemented'
|
||||||
|
};
|
||||||
|
};
|
||||||
|
augmentation.listenToFeed = async (feedUrl, callback) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.listenToFeed) {
|
||||||
|
return options.listenToFeed(feedUrl, callback);
|
||||||
|
}
|
||||||
|
throw new Error('listenToFeed not implemented');
|
||||||
|
};
|
||||||
|
// Auto-register if requested
|
||||||
|
if (options.autoRegister) {
|
||||||
|
registerAugmentation(augmentation);
|
||||||
|
// Auto-initialize if requested
|
||||||
|
if (options.autoInitialize) {
|
||||||
|
augmentation.initialize().catch((error) => {
|
||||||
|
console.error(`Failed to initialize augmentation ${augmentation.name}:`, error);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return augmentation;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Factory for creating conduit augmentations
|
||||||
|
*/
|
||||||
|
export function createConduitAugmentation(options) {
|
||||||
|
const augmentation = new BaseAugmentation(options);
|
||||||
|
// Implement the conduit augmentation methods
|
||||||
|
augmentation.establishConnection = async (targetSystemId, config) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.establishConnection) {
|
||||||
|
const result = options.establishConnection(targetSystemId, config);
|
||||||
|
return result instanceof Promise ? await result : result;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: null,
|
||||||
|
error: 'establishConnection not implemented'
|
||||||
|
};
|
||||||
|
};
|
||||||
|
augmentation.readData = async (query, opts) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.readData) {
|
||||||
|
const result = options.readData(query, opts);
|
||||||
|
return result instanceof Promise ? await result : result;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: null,
|
||||||
|
error: 'readData not implemented'
|
||||||
|
};
|
||||||
|
};
|
||||||
|
augmentation.writeData = async (data, opts) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.writeData) {
|
||||||
|
const result = options.writeData(data, opts);
|
||||||
|
return result instanceof Promise ? await result : result;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: null,
|
||||||
|
error: 'writeData not implemented'
|
||||||
|
};
|
||||||
|
};
|
||||||
|
augmentation.monitorStream = async (streamId, callback) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.monitorStream) {
|
||||||
|
return options.monitorStream(streamId, callback);
|
||||||
|
}
|
||||||
|
throw new Error('monitorStream not implemented');
|
||||||
|
};
|
||||||
|
// Auto-register if requested
|
||||||
|
if (options.autoRegister) {
|
||||||
|
registerAugmentation(augmentation);
|
||||||
|
// Auto-initialize if requested
|
||||||
|
if (options.autoInitialize) {
|
||||||
|
augmentation.initialize().catch((error) => {
|
||||||
|
console.error(`Failed to initialize augmentation ${augmentation.name}:`, error);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return augmentation;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Factory for creating memory augmentations
|
||||||
|
*/
|
||||||
|
export function createMemoryAugmentation(options) {
|
||||||
|
const augmentation = new BaseAugmentation(options);
|
||||||
|
// Implement the memory augmentation methods
|
||||||
|
augmentation.storeData = async (key, data, opts) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.storeData) {
|
||||||
|
const result = options.storeData(key, data, opts);
|
||||||
|
return result instanceof Promise ? await result : result;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: false,
|
||||||
|
error: 'storeData not implemented'
|
||||||
|
};
|
||||||
|
};
|
||||||
|
augmentation.retrieveData = async (key, opts) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.retrieveData) {
|
||||||
|
const result = options.retrieveData(key, opts);
|
||||||
|
return result instanceof Promise ? await result : result;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: null,
|
||||||
|
error: 'retrieveData not implemented'
|
||||||
|
};
|
||||||
|
};
|
||||||
|
augmentation.updateData = async (key, data, opts) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.updateData) {
|
||||||
|
const result = options.updateData(key, data, opts);
|
||||||
|
return result instanceof Promise ? await result : result;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: false,
|
||||||
|
error: 'updateData not implemented'
|
||||||
|
};
|
||||||
|
};
|
||||||
|
augmentation.deleteData = async (key, opts) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.deleteData) {
|
||||||
|
const result = options.deleteData(key, opts);
|
||||||
|
return result instanceof Promise ? await result : result;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: false,
|
||||||
|
error: 'deleteData not implemented'
|
||||||
|
};
|
||||||
|
};
|
||||||
|
augmentation.listDataKeys = async (pattern, opts) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.listDataKeys) {
|
||||||
|
const result = options.listDataKeys(pattern, opts);
|
||||||
|
return result instanceof Promise ? await result : result;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: [],
|
||||||
|
error: 'listDataKeys not implemented'
|
||||||
|
};
|
||||||
|
};
|
||||||
|
augmentation.search = async (query, k, opts) => {
|
||||||
|
await augmentation.ensureInitialized();
|
||||||
|
if (options.search) {
|
||||||
|
const result = options.search(query, k, opts);
|
||||||
|
return result instanceof Promise ? await result : result;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: [],
|
||||||
|
error: 'search not implemented'
|
||||||
|
};
|
||||||
|
};
|
||||||
|
// Auto-register if requested
|
||||||
|
if (options.autoRegister) {
|
||||||
|
registerAugmentation(augmentation);
|
||||||
|
// Auto-initialize if requested
|
||||||
|
if (options.autoInitialize) {
|
||||||
|
augmentation.initialize().catch((error) => {
|
||||||
|
console.error(`Failed to initialize augmentation ${augmentation.name}:`, error);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return augmentation;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Factory for creating WebSocket-enabled augmentations
|
||||||
|
* This can be combined with other augmentation factories to create WebSocket-enabled versions
|
||||||
|
*/
|
||||||
|
export function addWebSocketSupport(augmentation, options) {
|
||||||
|
const wsAugmentation = augmentation;
|
||||||
|
// Add WebSocket methods
|
||||||
|
wsAugmentation.connectWebSocket = async (url, protocols) => {
|
||||||
|
await augmentation.ensureInitialized?.();
|
||||||
|
if (options.connectWebSocket) {
|
||||||
|
return options.connectWebSocket(url, protocols);
|
||||||
|
}
|
||||||
|
throw new Error('connectWebSocket not implemented');
|
||||||
|
};
|
||||||
|
wsAugmentation.sendWebSocketMessage = async (connectionId, data) => {
|
||||||
|
await augmentation.ensureInitialized?.();
|
||||||
|
if (options.sendWebSocketMessage) {
|
||||||
|
return options.sendWebSocketMessage(connectionId, data);
|
||||||
|
}
|
||||||
|
throw new Error('sendWebSocketMessage not implemented');
|
||||||
|
};
|
||||||
|
wsAugmentation.onWebSocketMessage = async (connectionId, callback) => {
|
||||||
|
await augmentation.ensureInitialized?.();
|
||||||
|
if (options.onWebSocketMessage) {
|
||||||
|
return options.onWebSocketMessage(connectionId, callback);
|
||||||
|
}
|
||||||
|
throw new Error('onWebSocketMessage not implemented');
|
||||||
|
};
|
||||||
|
wsAugmentation.offWebSocketMessage = async (connectionId, callback) => {
|
||||||
|
await augmentation.ensureInitialized?.();
|
||||||
|
if (options.offWebSocketMessage) {
|
||||||
|
return options.offWebSocketMessage(connectionId, callback);
|
||||||
|
}
|
||||||
|
throw new Error('offWebSocketMessage not implemented');
|
||||||
|
};
|
||||||
|
wsAugmentation.closeWebSocket = async (connectionId, code, reason) => {
|
||||||
|
await augmentation.ensureInitialized?.();
|
||||||
|
if (options.closeWebSocket) {
|
||||||
|
return options.closeWebSocket(connectionId, code, reason);
|
||||||
|
}
|
||||||
|
throw new Error('closeWebSocket not implemented');
|
||||||
|
};
|
||||||
|
return wsAugmentation;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Simplified function to execute an augmentation method with automatic error handling
|
||||||
|
* This provides a more concise way to execute augmentation methods compared to the full pipeline
|
||||||
|
*/
|
||||||
|
export async function executeAugmentation(augmentation, method, ...args) {
|
||||||
|
try {
|
||||||
|
if (!augmentation.enabled) {
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: null,
|
||||||
|
error: `Augmentation ${augmentation.name} is disabled`
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if (typeof augmentation[method] !== 'function') {
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: null,
|
||||||
|
error: `Method ${method} not found on augmentation ${augmentation.name}`
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const result = await augmentation[method](...args);
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.error(`Error executing ${method} on ${augmentation.name}:`, error);
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: null,
|
||||||
|
error: error instanceof Error ? error.message : String(error)
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Dynamically load augmentations from a module at runtime
|
||||||
|
* This allows for lazy-loading augmentations when needed instead of at build time
|
||||||
|
*/
|
||||||
|
export async function loadAugmentationModule(modulePromise, options = {}) {
|
||||||
|
try {
|
||||||
|
const module = await modulePromise;
|
||||||
|
const augmentations = [];
|
||||||
|
// Extract augmentations from the module
|
||||||
|
for (const key in module) {
|
||||||
|
const exported = module[key];
|
||||||
|
// Skip non-objects and null
|
||||||
|
if (!exported || typeof exported !== 'object') {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Check if it's an augmentation
|
||||||
|
if (typeof exported.name === 'string' &&
|
||||||
|
typeof exported.initialize === 'function' &&
|
||||||
|
typeof exported.shutDown === 'function' &&
|
||||||
|
typeof exported.getStatus === 'function') {
|
||||||
|
augmentations.push(exported);
|
||||||
|
// Auto-register if requested
|
||||||
|
if (options.autoRegister) {
|
||||||
|
registerAugmentation(exported);
|
||||||
|
// Auto-initialize if requested
|
||||||
|
if (options.autoInitialize) {
|
||||||
|
exported.initialize().catch((error) => {
|
||||||
|
console.error(`Failed to initialize augmentation ${exported.name}:`, error);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return augmentations;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.error('Error loading augmentation module:', error);
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=augmentationFactory.js.map
|
||||||
File diff suppressed because one or more lines are too long
87
.recovery-workspace/dist-backup-20250910-141917/augmentationManager.d.ts
vendored
Normal file
87
.recovery-workspace/dist-backup-20250910-141917/augmentationManager.d.ts
vendored
Normal file
|
|
@ -0,0 +1,87 @@
|
||||||
|
/**
|
||||||
|
* Type-safe augmentation management system for Brainy
|
||||||
|
* Provides a clean API for managing augmentations without string literals
|
||||||
|
*/
|
||||||
|
import { IAugmentation, AugmentationType } from './types/augmentations.js';
|
||||||
|
export interface AugmentationInfo {
|
||||||
|
name: string;
|
||||||
|
type: string;
|
||||||
|
enabled: boolean;
|
||||||
|
description: string;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Type-safe augmentation manager
|
||||||
|
* Accessed via brain.augmentations for all management operations
|
||||||
|
*/
|
||||||
|
export declare class AugmentationManager {
|
||||||
|
private pipeline;
|
||||||
|
/**
|
||||||
|
* List all registered augmentations with their status
|
||||||
|
* @returns Array of augmentation information
|
||||||
|
*/
|
||||||
|
list(): AugmentationInfo[];
|
||||||
|
/**
|
||||||
|
* Get information about a specific augmentation
|
||||||
|
* @param name The augmentation name
|
||||||
|
* @returns Augmentation info or undefined if not found
|
||||||
|
*/
|
||||||
|
get(name: string): AugmentationInfo | undefined;
|
||||||
|
/**
|
||||||
|
* Check if an augmentation is enabled
|
||||||
|
* @param name The augmentation name
|
||||||
|
* @returns True if enabled, false otherwise
|
||||||
|
*/
|
||||||
|
isEnabled(name: string): boolean;
|
||||||
|
/**
|
||||||
|
* Enable a specific augmentation
|
||||||
|
* @param name The augmentation name
|
||||||
|
* @returns True if successfully enabled
|
||||||
|
*/
|
||||||
|
enable(name: string): boolean;
|
||||||
|
/**
|
||||||
|
* Disable a specific augmentation
|
||||||
|
* @param name The augmentation name
|
||||||
|
* @returns True if successfully disabled
|
||||||
|
*/
|
||||||
|
disable(name: string): boolean;
|
||||||
|
/**
|
||||||
|
* Remove an augmentation from the pipeline
|
||||||
|
* @param name The augmentation name
|
||||||
|
* @returns True if successfully removed
|
||||||
|
*/
|
||||||
|
remove(name: string): boolean;
|
||||||
|
/**
|
||||||
|
* Enable all augmentations of a specific type
|
||||||
|
* @param type The augmentation type
|
||||||
|
* @returns Number of augmentations enabled
|
||||||
|
*/
|
||||||
|
enableType(type: AugmentationType): number;
|
||||||
|
/**
|
||||||
|
* Disable all augmentations of a specific type
|
||||||
|
* @param type The augmentation type
|
||||||
|
* @returns Number of augmentations disabled
|
||||||
|
*/
|
||||||
|
disableType(type: AugmentationType): number;
|
||||||
|
/**
|
||||||
|
* Get all augmentations of a specific type
|
||||||
|
* @param type The augmentation type
|
||||||
|
* @returns Array of augmentations of that type
|
||||||
|
*/
|
||||||
|
listByType(type: AugmentationType): AugmentationInfo[];
|
||||||
|
/**
|
||||||
|
* Get all enabled augmentations
|
||||||
|
* @returns Array of enabled augmentations
|
||||||
|
*/
|
||||||
|
listEnabled(): AugmentationInfo[];
|
||||||
|
/**
|
||||||
|
* Get all disabled augmentations
|
||||||
|
* @returns Array of disabled augmentations
|
||||||
|
*/
|
||||||
|
listDisabled(): AugmentationInfo[];
|
||||||
|
/**
|
||||||
|
* Register a new augmentation (internal use)
|
||||||
|
* @param augmentation The augmentation to register
|
||||||
|
*/
|
||||||
|
register(augmentation: IAugmentation): void;
|
||||||
|
}
|
||||||
|
export { AugmentationType } from './types/augmentations.js';
|
||||||
|
|
@ -0,0 +1,117 @@
|
||||||
|
/**
|
||||||
|
* Type-safe augmentation management system for Brainy
|
||||||
|
* Provides a clean API for managing augmentations without string literals
|
||||||
|
*/
|
||||||
|
import { augmentationPipeline } from './augmentationPipeline.js';
|
||||||
|
/**
|
||||||
|
* Type-safe augmentation manager
|
||||||
|
* Accessed via brain.augmentations for all management operations
|
||||||
|
*/
|
||||||
|
export class AugmentationManager {
|
||||||
|
constructor() {
|
||||||
|
this.pipeline = augmentationPipeline;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* List all registered augmentations with their status
|
||||||
|
* @returns Array of augmentation information
|
||||||
|
*/
|
||||||
|
list() {
|
||||||
|
// Deprecated: use brain.augmentations instead
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get information about a specific augmentation
|
||||||
|
* @param name The augmentation name
|
||||||
|
* @returns Augmentation info or undefined if not found
|
||||||
|
*/
|
||||||
|
get(name) {
|
||||||
|
const all = this.list();
|
||||||
|
return all.find(a => a.name === name);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if an augmentation is enabled
|
||||||
|
* @param name The augmentation name
|
||||||
|
* @returns True if enabled, false otherwise
|
||||||
|
*/
|
||||||
|
isEnabled(name) {
|
||||||
|
const aug = this.get(name);
|
||||||
|
return aug?.enabled ?? false;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Enable a specific augmentation
|
||||||
|
* @param name The augmentation name
|
||||||
|
* @returns True if successfully enabled
|
||||||
|
*/
|
||||||
|
enable(name) {
|
||||||
|
// Deprecated: use brain.augmentations instead
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Disable a specific augmentation
|
||||||
|
* @param name The augmentation name
|
||||||
|
* @returns True if successfully disabled
|
||||||
|
*/
|
||||||
|
disable(name) {
|
||||||
|
// Deprecated: use brain.augmentations instead
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Remove an augmentation from the pipeline
|
||||||
|
* @param name The augmentation name
|
||||||
|
* @returns True if successfully removed
|
||||||
|
*/
|
||||||
|
remove(name) {
|
||||||
|
// Deprecated: use brain.augmentations instead
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Enable all augmentations of a specific type
|
||||||
|
* @param type The augmentation type
|
||||||
|
* @returns Number of augmentations enabled
|
||||||
|
*/
|
||||||
|
enableType(type) {
|
||||||
|
// Deprecated: use brain.augmentations instead
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Disable all augmentations of a specific type
|
||||||
|
* @param type The augmentation type
|
||||||
|
* @returns Number of augmentations disabled
|
||||||
|
*/
|
||||||
|
disableType(type) {
|
||||||
|
// Deprecated: use brain.augmentations instead
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get all augmentations of a specific type
|
||||||
|
* @param type The augmentation type
|
||||||
|
* @returns Array of augmentations of that type
|
||||||
|
*/
|
||||||
|
listByType(type) {
|
||||||
|
return this.list().filter(a => a.type === type);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get all enabled augmentations
|
||||||
|
* @returns Array of enabled augmentations
|
||||||
|
*/
|
||||||
|
listEnabled() {
|
||||||
|
return this.list().filter(a => a.enabled);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get all disabled augmentations
|
||||||
|
* @returns Array of disabled augmentations
|
||||||
|
*/
|
||||||
|
listDisabled() {
|
||||||
|
return this.list().filter(a => !a.enabled);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Register a new augmentation (internal use)
|
||||||
|
* @param augmentation The augmentation to register
|
||||||
|
*/
|
||||||
|
register(augmentation) {
|
||||||
|
// Deprecated: use brain.augmentations instead
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Export types for external use
|
||||||
|
export { AugmentationType } from './types/augmentations.js';
|
||||||
|
//# sourceMappingURL=augmentationManager.js.map
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
{"version":3,"file":"augmentationManager.js","sourceRoot":"","sources":["../src/augmentationManager.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAGH,OAAO,EAAE,oBAAoB,EAAE,MAAM,2BAA2B,CAAA;AAShE;;;GAGG;AACH,MAAM,OAAO,mBAAmB;IAAhC;QACU,aAAQ,GAAG,oBAAoB,CAAA;IAiHzC,CAAC;IA/GC;;;OAGG;IACH,IAAI;QACF,8CAA8C;QAC9C,OAAO,EAAE,CAAA;IACX,CAAC;IAED;;;;OAIG;IACH,GAAG,CAAC,IAAY;QACd,MAAM,GAAG,GAAG,IAAI,CAAC,IAAI,EAAE,CAAA;QACvB,OAAO,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,IAAI,CAAC,CAAA;IACvC,CAAC;IAED;;;;OAIG;IACH,SAAS,CAAC,IAAY;QACpB,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,CAAA;QAC1B,OAAO,GAAG,EAAE,OAAO,IAAI,KAAK,CAAA;IAC9B,CAAC;IAED;;;;OAIG;IACH,MAAM,CAAC,IAAY;QACjB,8CAA8C;QAC9C,OAAO,KAAK,CAAA;IACd,CAAC;IAED;;;;OAIG;IACH,OAAO,CAAC,IAAY;QAClB,8CAA8C;QAC9C,OAAO,KAAK,CAAA;IACd,CAAC;IAED;;;;OAIG;IACH,MAAM,CAAC,IAAY;QACjB,8CAA8C;QAC9C,OAAO,IAAI,CAAA;IACb,CAAC;IAED;;;;OAIG;IACH,UAAU,CAAC,IAAsB;QAC/B,8CAA8C;QAC9C,OAAO,CAAC,CAAA;IACV,CAAC;IAED;;;;OAIG;IACH,WAAW,CAAC,IAAsB;QAChC,8CAA8C;QAC9C,OAAO,CAAC,CAAA;IACV,CAAC;IAED;;;;OAIG;IACH,UAAU,CAAC,IAAsB;QAC/B,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,IAAI,CAAC,CAAA;IACjD,CAAC;IAED;;;OAGG;IACH,WAAW;QACT,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,CAAA;IAC3C,CAAC;IAED;;;OAGG;IACH,YAAY;QACV,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAA;IAC5C,CAAC;IAED;;;OAGG;IACH,QAAQ,CAAC,YAA2B;QAClC,8CAA8C;IAChD,CAAC;CACF;AAED,gCAAgC;AAChC,OAAO,EAAE,gBAAgB,EAAE,MAAM,0BAA0B,CAAA"}
|
||||||
38
.recovery-workspace/dist-backup-20250910-141917/augmentationPipeline.d.ts
vendored
Normal file
38
.recovery-workspace/dist-backup-20250910-141917/augmentationPipeline.d.ts
vendored
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Pipeline (Compatibility Layer)
|
||||||
|
*
|
||||||
|
* @deprecated This file provides backward compatibility for code that imports
|
||||||
|
* from augmentationPipeline. All new code should use AugmentationRegistry directly.
|
||||||
|
*
|
||||||
|
* This minimal implementation redirects to the new AugmentationRegistry system.
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* Execution mode for pipeline operations
|
||||||
|
*/
|
||||||
|
export declare enum ExecutionMode {
|
||||||
|
SEQUENTIAL = "sequential",
|
||||||
|
PARALLEL = "parallel",
|
||||||
|
FIRST_SUCCESS = "firstSuccess",
|
||||||
|
FIRST_RESULT = "firstResult",
|
||||||
|
THREADED = "threaded"
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Options for pipeline execution
|
||||||
|
*/
|
||||||
|
export interface PipelineOptions {
|
||||||
|
mode?: ExecutionMode;
|
||||||
|
timeout?: number;
|
||||||
|
retries?: number;
|
||||||
|
throwOnError?: boolean;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Minimal Cortex class for backward compatibility
|
||||||
|
* Redirects all operations to the new AugmentationRegistry system
|
||||||
|
*/
|
||||||
|
export declare class Cortex {
|
||||||
|
private static instance?;
|
||||||
|
constructor();
|
||||||
|
}
|
||||||
|
export declare const cortex: Cortex;
|
||||||
|
export declare const AugmentationPipeline: typeof Cortex;
|
||||||
|
export declare const augmentationPipeline: Cortex;
|
||||||
|
|
@ -0,0 +1,39 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Pipeline (Compatibility Layer)
|
||||||
|
*
|
||||||
|
* @deprecated This file provides backward compatibility for code that imports
|
||||||
|
* from augmentationPipeline. All new code should use AugmentationRegistry directly.
|
||||||
|
*
|
||||||
|
* This minimal implementation redirects to the new AugmentationRegistry system.
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* Execution mode for pipeline operations
|
||||||
|
*/
|
||||||
|
export var ExecutionMode;
|
||||||
|
(function (ExecutionMode) {
|
||||||
|
ExecutionMode["SEQUENTIAL"] = "sequential";
|
||||||
|
ExecutionMode["PARALLEL"] = "parallel";
|
||||||
|
ExecutionMode["FIRST_SUCCESS"] = "firstSuccess";
|
||||||
|
ExecutionMode["FIRST_RESULT"] = "firstResult";
|
||||||
|
ExecutionMode["THREADED"] = "threaded";
|
||||||
|
})(ExecutionMode || (ExecutionMode = {}));
|
||||||
|
/**
|
||||||
|
* Minimal Cortex class for backward compatibility
|
||||||
|
* Redirects all operations to the new AugmentationRegistry system
|
||||||
|
*/
|
||||||
|
export class Cortex {
|
||||||
|
constructor() {
|
||||||
|
if (Cortex.instance) {
|
||||||
|
return Cortex.instance;
|
||||||
|
}
|
||||||
|
Cortex.instance = this;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Create and export a default instance of the cortex
|
||||||
|
export const cortex = new Cortex();
|
||||||
|
// Backward compatibility exports
|
||||||
|
export const AugmentationPipeline = Cortex;
|
||||||
|
export const augmentationPipeline = cortex;
|
||||||
|
// Export types for compatibility (avoid duplicate export)
|
||||||
|
// PipelineOptions already exported above
|
||||||
|
//# sourceMappingURL=augmentationPipeline.js.map
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
{"version":3,"file":"augmentationPipeline.js","sourceRoot":"","sources":["../src/augmentationPipeline.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAIH;;GAEG;AACH,MAAM,CAAN,IAAY,aAMX;AAND,WAAY,aAAa;IACvB,0CAAyB,CAAA;IACzB,sCAAqB,CAAA;IACrB,+CAA8B,CAAA;IAC9B,6CAA4B,CAAA;IAC5B,sCAAqB,CAAA;AACvB,CAAC,EANW,aAAa,KAAb,aAAa,QAMxB;AAYD;;;GAGG;AACH,MAAM,OAAO,MAAM;IAGjB;QACE,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;YACpB,OAAO,MAAM,CAAC,QAAQ,CAAA;QACxB,CAAC;QACD,MAAM,CAAC,QAAQ,GAAG,IAAI,CAAA;IACxB,CAAC;CAYF;AAED,qDAAqD;AACrD,MAAM,CAAC,MAAM,MAAM,GAAG,IAAI,MAAM,EAAE,CAAA;AAElC,iCAAiC;AACjC,MAAM,CAAC,MAAM,oBAAoB,GAAG,MAAM,CAAA;AAC1C,MAAM,CAAC,MAAM,oBAAoB,GAAG,MAAM,CAAA;AAE1C,0DAA0D;AAC1D,yCAAyC"}
|
||||||
38
.recovery-workspace/dist-backup-20250910-141917/augmentationRegistry.d.ts
vendored
Normal file
38
.recovery-workspace/dist-backup-20250910-141917/augmentationRegistry.d.ts
vendored
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Registry (Compatibility Layer)
|
||||||
|
*
|
||||||
|
* @deprecated This module provides backward compatibility for old augmentation
|
||||||
|
* loading code. All new code should use the AugmentationRegistry class directly
|
||||||
|
* on Brainy instances.
|
||||||
|
*/
|
||||||
|
import { BrainyAugmentation } from './types/augmentations.js';
|
||||||
|
/**
|
||||||
|
* Registry of all available augmentations (for compatibility)
|
||||||
|
* @deprecated Use brain.augmentations instead
|
||||||
|
*/
|
||||||
|
export declare const availableAugmentations: any[];
|
||||||
|
/**
|
||||||
|
* Compatibility wrapper for registerAugmentation
|
||||||
|
* @deprecated Use brain.augmentations.register instead
|
||||||
|
*/
|
||||||
|
export declare function registerAugmentation<T extends BrainyAugmentation>(augmentation: T): T;
|
||||||
|
/**
|
||||||
|
* Sets the default pipeline instance (compatibility)
|
||||||
|
* @deprecated Use brain.augmentations instead
|
||||||
|
*/
|
||||||
|
export declare function setDefaultPipeline(pipeline: any): void;
|
||||||
|
/**
|
||||||
|
* Initializes the augmentation pipeline (compatibility)
|
||||||
|
* @deprecated Use brain.augmentations instead
|
||||||
|
*/
|
||||||
|
export declare function initializeAugmentationPipeline(pipelineInstance?: any): any;
|
||||||
|
/**
|
||||||
|
* Enables or disables an augmentation by name (compatibility)
|
||||||
|
* @deprecated Use brain.augmentations instead
|
||||||
|
*/
|
||||||
|
export declare function setAugmentationEnabled(name: string, enabled: boolean): boolean;
|
||||||
|
/**
|
||||||
|
* Gets all augmentations of a specific type (compatibility)
|
||||||
|
* @deprecated Use brain.augmentations instead
|
||||||
|
*/
|
||||||
|
export declare function getAugmentationsByType(type: any): any[];
|
||||||
|
|
@ -0,0 +1,54 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Registry (Compatibility Layer)
|
||||||
|
*
|
||||||
|
* @deprecated This module provides backward compatibility for old augmentation
|
||||||
|
* loading code. All new code should use the AugmentationRegistry class directly
|
||||||
|
* on Brainy instances.
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* Registry of all available augmentations (for compatibility)
|
||||||
|
* @deprecated Use brain.augmentations instead
|
||||||
|
*/
|
||||||
|
export const availableAugmentations = [];
|
||||||
|
/**
|
||||||
|
* Compatibility wrapper for registerAugmentation
|
||||||
|
* @deprecated Use brain.augmentations.register instead
|
||||||
|
*/
|
||||||
|
export function registerAugmentation(augmentation) {
|
||||||
|
console.warn('registerAugmentation is deprecated. Use brain.augmentations.register instead.');
|
||||||
|
// For compatibility, just add to the list (but it won't actually do anything)
|
||||||
|
availableAugmentations.push(augmentation);
|
||||||
|
return augmentation;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Sets the default pipeline instance (compatibility)
|
||||||
|
* @deprecated Use brain.augmentations instead
|
||||||
|
*/
|
||||||
|
export function setDefaultPipeline(pipeline) {
|
||||||
|
console.warn('setDefaultPipeline is deprecated. Use brain.augmentations instead.');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Initializes the augmentation pipeline (compatibility)
|
||||||
|
* @deprecated Use brain.augmentations instead
|
||||||
|
*/
|
||||||
|
export function initializeAugmentationPipeline(pipelineInstance) {
|
||||||
|
console.warn('initializeAugmentationPipeline is deprecated. Use brain.augmentations instead.');
|
||||||
|
return pipelineInstance || {};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Enables or disables an augmentation by name (compatibility)
|
||||||
|
* @deprecated Use brain.augmentations instead
|
||||||
|
*/
|
||||||
|
export function setAugmentationEnabled(name, enabled) {
|
||||||
|
console.warn('setAugmentationEnabled is deprecated. Use brain.augmentations instead.');
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Gets all augmentations of a specific type (compatibility)
|
||||||
|
* @deprecated Use brain.augmentations instead
|
||||||
|
*/
|
||||||
|
export function getAugmentationsByType(type) {
|
||||||
|
console.warn('getAugmentationsByType is deprecated. Use brain.augmentations instead.');
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=augmentationRegistry.js.map
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
{"version":3,"file":"augmentationRegistry.js","sourceRoot":"","sources":["../src/augmentationRegistry.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAIH;;;GAGG;AACH,MAAM,CAAC,MAAM,sBAAsB,GAAU,EAAE,CAAA;AAE/C;;;GAGG;AACH,MAAM,UAAU,oBAAoB,CAA+B,YAAe;IAChF,OAAO,CAAC,IAAI,CAAC,+EAA+E,CAAC,CAAA;IAE7F,8EAA8E;IAC9E,sBAAsB,CAAC,IAAI,CAAC,YAAY,CAAC,CAAA;IAEzC,OAAO,YAAY,CAAA;AACrB,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,kBAAkB,CAAC,QAAa;IAC9C,OAAO,CAAC,IAAI,CAAC,oEAAoE,CAAC,CAAA;AACpF,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,8BAA8B,CAAC,gBAAsB;IACnE,OAAO,CAAC,IAAI,CAAC,gFAAgF,CAAC,CAAA;IAC9F,OAAO,gBAAgB,IAAI,EAAE,CAAA;AAC/B,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,sBAAsB,CAAC,IAAY,EAAE,OAAgB;IACnE,OAAO,CAAC,IAAI,CAAC,wEAAwE,CAAC,CAAA;IACtF,OAAO,KAAK,CAAA;AACd,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,sBAAsB,CAAC,IAAS;IAC9C,OAAO,CAAC,IAAI,CAAC,wEAAwE,CAAC,CAAA;IACtF,OAAO,EAAE,CAAA;AACX,CAAC"}
|
||||||
146
.recovery-workspace/dist-backup-20250910-141917/augmentationRegistryLoader.d.ts
vendored
Normal file
146
.recovery-workspace/dist-backup-20250910-141917/augmentationRegistryLoader.d.ts
vendored
Normal file
|
|
@ -0,0 +1,146 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Registry Loader
|
||||||
|
*
|
||||||
|
* This module provides functionality for loading augmentation registrations
|
||||||
|
* at build time. It's designed to be used with build tools like webpack or rollup
|
||||||
|
* to automatically discover and register augmentations.
|
||||||
|
*/
|
||||||
|
import { IAugmentation } from './types/augmentations.js';
|
||||||
|
/**
|
||||||
|
* Options for the augmentation registry loader
|
||||||
|
*/
|
||||||
|
export interface AugmentationRegistryLoaderOptions {
|
||||||
|
/**
|
||||||
|
* Whether to automatically initialize the augmentations after loading
|
||||||
|
* @default false
|
||||||
|
*/
|
||||||
|
autoInitialize?: boolean;
|
||||||
|
/**
|
||||||
|
* Whether to log debug information during loading
|
||||||
|
* @default false
|
||||||
|
*/
|
||||||
|
debug?: boolean;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Result of loading augmentations
|
||||||
|
*/
|
||||||
|
export interface AugmentationLoadResult {
|
||||||
|
/**
|
||||||
|
* The augmentations that were loaded
|
||||||
|
*/
|
||||||
|
augmentations: IAugmentation[];
|
||||||
|
/**
|
||||||
|
* Any errors that occurred during loading
|
||||||
|
*/
|
||||||
|
errors: Error[];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Loads augmentations from the specified modules
|
||||||
|
*
|
||||||
|
* This function is designed to be used with build tools like webpack or rollup
|
||||||
|
* to automatically discover and register augmentations.
|
||||||
|
*
|
||||||
|
* @param modules An object containing modules with augmentations to register
|
||||||
|
* @param options Options for the loader
|
||||||
|
* @returns A promise that resolves with the result of loading the augmentations
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* // webpack.config.js
|
||||||
|
* const { AugmentationRegistryPlugin } = require('brainy/dist/webpack');
|
||||||
|
*
|
||||||
|
* module.exports = {
|
||||||
|
* // ... other webpack config
|
||||||
|
* plugins: [
|
||||||
|
* new AugmentationRegistryPlugin({
|
||||||
|
* // Pattern to match files containing augmentations
|
||||||
|
* pattern: /augmentation\.js$/,
|
||||||
|
* // Options for the loader
|
||||||
|
* options: {
|
||||||
|
* autoInitialize: true,
|
||||||
|
* debug: true
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
* ]
|
||||||
|
* };
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
export declare function loadAugmentationsFromModules(modules: Record<string, any>, options?: AugmentationRegistryLoaderOptions): Promise<AugmentationLoadResult>;
|
||||||
|
/**
|
||||||
|
* Creates a webpack plugin for automatically loading augmentations
|
||||||
|
*
|
||||||
|
* @param options Options for the plugin
|
||||||
|
* @returns A webpack plugin
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* // webpack.config.js
|
||||||
|
* const { createAugmentationRegistryPlugin } = require('brainy/dist/webpack');
|
||||||
|
*
|
||||||
|
* module.exports = {
|
||||||
|
* // ... other webpack config
|
||||||
|
* plugins: [
|
||||||
|
* createAugmentationRegistryPlugin({
|
||||||
|
* pattern: /augmentation\.js$/,
|
||||||
|
* options: {
|
||||||
|
* autoInitialize: true,
|
||||||
|
* debug: true
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
* ]
|
||||||
|
* };
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
export declare function createAugmentationRegistryPlugin(options: {
|
||||||
|
/**
|
||||||
|
* Pattern to match files containing augmentations
|
||||||
|
*/
|
||||||
|
pattern: RegExp;
|
||||||
|
/**
|
||||||
|
* Options for the loader
|
||||||
|
*/
|
||||||
|
options?: AugmentationRegistryLoaderOptions;
|
||||||
|
}): {
|
||||||
|
name: string;
|
||||||
|
pattern: RegExp;
|
||||||
|
options: AugmentationRegistryLoaderOptions;
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Creates a rollup plugin for automatically loading augmentations
|
||||||
|
*
|
||||||
|
* @param options Options for the plugin
|
||||||
|
* @returns A rollup plugin
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* // rollup.config.js
|
||||||
|
* import { createAugmentationRegistryRollupPlugin } from 'brainy/dist/rollup';
|
||||||
|
*
|
||||||
|
* export default {
|
||||||
|
* // ... other rollup config
|
||||||
|
* plugins: [
|
||||||
|
* createAugmentationRegistryRollupPlugin({
|
||||||
|
* pattern: /augmentation\.js$/,
|
||||||
|
* options: {
|
||||||
|
* autoInitialize: true,
|
||||||
|
* debug: true
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
* ]
|
||||||
|
* };
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
export declare function createAugmentationRegistryRollupPlugin(options: {
|
||||||
|
/**
|
||||||
|
* Pattern to match files containing augmentations
|
||||||
|
*/
|
||||||
|
pattern: RegExp;
|
||||||
|
/**
|
||||||
|
* Options for the loader
|
||||||
|
*/
|
||||||
|
options?: AugmentationRegistryLoaderOptions;
|
||||||
|
}): {
|
||||||
|
name: string;
|
||||||
|
pattern: RegExp;
|
||||||
|
options: AugmentationRegistryLoaderOptions;
|
||||||
|
};
|
||||||
|
|
@ -0,0 +1,213 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Registry Loader
|
||||||
|
*
|
||||||
|
* This module provides functionality for loading augmentation registrations
|
||||||
|
* at build time. It's designed to be used with build tools like webpack or rollup
|
||||||
|
* to automatically discover and register augmentations.
|
||||||
|
*/
|
||||||
|
import { registerAugmentation } from './augmentationRegistry.js';
|
||||||
|
/**
|
||||||
|
* Default options for the augmentation registry loader
|
||||||
|
*/
|
||||||
|
const DEFAULT_OPTIONS = {
|
||||||
|
autoInitialize: false,
|
||||||
|
debug: false
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Loads augmentations from the specified modules
|
||||||
|
*
|
||||||
|
* This function is designed to be used with build tools like webpack or rollup
|
||||||
|
* to automatically discover and register augmentations.
|
||||||
|
*
|
||||||
|
* @param modules An object containing modules with augmentations to register
|
||||||
|
* @param options Options for the loader
|
||||||
|
* @returns A promise that resolves with the result of loading the augmentations
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* // webpack.config.js
|
||||||
|
* const { AugmentationRegistryPlugin } = require('brainy/dist/webpack');
|
||||||
|
*
|
||||||
|
* module.exports = {
|
||||||
|
* // ... other webpack config
|
||||||
|
* plugins: [
|
||||||
|
* new AugmentationRegistryPlugin({
|
||||||
|
* // Pattern to match files containing augmentations
|
||||||
|
* pattern: /augmentation\.js$/,
|
||||||
|
* // Options for the loader
|
||||||
|
* options: {
|
||||||
|
* autoInitialize: true,
|
||||||
|
* debug: true
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
* ]
|
||||||
|
* };
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
export async function loadAugmentationsFromModules(modules, options = {}) {
|
||||||
|
const opts = { ...DEFAULT_OPTIONS, ...options };
|
||||||
|
const result = {
|
||||||
|
augmentations: [],
|
||||||
|
errors: []
|
||||||
|
};
|
||||||
|
if (opts.debug) {
|
||||||
|
console.log(`[AugmentationRegistryLoader] Loading augmentations from ${Object.keys(modules).length} modules`);
|
||||||
|
}
|
||||||
|
// Process each module
|
||||||
|
for (const [modulePath, module] of Object.entries(modules)) {
|
||||||
|
try {
|
||||||
|
if (opts.debug) {
|
||||||
|
console.log(`[AugmentationRegistryLoader] Processing module: ${modulePath}`);
|
||||||
|
}
|
||||||
|
// Extract augmentations from the module
|
||||||
|
const augmentations = extractAugmentationsFromModule(module);
|
||||||
|
if (augmentations.length === 0) {
|
||||||
|
if (opts.debug) {
|
||||||
|
console.log(`[AugmentationRegistryLoader] No augmentations found in module: ${modulePath}`);
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Register each augmentation
|
||||||
|
for (const augmentation of augmentations) {
|
||||||
|
try {
|
||||||
|
const registered = registerAugmentation(augmentation);
|
||||||
|
result.augmentations.push(registered);
|
||||||
|
if (opts.debug) {
|
||||||
|
console.log(`[AugmentationRegistryLoader] Registered augmentation: ${registered.name}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
const err = error instanceof Error ? error : new Error(String(error));
|
||||||
|
result.errors.push(err);
|
||||||
|
if (opts.debug) {
|
||||||
|
console.error(`[AugmentationRegistryLoader] Failed to register augmentation: ${err.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
const err = error instanceof Error ? error : new Error(String(error));
|
||||||
|
result.errors.push(err);
|
||||||
|
if (opts.debug) {
|
||||||
|
console.error(`[AugmentationRegistryLoader] Error processing module ${modulePath}: ${err.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (opts.debug) {
|
||||||
|
console.log(`[AugmentationRegistryLoader] Loaded ${result.augmentations.length} augmentations with ${result.errors.length} errors`);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Extracts augmentations from a module
|
||||||
|
*
|
||||||
|
* @param module The module to extract augmentations from
|
||||||
|
* @returns An array of augmentations found in the module
|
||||||
|
*/
|
||||||
|
function extractAugmentationsFromModule(module) {
|
||||||
|
const augmentations = [];
|
||||||
|
// If the module itself is an augmentation, add it
|
||||||
|
if (isAugmentation(module)) {
|
||||||
|
augmentations.push(module);
|
||||||
|
}
|
||||||
|
// Check for exported augmentations
|
||||||
|
if (module && typeof module === 'object') {
|
||||||
|
for (const key of Object.keys(module)) {
|
||||||
|
const exported = module[key];
|
||||||
|
// Skip non-objects and null
|
||||||
|
if (!exported || typeof exported !== 'object') {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// If the exported value is an augmentation, add it
|
||||||
|
if (isAugmentation(exported)) {
|
||||||
|
augmentations.push(exported);
|
||||||
|
}
|
||||||
|
// If the exported value is an array of augmentations, add them
|
||||||
|
if (Array.isArray(exported) && exported.every(isAugmentation)) {
|
||||||
|
augmentations.push(...exported);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return augmentations;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Checks if an object is an augmentation
|
||||||
|
*
|
||||||
|
* @param obj The object to check
|
||||||
|
* @returns True if the object is an augmentation
|
||||||
|
*/
|
||||||
|
function isAugmentation(obj) {
|
||||||
|
return (obj &&
|
||||||
|
typeof obj === 'object' &&
|
||||||
|
typeof obj.name === 'string' &&
|
||||||
|
typeof obj.initialize === 'function' &&
|
||||||
|
typeof obj.shutDown === 'function' &&
|
||||||
|
typeof obj.getStatus === 'function');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Creates a webpack plugin for automatically loading augmentations
|
||||||
|
*
|
||||||
|
* @param options Options for the plugin
|
||||||
|
* @returns A webpack plugin
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* // webpack.config.js
|
||||||
|
* const { createAugmentationRegistryPlugin } = require('brainy/dist/webpack');
|
||||||
|
*
|
||||||
|
* module.exports = {
|
||||||
|
* // ... other webpack config
|
||||||
|
* plugins: [
|
||||||
|
* createAugmentationRegistryPlugin({
|
||||||
|
* pattern: /augmentation\.js$/,
|
||||||
|
* options: {
|
||||||
|
* autoInitialize: true,
|
||||||
|
* debug: true
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
* ]
|
||||||
|
* };
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
export function createAugmentationRegistryPlugin(options) {
|
||||||
|
// This is just a placeholder - the actual implementation would depend on the build tool
|
||||||
|
return {
|
||||||
|
name: 'AugmentationRegistryPlugin',
|
||||||
|
pattern: options.pattern,
|
||||||
|
options: options.options || {}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Creates a rollup plugin for automatically loading augmentations
|
||||||
|
*
|
||||||
|
* @param options Options for the plugin
|
||||||
|
* @returns A rollup plugin
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* // rollup.config.js
|
||||||
|
* import { createAugmentationRegistryRollupPlugin } from 'brainy/dist/rollup';
|
||||||
|
*
|
||||||
|
* export default {
|
||||||
|
* // ... other rollup config
|
||||||
|
* plugins: [
|
||||||
|
* createAugmentationRegistryRollupPlugin({
|
||||||
|
* pattern: /augmentation\.js$/,
|
||||||
|
* options: {
|
||||||
|
* autoInitialize: true,
|
||||||
|
* debug: true
|
||||||
|
* }
|
||||||
|
* })
|
||||||
|
* ]
|
||||||
|
* };
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
export function createAugmentationRegistryRollupPlugin(options) {
|
||||||
|
// This is just a placeholder - the actual implementation would depend on the build tool
|
||||||
|
return {
|
||||||
|
name: 'augmentation-registry-rollup-plugin',
|
||||||
|
pattern: options.pattern,
|
||||||
|
options: options.options || {}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=augmentationRegistryLoader.js.map
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
{"version":3,"file":"augmentationRegistryLoader.js","sourceRoot":"","sources":["../src/augmentationRegistryLoader.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAGH,OAAO,EAAE,oBAAoB,EAAE,MAAM,2BAA2B,CAAA;AAmBhE;;GAEG;AACH,MAAM,eAAe,GAAsC;IACzD,cAAc,EAAE,KAAK;IACrB,KAAK,EAAE,KAAK;CACb,CAAA;AAiBD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA8BG;AACH,MAAM,CAAC,KAAK,UAAU,4BAA4B,CAChD,OAA4B,EAC5B,UAA6C,EAAE;IAE/C,MAAM,IAAI,GAAG,EAAE,GAAG,eAAe,EAAE,GAAG,OAAO,EAAE,CAAA;IAC/C,MAAM,MAAM,GAA2B;QACrC,aAAa,EAAE,EAAE;QACjB,MAAM,EAAE,EAAE;KACX,CAAA;IAED,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;QACf,OAAO,CAAC,GAAG,CAAC,2DAA2D,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,MAAM,UAAU,CAAC,CAAA;IAC/G,CAAC;IAED,sBAAsB;IACtB,KAAK,MAAM,CAAC,UAAU,EAAE,MAAM,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC;QAC3D,IAAI,CAAC;YACH,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;gBACf,OAAO,CAAC,GAAG,CAAC,mDAAmD,UAAU,EAAE,CAAC,CAAA;YAC9E,CAAC;YAED,wCAAwC;YACxC,MAAM,aAAa,GAAG,8BAA8B,CAAC,MAAM,CAAC,CAAA;YAE5D,IAAI,aAAa,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBAC/B,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;oBACf,OAAO,CAAC,GAAG,CAAC,kEAAkE,UAAU,EAAE,CAAC,CAAA;gBAC7F,CAAC;gBACD,SAAQ;YACV,CAAC;YAED,6BAA6B;YAC7B,KAAK,MAAM,YAAY,IAAI,aAAa,EAAE,CAAC;gBACzC,IAAI,CAAC;oBACH,MAAM,UAAU,GAAG,oBAAoB,CAAC,YAAY,CAAC,CAAA;oBACrD,MAAM,CAAC,aAAa,CAAC,IAAI,CAAC,UAAU,CAAC,CAAA;oBAErC,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;wBACf,OAAO,CAAC,GAAG,CAAC,yDAAyD,UAAU,CAAC,IAAI,EAAE,CAAC,CAAA;oBACzF,CAAC;gBACH,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBACf,MAAM,GAAG,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAA;oBACrE,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAA;oBAEvB,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;wBACf,OAAO,CAAC,KAAK,CAAC,iEAAiE,GAAG,CAAC,OAAO,EAAE,CAAC,CAAA;oBAC/F,CAAC;gBACH,CAAC;YACH,CAAC;QACH,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,GAAG,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAA;YACrE,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAA;YAEvB,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;gBACf,OAAO,CAAC,KAAK,CAAC,wDAAwD,UAAU,KAAK,GAAG,CAAC,OAAO,EAAE,CAAC,CAAA;YACrG,CAAC;QACH,CAAC;IACH,CAAC;IAED,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;QACf,OAAO,CAAC,GAAG,CAAC,uCAAuC,MAAM,CAAC,aAAa,CAAC,MAAM,uBAAuB,MAAM,CAAC,MAAM,CAAC,MAAM,SAAS,CAAC,CAAA;IACrI,CAAC;IAED,OAAO,MAAM,CAAA;AACf,CAAC;AAED;;;;;GAKG;AACH,SAAS,8BAA8B,CAAC,MAAW;IACjD,MAAM,aAAa,GAAoB,EAAE,CAAA;IAEzC,kDAAkD;IAClD,IAAI,cAAc,CAAC,MAAM,CAAC,EAAE,CAAC;QAC3B,aAAa,CAAC,IAAI,CAAC,MAAM,CAAC,CAAA;IAC5B,CAAC;IAED,mCAAmC;IACnC,IAAI,MAAM,IAAI,OAAO,MAAM,KAAK,QAAQ,EAAE,CAAC;QACzC,KAAK,MAAM,GAAG,IAAI,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;YACtC,MAAM,QAAQ,GAAG,MAAM,CAAC,GAAG,CAAC,CAAA;YAE5B,4BAA4B;YAC5B,IAAI,CAAC,QAAQ,IAAI,OAAO,QAAQ,KAAK,QAAQ,EAAE,CAAC;gBAC9C,SAAQ;YACV,CAAC;YAED,mDAAmD;YACnD,IAAI,cAAc,CAAC,QAAQ,CAAC,EAAE,CAAC;gBAC7B,aAAa,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAA;YAC9B,CAAC;YAED,+DAA+D;YAC/D,IAAI,KAAK,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,QAAQ,CAAC,KAAK,CAAC,cAAc,CAAC,EAAE,CAAC;gBAC9D,aAAa,CAAC,IAAI,CAAC,GAAG,QAAQ,CAAC,CAAA;YACjC,CAAC;QACH,CAAC;IACH,CAAC;IAED,OAAO,aAAa,CAAA;AACtB,CAAC;AAED;;;;;GAKG;AACH,SAAS,cAAc,CAAC,GAAQ;IAC9B,OAAO,CACL,GAAG;QACH,OAAO,GAAG,KAAK,QAAQ;QACvB,OAAO,GAAG,CAAC,IAAI,KAAK,QAAQ;QAC5B,OAAO,GAAG,CAAC,UAAU,KAAK,UAAU;QACpC,OAAO,GAAG,CAAC,QAAQ,KAAK,UAAU;QAClC,OAAO,GAAG,CAAC,SAAS,KAAK,UAAU,CACpC,CAAA;AACH,CAAC;AAED;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AACH,MAAM,UAAU,gCAAgC,CAAC,OAUhD;IACC,wFAAwF;IACxF,OAAO;QACL,IAAI,EAAE,4BAA4B;QAClC,OAAO,EAAE,OAAO,CAAC,OAAO;QACxB,OAAO,EAAE,OAAO,CAAC,OAAO,IAAI,EAAE;KAC/B,CAAA;AACH,CAAC;AAED;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AACH,MAAM,UAAU,sCAAsC,CAAC,OAUtD;IACC,wFAAwF;IACxF,OAAO;QACL,IAAI,EAAE,qCAAqC;QAC3C,OAAO,EAAE,OAAO,CAAC,OAAO;QACxB,OAAO,EAAE,OAAO,CAAC,OAAO,IAAI,EAAE;KAC/B,CAAA;AACH,CAAC"}
|
||||||
94
.recovery-workspace/dist-backup-20250910-141917/augmentations/AugmentationMetadataContract.d.ts
vendored
Normal file
94
.recovery-workspace/dist-backup-20250910-141917/augmentations/AugmentationMetadataContract.d.ts
vendored
Normal file
|
|
@ -0,0 +1,94 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Metadata Contract System
|
||||||
|
*
|
||||||
|
* Prevents accidental metadata corruption while allowing intentional enrichment
|
||||||
|
* Each augmentation declares its metadata intentions upfront
|
||||||
|
*/
|
||||||
|
export interface AugmentationMetadataContract {
|
||||||
|
name: string;
|
||||||
|
version: string;
|
||||||
|
reads?: {
|
||||||
|
userFields?: string[];
|
||||||
|
internalFields?: string[];
|
||||||
|
augmentationFields?: string[];
|
||||||
|
};
|
||||||
|
writes?: {
|
||||||
|
userFields?: Array<{
|
||||||
|
field: string;
|
||||||
|
type: 'create' | 'update' | 'merge' | 'delete';
|
||||||
|
description: string;
|
||||||
|
example?: any;
|
||||||
|
}>;
|
||||||
|
augmentationFields?: Array<{
|
||||||
|
field: string;
|
||||||
|
description: string;
|
||||||
|
}>;
|
||||||
|
internalFields?: Array<{
|
||||||
|
field: string;
|
||||||
|
permission: 'granted' | 'requested';
|
||||||
|
reason: string;
|
||||||
|
}>;
|
||||||
|
};
|
||||||
|
conflictResolution?: {
|
||||||
|
strategy: 'error' | 'warn' | 'merge' | 'skip' | 'override';
|
||||||
|
priority?: number;
|
||||||
|
};
|
||||||
|
guarantees?: {
|
||||||
|
preservesExisting?: boolean;
|
||||||
|
reversible?: boolean;
|
||||||
|
idempotent?: boolean;
|
||||||
|
validatesTypes?: boolean;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Runtime metadata safety enforcer
|
||||||
|
*/
|
||||||
|
export declare class MetadataSafetyEnforcer {
|
||||||
|
private contracts;
|
||||||
|
private modifications;
|
||||||
|
/**
|
||||||
|
* Register an augmentation's contract
|
||||||
|
*/
|
||||||
|
registerContract(contract: AugmentationMetadataContract): void;
|
||||||
|
/**
|
||||||
|
* Check if an augmentation can modify a field
|
||||||
|
*/
|
||||||
|
canModifyField(augName: string, field: string, value: any): {
|
||||||
|
allowed: boolean;
|
||||||
|
reason?: string;
|
||||||
|
warnings?: string[];
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Create safe metadata proxy for an augmentation
|
||||||
|
*/
|
||||||
|
createSafeProxy(metadata: any, augName: string): any;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Example augmentation contracts
|
||||||
|
*/
|
||||||
|
export declare const EXAMPLE_CONTRACTS: Record<string, AugmentationMetadataContract>;
|
||||||
|
/**
|
||||||
|
* Augmentation base class with safety
|
||||||
|
*/
|
||||||
|
export declare abstract class SafeAugmentation {
|
||||||
|
protected enforcer: MetadataSafetyEnforcer;
|
||||||
|
protected contract: AugmentationMetadataContract;
|
||||||
|
constructor(contract: AugmentationMetadataContract);
|
||||||
|
/**
|
||||||
|
* Get safe metadata proxy
|
||||||
|
*/
|
||||||
|
protected getSafeMetadata(metadata: any): any;
|
||||||
|
/**
|
||||||
|
* Abstract method to implement augmentation logic
|
||||||
|
*/
|
||||||
|
abstract execute(metadata: any): Promise<any>;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Example: Category enricher implementation
|
||||||
|
*/
|
||||||
|
export declare class CategoryEnricherAugmentation extends SafeAugmentation {
|
||||||
|
constructor();
|
||||||
|
execute(metadata: any): Promise<any>;
|
||||||
|
private detectCategory;
|
||||||
|
private detectSubcategories;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,306 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Metadata Contract System
|
||||||
|
*
|
||||||
|
* Prevents accidental metadata corruption while allowing intentional enrichment
|
||||||
|
* Each augmentation declares its metadata intentions upfront
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* Runtime metadata safety enforcer
|
||||||
|
*/
|
||||||
|
export class MetadataSafetyEnforcer {
|
||||||
|
constructor() {
|
||||||
|
this.contracts = new Map();
|
||||||
|
this.modifications = new Map(); // field -> augmentations that modify it
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Register an augmentation's contract
|
||||||
|
*/
|
||||||
|
registerContract(contract) {
|
||||||
|
this.contracts.set(contract.name, contract);
|
||||||
|
// Track which augmentations modify which fields
|
||||||
|
if (contract.writes?.userFields) {
|
||||||
|
for (const fieldDef of contract.writes.userFields) {
|
||||||
|
if (!this.modifications.has(fieldDef.field)) {
|
||||||
|
this.modifications.set(fieldDef.field, new Set());
|
||||||
|
}
|
||||||
|
this.modifications.get(fieldDef.field).add(contract.name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if an augmentation can modify a field
|
||||||
|
*/
|
||||||
|
canModifyField(augName, field, value) {
|
||||||
|
const contract = this.contracts.get(augName);
|
||||||
|
if (!contract) {
|
||||||
|
return {
|
||||||
|
allowed: false,
|
||||||
|
reason: `Augmentation '${augName}' has no registered contract`
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// Check if field is in user namespace
|
||||||
|
if (!field.startsWith('_brainy.') && !field.startsWith('_augmentations.')) {
|
||||||
|
// It's a user field
|
||||||
|
const declaredField = contract.writes?.userFields?.find(f => f.field === field);
|
||||||
|
if (!declaredField) {
|
||||||
|
return {
|
||||||
|
allowed: false,
|
||||||
|
reason: `Augmentation '${augName}' did not declare intent to modify '${field}'`
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// Check for conflicts
|
||||||
|
const modifiers = this.modifications.get(field);
|
||||||
|
if (modifiers && modifiers.size > 1) {
|
||||||
|
const others = Array.from(modifiers).filter(a => a !== augName);
|
||||||
|
return {
|
||||||
|
allowed: true, // Still allowed but with warning
|
||||||
|
warnings: [`Field '${field}' is also modified by: ${others.join(', ')}`]
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return { allowed: true };
|
||||||
|
}
|
||||||
|
// Check internal fields
|
||||||
|
if (field.startsWith('_brainy.')) {
|
||||||
|
const internalField = contract.writes?.internalFields?.find(f => field === `_brainy.${f.field}`);
|
||||||
|
if (!internalField) {
|
||||||
|
return {
|
||||||
|
allowed: false,
|
||||||
|
reason: `Augmentation '${augName}' cannot modify internal field '${field}'`
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if (internalField.permission !== 'granted') {
|
||||||
|
return {
|
||||||
|
allowed: false,
|
||||||
|
reason: `Permission not granted for internal field '${field}'`
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return { allowed: true };
|
||||||
|
}
|
||||||
|
// Check augmentation namespace
|
||||||
|
if (field.startsWith('_augmentations.')) {
|
||||||
|
const parts = field.split('.');
|
||||||
|
const targetAug = parts[1];
|
||||||
|
// Can only modify own namespace
|
||||||
|
if (targetAug !== augName) {
|
||||||
|
return {
|
||||||
|
allowed: false,
|
||||||
|
reason: `Cannot modify another augmentation's namespace: ${targetAug}`
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return { allowed: true };
|
||||||
|
}
|
||||||
|
return { allowed: true };
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Create safe metadata proxy for an augmentation
|
||||||
|
*/
|
||||||
|
createSafeProxy(metadata, augName) {
|
||||||
|
const self = this;
|
||||||
|
return new Proxy(metadata, {
|
||||||
|
set(target, prop, value) {
|
||||||
|
const field = String(prop);
|
||||||
|
// Check permission
|
||||||
|
const permission = self.canModifyField(augName, field, value);
|
||||||
|
if (!permission.allowed) {
|
||||||
|
throw new Error(`[${augName}] ${permission.reason}`);
|
||||||
|
}
|
||||||
|
if (permission.warnings) {
|
||||||
|
console.warn(`[${augName}] Warning:`, ...permission.warnings);
|
||||||
|
}
|
||||||
|
// Track modification for audit
|
||||||
|
if (!target._audit) {
|
||||||
|
target._audit = [];
|
||||||
|
}
|
||||||
|
target._audit.push({
|
||||||
|
augmentation: augName,
|
||||||
|
field,
|
||||||
|
oldValue: target[prop],
|
||||||
|
newValue: value,
|
||||||
|
timestamp: Date.now()
|
||||||
|
});
|
||||||
|
target[prop] = value;
|
||||||
|
return true;
|
||||||
|
},
|
||||||
|
deleteProperty(target, prop) {
|
||||||
|
const field = String(prop);
|
||||||
|
const permission = self.canModifyField(augName, field, undefined);
|
||||||
|
if (!permission.allowed) {
|
||||||
|
throw new Error(`[${augName}] Cannot delete field: ${permission.reason}`);
|
||||||
|
}
|
||||||
|
delete target[prop];
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Example augmentation contracts
|
||||||
|
*/
|
||||||
|
export const EXAMPLE_CONTRACTS = {
|
||||||
|
// Enrichment augmentation that adds categories
|
||||||
|
categoryEnricher: {
|
||||||
|
name: 'categoryEnricher',
|
||||||
|
version: '1.0.0',
|
||||||
|
reads: {
|
||||||
|
userFields: ['title', 'description', 'content']
|
||||||
|
},
|
||||||
|
writes: {
|
||||||
|
userFields: [
|
||||||
|
{
|
||||||
|
field: 'category',
|
||||||
|
type: 'create',
|
||||||
|
description: 'Auto-detected category',
|
||||||
|
example: 'technology'
|
||||||
|
},
|
||||||
|
{
|
||||||
|
field: 'subcategories',
|
||||||
|
type: 'create',
|
||||||
|
description: 'List of relevant subcategories',
|
||||||
|
example: ['web', 'framework']
|
||||||
|
}
|
||||||
|
],
|
||||||
|
augmentationFields: [
|
||||||
|
{
|
||||||
|
field: 'confidence',
|
||||||
|
description: 'Confidence score of categorization'
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
guarantees: {
|
||||||
|
preservesExisting: true,
|
||||||
|
idempotent: true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
// Translation augmentation
|
||||||
|
translator: {
|
||||||
|
name: 'translator',
|
||||||
|
version: '1.0.0',
|
||||||
|
reads: {
|
||||||
|
userFields: ['title', 'description']
|
||||||
|
},
|
||||||
|
writes: {
|
||||||
|
userFields: [
|
||||||
|
{
|
||||||
|
field: 'translations',
|
||||||
|
type: 'merge',
|
||||||
|
description: 'Translations in multiple languages',
|
||||||
|
example: { es: 'Título', fr: 'Titre' }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
field: 'detectedLanguage',
|
||||||
|
type: 'create',
|
||||||
|
description: 'Detected source language',
|
||||||
|
example: 'en'
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
conflictResolution: {
|
||||||
|
strategy: 'merge',
|
||||||
|
priority: 10
|
||||||
|
}
|
||||||
|
},
|
||||||
|
// Sentiment analyzer
|
||||||
|
sentimentAnalyzer: {
|
||||||
|
name: 'sentimentAnalyzer',
|
||||||
|
version: '1.0.0',
|
||||||
|
reads: {
|
||||||
|
userFields: ['content', 'description', 'reviews']
|
||||||
|
},
|
||||||
|
writes: {
|
||||||
|
userFields: [
|
||||||
|
{
|
||||||
|
field: 'sentiment',
|
||||||
|
type: 'update',
|
||||||
|
description: 'Overall sentiment score',
|
||||||
|
example: { score: 0.8, label: 'positive' }
|
||||||
|
}
|
||||||
|
],
|
||||||
|
augmentationFields: [
|
||||||
|
{
|
||||||
|
field: 'analysis',
|
||||||
|
description: 'Detailed sentiment breakdown'
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
guarantees: {
|
||||||
|
reversible: true,
|
||||||
|
validatesTypes: true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
// System augmentation with internal access
|
||||||
|
garbageCollector: {
|
||||||
|
name: 'garbageCollector',
|
||||||
|
version: '1.0.0',
|
||||||
|
reads: {
|
||||||
|
internalFields: ['_brainy.createdAt', '_brainy.lastAccessed']
|
||||||
|
},
|
||||||
|
writes: {
|
||||||
|
internalFields: [
|
||||||
|
{
|
||||||
|
field: 'deleted',
|
||||||
|
permission: 'granted',
|
||||||
|
reason: 'Soft delete expired items'
|
||||||
|
},
|
||||||
|
{
|
||||||
|
field: 'archived',
|
||||||
|
permission: 'granted',
|
||||||
|
reason: 'Archive old items'
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Augmentation base class with safety
|
||||||
|
*/
|
||||||
|
export class SafeAugmentation {
|
||||||
|
constructor(contract) {
|
||||||
|
this.contract = contract;
|
||||||
|
this.enforcer = new MetadataSafetyEnforcer();
|
||||||
|
this.enforcer.registerContract(contract);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get safe metadata proxy
|
||||||
|
*/
|
||||||
|
getSafeMetadata(metadata) {
|
||||||
|
return this.enforcer.createSafeProxy(metadata, this.contract.name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Example: Category enricher implementation
|
||||||
|
*/
|
||||||
|
export class CategoryEnricherAugmentation extends SafeAugmentation {
|
||||||
|
constructor() {
|
||||||
|
super(EXAMPLE_CONTRACTS.categoryEnricher);
|
||||||
|
}
|
||||||
|
async execute(metadata) {
|
||||||
|
const safe = this.getSafeMetadata(metadata);
|
||||||
|
// Read declared fields
|
||||||
|
const title = safe.title;
|
||||||
|
const description = safe.description;
|
||||||
|
// Analyze and categorize
|
||||||
|
const category = this.detectCategory(title, description);
|
||||||
|
const subcategories = this.detectSubcategories(title, description);
|
||||||
|
// Write to declared fields (will be checked by proxy)
|
||||||
|
safe.category = category; // ✅ Allowed - declared in contract
|
||||||
|
safe.subcategories = subcategories; // ✅ Allowed
|
||||||
|
// Try to write undeclared field
|
||||||
|
// safe.randomField = 'test' // ❌ Would throw error!
|
||||||
|
// Write to our augmentation namespace
|
||||||
|
if (!safe._augmentations)
|
||||||
|
safe._augmentations = {};
|
||||||
|
if (!safe._augmentations.categoryEnricher) {
|
||||||
|
safe._augmentations.categoryEnricher = {};
|
||||||
|
}
|
||||||
|
safe._augmentations.categoryEnricher.confidence = 0.95; // ✅ Allowed
|
||||||
|
return safe;
|
||||||
|
}
|
||||||
|
detectCategory(title, description) {
|
||||||
|
// Simplified logic
|
||||||
|
return 'technology';
|
||||||
|
}
|
||||||
|
detectSubcategories(title, description) {
|
||||||
|
return ['web', 'framework'];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=AugmentationMetadataContract.js.map
|
||||||
File diff suppressed because one or more lines are too long
109
.recovery-workspace/dist-backup-20250910-141917/augmentations/apiServerAugmentation.d.ts
vendored
Normal file
109
.recovery-workspace/dist-backup-20250910-141917/augmentations/apiServerAugmentation.d.ts
vendored
Normal file
|
|
@ -0,0 +1,109 @@
|
||||||
|
/**
|
||||||
|
* API Server Augmentation - Universal API Exposure
|
||||||
|
*
|
||||||
|
* 🌐 Exposes Brainy through REST, WebSocket, and MCP
|
||||||
|
* 🔌 Works in Node.js, Deno, and Service Workers
|
||||||
|
* 🚀 Single augmentation for all API needs
|
||||||
|
*
|
||||||
|
* This unifies and replaces:
|
||||||
|
* - BrainyMCPBroadcast (Node-specific server)
|
||||||
|
* - WebSocketConduitAugmentation (client connections)
|
||||||
|
* - Future REST API implementations
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
export interface APIServerConfig {
|
||||||
|
enabled?: boolean;
|
||||||
|
port?: number;
|
||||||
|
mcpPort?: number;
|
||||||
|
wsPort?: number;
|
||||||
|
host?: string;
|
||||||
|
cors?: {
|
||||||
|
origin?: string | string[];
|
||||||
|
credentials?: boolean;
|
||||||
|
};
|
||||||
|
auth?: {
|
||||||
|
required?: boolean;
|
||||||
|
apiKeys?: string[];
|
||||||
|
bearerTokens?: string[];
|
||||||
|
};
|
||||||
|
rateLimit?: {
|
||||||
|
windowMs?: number;
|
||||||
|
max?: number;
|
||||||
|
};
|
||||||
|
ssl?: {
|
||||||
|
cert?: string;
|
||||||
|
key?: string;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Unified API Server Augmentation
|
||||||
|
* Exposes Brainy through multiple protocols
|
||||||
|
*/
|
||||||
|
export declare class APIServerAugmentation extends BaseAugmentation {
|
||||||
|
readonly name = "api-server";
|
||||||
|
readonly timing: "after";
|
||||||
|
readonly metadata: "readonly";
|
||||||
|
readonly operations: ("all")[];
|
||||||
|
readonly priority = 5;
|
||||||
|
protected config: APIServerConfig;
|
||||||
|
private mcpService?;
|
||||||
|
private httpServer?;
|
||||||
|
private wsServer?;
|
||||||
|
private clients;
|
||||||
|
private operationHistory;
|
||||||
|
private maxHistorySize;
|
||||||
|
constructor(config?: APIServerConfig);
|
||||||
|
protected onInitialize(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Start Node.js server with Express
|
||||||
|
*/
|
||||||
|
private startNodeServer;
|
||||||
|
/**
|
||||||
|
* Setup REST API routes
|
||||||
|
*/
|
||||||
|
private setupRESTRoutes;
|
||||||
|
/**
|
||||||
|
* Setup WebSocket server
|
||||||
|
*/
|
||||||
|
private setupWebSocketServer;
|
||||||
|
/**
|
||||||
|
* Handle WebSocket message
|
||||||
|
*/
|
||||||
|
private handleWebSocketMessage;
|
||||||
|
/**
|
||||||
|
* Execute augmentation - broadcast operations to clients
|
||||||
|
*/
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
/**
|
||||||
|
* Auth middleware for Express
|
||||||
|
*/
|
||||||
|
private authMiddleware;
|
||||||
|
/**
|
||||||
|
* Rate limiting middleware
|
||||||
|
*/
|
||||||
|
private rateLimitMiddleware;
|
||||||
|
/**
|
||||||
|
* Sanitize parameters before broadcasting
|
||||||
|
*/
|
||||||
|
private sanitizeParams;
|
||||||
|
/**
|
||||||
|
* Send heartbeats to all connected clients
|
||||||
|
*/
|
||||||
|
private sendHeartbeats;
|
||||||
|
/**
|
||||||
|
* Start Deno server
|
||||||
|
*/
|
||||||
|
private startDenoServer;
|
||||||
|
/**
|
||||||
|
* Start Service Worker (for browser)
|
||||||
|
*/
|
||||||
|
private startServiceWorker;
|
||||||
|
/**
|
||||||
|
* Shutdown the server
|
||||||
|
*/
|
||||||
|
protected onShutdown(): Promise<void>;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Helper function to create and configure API server
|
||||||
|
*/
|
||||||
|
export declare function createAPIServer(config?: APIServerConfig): APIServerAugmentation;
|
||||||
|
|
@ -0,0 +1,503 @@
|
||||||
|
/**
|
||||||
|
* API Server Augmentation - Universal API Exposure
|
||||||
|
*
|
||||||
|
* 🌐 Exposes Brainy through REST, WebSocket, and MCP
|
||||||
|
* 🔌 Works in Node.js, Deno, and Service Workers
|
||||||
|
* 🚀 Single augmentation for all API needs
|
||||||
|
*
|
||||||
|
* This unifies and replaces:
|
||||||
|
* - BrainyMCPBroadcast (Node-specific server)
|
||||||
|
* - WebSocketConduitAugmentation (client connections)
|
||||||
|
* - Future REST API implementations
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
import { BrainyMCPService } from '../mcp/brainyMCPService.js';
|
||||||
|
import { v4 as uuidv4 } from '../universal/uuid.js';
|
||||||
|
import { isNode, isBrowser } from '../utils/environment.js';
|
||||||
|
/**
|
||||||
|
* Unified API Server Augmentation
|
||||||
|
* Exposes Brainy through multiple protocols
|
||||||
|
*/
|
||||||
|
export class APIServerAugmentation extends BaseAugmentation {
|
||||||
|
constructor(config = {}) {
|
||||||
|
super();
|
||||||
|
this.name = 'api-server';
|
||||||
|
this.timing = 'after';
|
||||||
|
this.metadata = 'readonly'; // API server reads metadata to serve data
|
||||||
|
this.operations = ['all'];
|
||||||
|
this.priority = 5; // Low priority, runs after other augmentations
|
||||||
|
this.clients = new Map();
|
||||||
|
this.operationHistory = [];
|
||||||
|
this.maxHistorySize = 1000;
|
||||||
|
this.config = {
|
||||||
|
enabled: true,
|
||||||
|
port: 3000,
|
||||||
|
host: '0.0.0.0',
|
||||||
|
cors: { origin: '*', credentials: true },
|
||||||
|
auth: { required: false },
|
||||||
|
rateLimit: { windowMs: 60000, max: 100 },
|
||||||
|
...config
|
||||||
|
};
|
||||||
|
}
|
||||||
|
async onInitialize() {
|
||||||
|
if (!this.config.enabled) {
|
||||||
|
this.log('API Server disabled in config');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Initialize MCP service
|
||||||
|
this.mcpService = new BrainyMCPService(this.context.brain, {
|
||||||
|
enableAuth: this.config.auth?.required
|
||||||
|
});
|
||||||
|
// Start appropriate server based on environment
|
||||||
|
if (isNode()) {
|
||||||
|
await this.startNodeServer();
|
||||||
|
}
|
||||||
|
else if (typeof globalThis.Deno !== 'undefined') {
|
||||||
|
await this.startDenoServer();
|
||||||
|
}
|
||||||
|
else if (isBrowser() && 'serviceWorker' in navigator) {
|
||||||
|
await this.startServiceWorker();
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
this.log('No suitable server environment detected', 'warn');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Start Node.js server with Express
|
||||||
|
*/
|
||||||
|
async startNodeServer() {
|
||||||
|
try {
|
||||||
|
// Dynamic imports for Node.js dependencies
|
||||||
|
const express = await import('express').catch(() => null);
|
||||||
|
const cors = await import('cors').catch(() => null);
|
||||||
|
const ws = await import('ws').catch(() => null);
|
||||||
|
const { createServer } = await import('http');
|
||||||
|
if (!express || !cors || !ws) {
|
||||||
|
this.log('Express, cors, or ws not available. Install with: npm install express cors ws', 'error');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const WebSocketServer = ws?.WebSocketServer || ws?.default?.WebSocketServer || ws?.Server;
|
||||||
|
const app = express.default();
|
||||||
|
// Middleware
|
||||||
|
app.use(cors.default(this.config.cors));
|
||||||
|
app.use((express.default || express).json({ limit: '50mb' }));
|
||||||
|
app.use(this.authMiddleware.bind(this));
|
||||||
|
app.use(this.rateLimitMiddleware.bind(this));
|
||||||
|
// REST API Routes
|
||||||
|
this.setupRESTRoutes(app);
|
||||||
|
// Create HTTP server
|
||||||
|
this.httpServer = createServer(app);
|
||||||
|
// WebSocket server
|
||||||
|
this.wsServer = new WebSocketServer({
|
||||||
|
server: this.httpServer,
|
||||||
|
path: '/ws'
|
||||||
|
});
|
||||||
|
this.setupWebSocketServer();
|
||||||
|
// Start listening
|
||||||
|
await new Promise((resolve, reject) => {
|
||||||
|
this.httpServer.listen(this.config.port, this.config.host, () => {
|
||||||
|
this.log(`🌐 API Server listening on http://${this.config.host}:${this.config.port}`);
|
||||||
|
this.log(`🔌 WebSocket: ws://${this.config.host}:${this.config.port}/ws`);
|
||||||
|
this.log(`🧠 MCP endpoint: http://${this.config.host}:${this.config.port}/api/mcp`);
|
||||||
|
resolve();
|
||||||
|
}).on('error', reject);
|
||||||
|
});
|
||||||
|
// Heartbeat interval
|
||||||
|
setInterval(() => this.sendHeartbeats(), 30000);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log(`Failed to start Node.js server: ${error}`, 'error');
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Setup REST API routes
|
||||||
|
*/
|
||||||
|
setupRESTRoutes(app) {
|
||||||
|
// Health check
|
||||||
|
app.get('/health', (_req, res) => {
|
||||||
|
res.json({
|
||||||
|
status: 'healthy',
|
||||||
|
version: '2.0.0',
|
||||||
|
clients: this.clients.size,
|
||||||
|
uptime: process.uptime ? process.uptime() : 0
|
||||||
|
});
|
||||||
|
});
|
||||||
|
// Search endpoint
|
||||||
|
app.post('/api/search', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { query, limit = 10, options = {} } = req.body;
|
||||||
|
const results = await this.context.brain.search(query, limit, options);
|
||||||
|
res.json({ success: true, results });
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Add data endpoint
|
||||||
|
app.post('/api/add', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { content, metadata } = req.body;
|
||||||
|
const id = await this.context.brain.addNoun(content, 'Content', metadata);
|
||||||
|
res.json({ success: true, id });
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Get by ID endpoint
|
||||||
|
app.get('/api/get/:id', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const data = await this.context.brain.get(req.params.id);
|
||||||
|
if (data) {
|
||||||
|
res.json({ success: true, data });
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
res.status(404).json({ success: false, error: 'Not found' });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Delete endpoint
|
||||||
|
app.delete('/api/delete/:id', async (req, res) => {
|
||||||
|
try {
|
||||||
|
await this.context.brain.delete(req.params.id);
|
||||||
|
res.json({ success: true });
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Relate endpoint
|
||||||
|
app.post('/api/relate', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { source, target, verb, metadata } = req.body;
|
||||||
|
await this.context.brain.relate(source, target, verb, metadata);
|
||||||
|
res.json({ success: true });
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Find endpoint (complex queries)
|
||||||
|
app.post('/api/find', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const results = await this.context.brain.find(req.body);
|
||||||
|
res.json({ success: true, results });
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Cluster endpoint
|
||||||
|
app.post('/api/cluster', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { algorithm = 'kmeans', options = {} } = req.body;
|
||||||
|
const clusters = await this.context.brain.cluster(algorithm, options);
|
||||||
|
res.json({ success: true, clusters });
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// MCP endpoint
|
||||||
|
app.post('/api/mcp', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const response = await this.mcpService.handleRequest(req.body);
|
||||||
|
res.json(response);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Statistics endpoint
|
||||||
|
app.get('/api/stats', async (_req, res) => {
|
||||||
|
try {
|
||||||
|
const stats = await this.context.brain.getStatistics();
|
||||||
|
res.json({ success: true, stats });
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Operation history endpoint
|
||||||
|
app.get('/api/history', (_req, res) => {
|
||||||
|
res.json({
|
||||||
|
success: true,
|
||||||
|
history: this.operationHistory.slice(-100)
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Setup WebSocket server
|
||||||
|
*/
|
||||||
|
setupWebSocketServer() {
|
||||||
|
if (!this.wsServer)
|
||||||
|
return;
|
||||||
|
this.wsServer.on('connection', (socket, request) => {
|
||||||
|
const clientId = uuidv4();
|
||||||
|
const client = {
|
||||||
|
id: clientId,
|
||||||
|
type: 'websocket',
|
||||||
|
socket,
|
||||||
|
subscriptions: [],
|
||||||
|
lastSeen: Date.now()
|
||||||
|
};
|
||||||
|
this.clients.set(clientId, client);
|
||||||
|
// Send welcome message
|
||||||
|
socket.send(JSON.stringify({
|
||||||
|
type: 'welcome',
|
||||||
|
clientId,
|
||||||
|
message: 'Connected to Brainy API Server',
|
||||||
|
capabilities: ['search', 'add', 'delete', 'relate', 'subscribe', 'mcp']
|
||||||
|
}));
|
||||||
|
// Handle messages
|
||||||
|
socket.on('message', async (message) => {
|
||||||
|
try {
|
||||||
|
const msg = JSON.parse(message);
|
||||||
|
await this.handleWebSocketMessage(msg, client);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
socket.send(JSON.stringify({
|
||||||
|
type: 'error',
|
||||||
|
error: error.message
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Handle disconnect
|
||||||
|
socket.on('close', () => {
|
||||||
|
this.clients.delete(clientId);
|
||||||
|
this.log(`Client ${clientId} disconnected`);
|
||||||
|
});
|
||||||
|
// Handle errors
|
||||||
|
socket.on('error', (error) => {
|
||||||
|
this.log(`WebSocket error for client ${clientId}: ${error}`, 'error');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Handle WebSocket message
|
||||||
|
*/
|
||||||
|
async handleWebSocketMessage(msg, client) {
|
||||||
|
const { socket } = client;
|
||||||
|
switch (msg.type) {
|
||||||
|
case 'subscribe':
|
||||||
|
// Subscribe to operation types
|
||||||
|
client.subscriptions = msg.operations || ['all'];
|
||||||
|
socket.send(JSON.stringify({
|
||||||
|
type: 'subscribed',
|
||||||
|
operations: client.subscriptions
|
||||||
|
}));
|
||||||
|
break;
|
||||||
|
case 'search':
|
||||||
|
const searchResults = await this.context.brain.search(msg.query, msg.limit || 10, msg.options || {});
|
||||||
|
socket.send(JSON.stringify({
|
||||||
|
type: 'searchResults',
|
||||||
|
requestId: msg.requestId,
|
||||||
|
results: searchResults
|
||||||
|
}));
|
||||||
|
break;
|
||||||
|
case 'add':
|
||||||
|
const id = await this.context.brain.addNoun(msg.content, 'Content', msg.metadata);
|
||||||
|
socket.send(JSON.stringify({
|
||||||
|
type: 'addResult',
|
||||||
|
requestId: msg.requestId,
|
||||||
|
id
|
||||||
|
}));
|
||||||
|
break;
|
||||||
|
case 'mcp':
|
||||||
|
const mcpResponse = await this.mcpService.handleRequest(msg.request);
|
||||||
|
socket.send(JSON.stringify({
|
||||||
|
type: 'mcpResponse',
|
||||||
|
requestId: msg.requestId,
|
||||||
|
response: mcpResponse
|
||||||
|
}));
|
||||||
|
break;
|
||||||
|
case 'heartbeat':
|
||||||
|
client.lastSeen = Date.now();
|
||||||
|
socket.send(JSON.stringify({
|
||||||
|
type: 'heartbeat',
|
||||||
|
timestamp: Date.now()
|
||||||
|
}));
|
||||||
|
break;
|
||||||
|
default:
|
||||||
|
socket.send(JSON.stringify({
|
||||||
|
type: 'error',
|
||||||
|
error: `Unknown message type: ${msg.type}`
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Execute augmentation - broadcast operations to clients
|
||||||
|
*/
|
||||||
|
async execute(operation, params, next) {
|
||||||
|
const startTime = Date.now();
|
||||||
|
const result = await next();
|
||||||
|
const duration = Date.now() - startTime;
|
||||||
|
// Record operation in history
|
||||||
|
const historyEntry = {
|
||||||
|
operation,
|
||||||
|
params: this.sanitizeParams(params),
|
||||||
|
timestamp: Date.now(),
|
||||||
|
duration
|
||||||
|
};
|
||||||
|
this.operationHistory.push(historyEntry);
|
||||||
|
if (this.operationHistory.length > this.maxHistorySize) {
|
||||||
|
this.operationHistory.shift();
|
||||||
|
}
|
||||||
|
// Broadcast to subscribed WebSocket clients
|
||||||
|
const message = JSON.stringify({
|
||||||
|
type: 'operation',
|
||||||
|
operation,
|
||||||
|
params: historyEntry.params,
|
||||||
|
timestamp: historyEntry.timestamp,
|
||||||
|
duration
|
||||||
|
});
|
||||||
|
for (const client of this.clients.values()) {
|
||||||
|
if (client.type === 'websocket' && client.socket) {
|
||||||
|
if (client.subscriptions?.includes('all') ||
|
||||||
|
client.subscriptions?.includes(operation)) {
|
||||||
|
try {
|
||||||
|
client.socket.send(message);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
// Client might be disconnected
|
||||||
|
this.clients.delete(client.id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Auth middleware for Express
|
||||||
|
*/
|
||||||
|
authMiddleware(req, res, next) {
|
||||||
|
if (!this.config.auth?.required) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
const apiKey = req.headers['x-api-key'];
|
||||||
|
const bearerToken = req.headers.authorization?.replace('Bearer ', '');
|
||||||
|
if (apiKey && this.config.auth.apiKeys?.includes(apiKey)) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
if (bearerToken && this.config.auth.bearerTokens?.includes(bearerToken)) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
res.status(401).json({ error: 'Unauthorized' });
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Rate limiting middleware
|
||||||
|
*/
|
||||||
|
rateLimitMiddleware(req, res, next) {
|
||||||
|
// Simple in-memory rate limiting
|
||||||
|
// In production, use redis or proper rate limiting library
|
||||||
|
const ip = req.ip || req.connection.remoteAddress;
|
||||||
|
const now = Date.now();
|
||||||
|
const windowMs = this.config.rateLimit?.windowMs || 60000;
|
||||||
|
const max = this.config.rateLimit?.max || 100;
|
||||||
|
// Clean old entries
|
||||||
|
for (const [key, client] of this.clients.entries()) {
|
||||||
|
if (now - client.lastSeen > windowMs) {
|
||||||
|
this.clients.delete(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// For now, just pass through
|
||||||
|
// Real implementation would track requests per IP
|
||||||
|
next();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Sanitize parameters before broadcasting
|
||||||
|
*/
|
||||||
|
sanitizeParams(params) {
|
||||||
|
if (!params)
|
||||||
|
return params;
|
||||||
|
const sanitized = { ...params };
|
||||||
|
// Remove sensitive fields
|
||||||
|
delete sanitized.password;
|
||||||
|
delete sanitized.apiKey;
|
||||||
|
delete sanitized.token;
|
||||||
|
delete sanitized.secret;
|
||||||
|
// Truncate large data
|
||||||
|
if (sanitized.content && sanitized.content.length > 1000) {
|
||||||
|
sanitized.content = sanitized.content.substring(0, 1000) + '...';
|
||||||
|
}
|
||||||
|
return sanitized;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Send heartbeats to all connected clients
|
||||||
|
*/
|
||||||
|
sendHeartbeats() {
|
||||||
|
const now = Date.now();
|
||||||
|
for (const [id, client] of this.clients.entries()) {
|
||||||
|
if (client.type === 'websocket' && client.socket) {
|
||||||
|
// Remove inactive clients
|
||||||
|
if (now - client.lastSeen > 60000) {
|
||||||
|
this.clients.delete(id);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Send heartbeat
|
||||||
|
try {
|
||||||
|
client.socket.send(JSON.stringify({
|
||||||
|
type: 'heartbeat',
|
||||||
|
timestamp: now
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
// Client disconnected
|
||||||
|
this.clients.delete(id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Start Deno server
|
||||||
|
*/
|
||||||
|
async startDenoServer() {
|
||||||
|
// Deno implementation would go here
|
||||||
|
// Using Deno.serve() or oak framework
|
||||||
|
this.log('Deno server not yet implemented', 'warn');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Start Service Worker (for browser)
|
||||||
|
*/
|
||||||
|
async startServiceWorker() {
|
||||||
|
// Service Worker implementation would go here
|
||||||
|
// Intercepts fetch() calls and handles them locally
|
||||||
|
this.log('Service Worker API not yet implemented', 'warn');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Shutdown the server
|
||||||
|
*/
|
||||||
|
async onShutdown() {
|
||||||
|
// Close all WebSocket connections
|
||||||
|
for (const client of this.clients.values()) {
|
||||||
|
if (client.socket) {
|
||||||
|
try {
|
||||||
|
client.socket.close();
|
||||||
|
}
|
||||||
|
catch { }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this.clients.clear();
|
||||||
|
// Close servers
|
||||||
|
if (this.wsServer) {
|
||||||
|
this.wsServer.close();
|
||||||
|
}
|
||||||
|
if (this.httpServer) {
|
||||||
|
await new Promise(resolve => {
|
||||||
|
this.httpServer.close(() => resolve());
|
||||||
|
});
|
||||||
|
}
|
||||||
|
this.log('API Server shut down');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Helper function to create and configure API server
|
||||||
|
*/
|
||||||
|
export function createAPIServer(config) {
|
||||||
|
return new APIServerAugmentation(config);
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=apiServerAugmentation.js.map
|
||||||
File diff suppressed because one or more lines are too long
109
.recovery-workspace/dist-backup-20250910-141917/augmentations/auditLogAugmentation.d.ts
vendored
Normal file
109
.recovery-workspace/dist-backup-20250910-141917/augmentations/auditLogAugmentation.d.ts
vendored
Normal file
|
|
@ -0,0 +1,109 @@
|
||||||
|
/**
|
||||||
|
* Audit Logging Augmentation
|
||||||
|
* Provides comprehensive audit trail for all Brainy operations
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
import { AugmentationManifest } from './manifest.js';
|
||||||
|
export interface AuditLogConfig {
|
||||||
|
enabled?: boolean;
|
||||||
|
logLevel?: 'minimal' | 'standard' | 'detailed';
|
||||||
|
includeData?: boolean;
|
||||||
|
includeMetadata?: boolean;
|
||||||
|
retention?: number;
|
||||||
|
storage?: 'memory' | 'file' | 'database';
|
||||||
|
filePath?: string;
|
||||||
|
maxMemoryLogs?: number;
|
||||||
|
}
|
||||||
|
export interface AuditLogEntry {
|
||||||
|
id: string;
|
||||||
|
timestamp: number;
|
||||||
|
operation: string;
|
||||||
|
params: any;
|
||||||
|
result?: any;
|
||||||
|
error?: any;
|
||||||
|
duration: number;
|
||||||
|
userId?: string;
|
||||||
|
sessionId?: string;
|
||||||
|
metadata?: Record<string, any>;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Audit Log Augmentation
|
||||||
|
*/
|
||||||
|
export declare class AuditLogAugmentation extends BaseAugmentation {
|
||||||
|
readonly name = "auditLogger";
|
||||||
|
readonly timing: "around";
|
||||||
|
readonly metadata: "readonly";
|
||||||
|
operations: any;
|
||||||
|
readonly priority = 90;
|
||||||
|
readonly category: "core";
|
||||||
|
readonly description = "Comprehensive audit logging for compliance and debugging";
|
||||||
|
private logs;
|
||||||
|
private sessionId;
|
||||||
|
constructor(config?: AuditLogConfig);
|
||||||
|
getManifest(): AugmentationManifest;
|
||||||
|
protected onInitialize(): Promise<void>;
|
||||||
|
protected onShutdown(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Execute augmentation - log operations
|
||||||
|
*/
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
/**
|
||||||
|
* Sanitize parameters to remove sensitive data
|
||||||
|
*/
|
||||||
|
private sanitizeParams;
|
||||||
|
/**
|
||||||
|
* Sanitize result data
|
||||||
|
*/
|
||||||
|
private sanitizeResult;
|
||||||
|
/**
|
||||||
|
* Sanitize error information
|
||||||
|
*/
|
||||||
|
private sanitizeError;
|
||||||
|
/**
|
||||||
|
* Write log entry
|
||||||
|
*/
|
||||||
|
private writeLog;
|
||||||
|
/**
|
||||||
|
* Flush logs to persistent storage
|
||||||
|
*/
|
||||||
|
private flushLogs;
|
||||||
|
/**
|
||||||
|
* Clean up old logs based on retention
|
||||||
|
*/
|
||||||
|
private cleanupOldLogs;
|
||||||
|
/**
|
||||||
|
* Generate unique ID
|
||||||
|
*/
|
||||||
|
private generateId;
|
||||||
|
/**
|
||||||
|
* Query audit logs
|
||||||
|
*/
|
||||||
|
queryLogs(filter?: {
|
||||||
|
operation?: string;
|
||||||
|
startTime?: number;
|
||||||
|
endTime?: number;
|
||||||
|
sessionId?: string;
|
||||||
|
hasError?: boolean;
|
||||||
|
}): AuditLogEntry[];
|
||||||
|
/**
|
||||||
|
* Get audit statistics
|
||||||
|
*/
|
||||||
|
getStats(): {
|
||||||
|
totalLogs: number;
|
||||||
|
operations: Record<string, number>;
|
||||||
|
averageDuration: number;
|
||||||
|
errorRate: number;
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Export logs for analysis
|
||||||
|
*/
|
||||||
|
exportLogs(): AuditLogEntry[];
|
||||||
|
/**
|
||||||
|
* Clear all logs
|
||||||
|
*/
|
||||||
|
clearLogs(): void;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Create audit log augmentation
|
||||||
|
*/
|
||||||
|
export declare function createAuditLogAugmentation(config?: AuditLogConfig): AuditLogAugmentation;
|
||||||
|
|
@ -0,0 +1,358 @@
|
||||||
|
/**
|
||||||
|
* Audit Logging Augmentation
|
||||||
|
* Provides comprehensive audit trail for all Brainy operations
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
import { createHash } from 'crypto';
|
||||||
|
/**
|
||||||
|
* Audit Log Augmentation
|
||||||
|
*/
|
||||||
|
export class AuditLogAugmentation extends BaseAugmentation {
|
||||||
|
constructor(config = {}) {
|
||||||
|
super(config);
|
||||||
|
this.name = 'auditLogger';
|
||||||
|
this.timing = 'around';
|
||||||
|
this.metadata = 'readonly'; // Read metadata for context
|
||||||
|
this.operations = ['all']; // Audit all operations
|
||||||
|
this.priority = 90; // Low priority, runs last
|
||||||
|
// Augmentation metadata
|
||||||
|
this.category = 'core';
|
||||||
|
this.description = 'Comprehensive audit logging for compliance and debugging';
|
||||||
|
this.logs = [];
|
||||||
|
// Merge with defaults
|
||||||
|
this.config = {
|
||||||
|
enabled: config.enabled ?? true,
|
||||||
|
logLevel: config.logLevel ?? 'standard',
|
||||||
|
includeData: config.includeData ?? false,
|
||||||
|
includeMetadata: config.includeMetadata ?? true,
|
||||||
|
retention: config.retention ?? 90, // 90 days default
|
||||||
|
storage: config.storage ?? 'memory',
|
||||||
|
filePath: config.filePath,
|
||||||
|
maxMemoryLogs: config.maxMemoryLogs ?? 10000
|
||||||
|
};
|
||||||
|
// Generate session ID
|
||||||
|
this.sessionId = this.generateId();
|
||||||
|
}
|
||||||
|
getManifest() {
|
||||||
|
return {
|
||||||
|
id: 'audit-logger',
|
||||||
|
name: 'Audit Logger',
|
||||||
|
version: '1.0.0',
|
||||||
|
description: 'Comprehensive audit trail for all operations',
|
||||||
|
longDescription: 'Records detailed audit logs of all Brainy operations for compliance, debugging, and analytics purposes.',
|
||||||
|
category: 'analytics',
|
||||||
|
configSchema: {
|
||||||
|
type: 'object',
|
||||||
|
properties: {
|
||||||
|
enabled: {
|
||||||
|
type: 'boolean',
|
||||||
|
default: true,
|
||||||
|
description: 'Enable audit logging'
|
||||||
|
},
|
||||||
|
logLevel: {
|
||||||
|
type: 'string',
|
||||||
|
enum: ['minimal', 'standard', 'detailed'],
|
||||||
|
default: 'standard',
|
||||||
|
description: 'Level of detail to log'
|
||||||
|
},
|
||||||
|
includeData: {
|
||||||
|
type: 'boolean',
|
||||||
|
default: false,
|
||||||
|
description: 'Include actual data in logs (privacy concern)'
|
||||||
|
},
|
||||||
|
includeMetadata: {
|
||||||
|
type: 'boolean',
|
||||||
|
default: true,
|
||||||
|
description: 'Include metadata in logs'
|
||||||
|
},
|
||||||
|
retention: {
|
||||||
|
type: 'number',
|
||||||
|
default: 90,
|
||||||
|
minimum: 1,
|
||||||
|
maximum: 365,
|
||||||
|
description: 'Days to retain logs'
|
||||||
|
},
|
||||||
|
storage: {
|
||||||
|
type: 'string',
|
||||||
|
enum: ['memory', 'file', 'database'],
|
||||||
|
default: 'memory',
|
||||||
|
description: 'Where to store audit logs'
|
||||||
|
},
|
||||||
|
maxMemoryLogs: {
|
||||||
|
type: 'number',
|
||||||
|
default: 10000,
|
||||||
|
description: 'Maximum logs to keep in memory'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
configDefaults: {
|
||||||
|
enabled: true,
|
||||||
|
logLevel: 'standard',
|
||||||
|
includeData: false,
|
||||||
|
includeMetadata: true,
|
||||||
|
retention: 90,
|
||||||
|
storage: 'memory',
|
||||||
|
maxMemoryLogs: 10000
|
||||||
|
},
|
||||||
|
minBrainyVersion: '3.0.0',
|
||||||
|
keywords: ['audit', 'logging', 'compliance', 'analytics'],
|
||||||
|
documentation: 'https://docs.brainy.dev/augmentations/audit-log',
|
||||||
|
status: 'stable',
|
||||||
|
performance: {
|
||||||
|
memoryUsage: 'medium',
|
||||||
|
cpuUsage: 'low',
|
||||||
|
networkUsage: 'none'
|
||||||
|
},
|
||||||
|
features: ['operation-logging', 'configurable-detail', 'retention-management'],
|
||||||
|
enhancedOperations: ['all'],
|
||||||
|
metrics: [
|
||||||
|
{
|
||||||
|
name: 'audit_logs_created',
|
||||||
|
type: 'counter',
|
||||||
|
description: 'Total audit logs created'
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'audit_log_size',
|
||||||
|
type: 'gauge',
|
||||||
|
description: 'Current audit log size'
|
||||||
|
}
|
||||||
|
]
|
||||||
|
};
|
||||||
|
}
|
||||||
|
async onInitialize() {
|
||||||
|
if (!this.config.enabled) {
|
||||||
|
this.log('Audit logger disabled by configuration');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.log(`Audit logger initialized (level: ${this.config.logLevel}, storage: ${this.config.storage})`);
|
||||||
|
// Start retention cleanup if using memory storage
|
||||||
|
if (this.config.storage === 'memory') {
|
||||||
|
setInterval(() => {
|
||||||
|
this.cleanupOldLogs();
|
||||||
|
}, 3600000); // Every hour
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async onShutdown() {
|
||||||
|
// Save any pending logs if using file storage
|
||||||
|
if (this.config.storage === 'file' && this.logs.length > 0) {
|
||||||
|
await this.flushLogs();
|
||||||
|
}
|
||||||
|
this.log('Audit logger shut down');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Execute augmentation - log operations
|
||||||
|
*/
|
||||||
|
async execute(operation, params, next) {
|
||||||
|
// If audit logging is disabled, just pass through
|
||||||
|
if (!this.config.enabled) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
const startTime = Date.now();
|
||||||
|
const logEntry = {
|
||||||
|
id: this.generateId(),
|
||||||
|
timestamp: startTime,
|
||||||
|
operation,
|
||||||
|
sessionId: this.sessionId
|
||||||
|
};
|
||||||
|
// Add params based on log level
|
||||||
|
if (this.config.logLevel !== 'minimal') {
|
||||||
|
logEntry.params = this.sanitizeParams(params);
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const result = await next();
|
||||||
|
// Log successful operation
|
||||||
|
logEntry.duration = Date.now() - startTime;
|
||||||
|
// Add result based on log level and config
|
||||||
|
if (this.config.logLevel === 'detailed' && this.config.includeData) {
|
||||||
|
logEntry.result = this.sanitizeResult(result);
|
||||||
|
}
|
||||||
|
await this.writeLog(logEntry);
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
// Log failed operation
|
||||||
|
logEntry.duration = Date.now() - startTime;
|
||||||
|
logEntry.error = this.sanitizeError(error);
|
||||||
|
await this.writeLog(logEntry);
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Sanitize parameters to remove sensitive data
|
||||||
|
*/
|
||||||
|
sanitizeParams(params) {
|
||||||
|
if (!params)
|
||||||
|
return params;
|
||||||
|
// Don't include actual data unless configured
|
||||||
|
if (!this.config.includeData && params.data) {
|
||||||
|
return {
|
||||||
|
...params,
|
||||||
|
data: '[REDACTED]'
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// Redact common sensitive fields
|
||||||
|
const sanitized = { ...params };
|
||||||
|
const sensitiveFields = ['password', 'token', 'apiKey', 'secret'];
|
||||||
|
for (const field of sensitiveFields) {
|
||||||
|
if (sanitized[field]) {
|
||||||
|
sanitized[field] = '[REDACTED]';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return sanitized;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Sanitize result data
|
||||||
|
*/
|
||||||
|
sanitizeResult(result) {
|
||||||
|
if (!result)
|
||||||
|
return result;
|
||||||
|
// For arrays, just log count
|
||||||
|
if (Array.isArray(result)) {
|
||||||
|
return { count: result.length, type: 'array' };
|
||||||
|
}
|
||||||
|
// For objects, remove sensitive fields
|
||||||
|
if (typeof result === 'object') {
|
||||||
|
const sanitized = {};
|
||||||
|
for (const key in result) {
|
||||||
|
if (!['password', 'token', 'apiKey', 'secret'].includes(key)) {
|
||||||
|
sanitized[key] = result[key];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return sanitized;
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Sanitize error information
|
||||||
|
*/
|
||||||
|
sanitizeError(error) {
|
||||||
|
if (!error)
|
||||||
|
return error;
|
||||||
|
return {
|
||||||
|
message: error.message || 'Unknown error',
|
||||||
|
code: error.code,
|
||||||
|
statusCode: error.statusCode,
|
||||||
|
stack: this.config.logLevel === 'detailed' ? error.stack : undefined
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Write log entry
|
||||||
|
*/
|
||||||
|
async writeLog(entry) {
|
||||||
|
switch (this.config.storage) {
|
||||||
|
case 'memory':
|
||||||
|
this.logs.push(entry);
|
||||||
|
// Enforce max memory logs
|
||||||
|
if (this.logs.length > this.config.maxMemoryLogs) {
|
||||||
|
this.logs.shift(); // Remove oldest
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
case 'file':
|
||||||
|
// In production, would write to file
|
||||||
|
// For now, just add to memory
|
||||||
|
this.logs.push(entry);
|
||||||
|
break;
|
||||||
|
case 'database':
|
||||||
|
// In production, would write to database
|
||||||
|
// For now, just add to memory
|
||||||
|
this.logs.push(entry);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Flush logs to persistent storage
|
||||||
|
*/
|
||||||
|
async flushLogs() {
|
||||||
|
// In production, would write to file/database
|
||||||
|
// For now, just clear old logs
|
||||||
|
if (this.logs.length > this.config.maxMemoryLogs) {
|
||||||
|
this.logs = this.logs.slice(-this.config.maxMemoryLogs);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clean up old logs based on retention
|
||||||
|
*/
|
||||||
|
cleanupOldLogs() {
|
||||||
|
const cutoffTime = Date.now() - (this.config.retention * 24 * 60 * 60 * 1000);
|
||||||
|
this.logs = this.logs.filter(log => log.timestamp >= cutoffTime);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Generate unique ID
|
||||||
|
*/
|
||||||
|
generateId() {
|
||||||
|
return createHash('sha256')
|
||||||
|
.update(`${Date.now()}-${Math.random()}`)
|
||||||
|
.digest('hex')
|
||||||
|
.substring(0, 16);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Query audit logs
|
||||||
|
*/
|
||||||
|
queryLogs(filter) {
|
||||||
|
let results = [...this.logs];
|
||||||
|
if (filter) {
|
||||||
|
if (filter.operation) {
|
||||||
|
results = results.filter(log => log.operation === filter.operation);
|
||||||
|
}
|
||||||
|
if (filter.startTime) {
|
||||||
|
results = results.filter(log => log.timestamp >= filter.startTime);
|
||||||
|
}
|
||||||
|
if (filter.endTime) {
|
||||||
|
results = results.filter(log => log.timestamp <= filter.endTime);
|
||||||
|
}
|
||||||
|
if (filter.sessionId) {
|
||||||
|
results = results.filter(log => log.sessionId === filter.sessionId);
|
||||||
|
}
|
||||||
|
if (filter.hasError !== undefined) {
|
||||||
|
results = results.filter(log => (log.error !== undefined) === filter.hasError);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return results;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get audit statistics
|
||||||
|
*/
|
||||||
|
getStats() {
|
||||||
|
const stats = {
|
||||||
|
totalLogs: this.logs.length,
|
||||||
|
operations: {},
|
||||||
|
averageDuration: 0,
|
||||||
|
errorRate: 0
|
||||||
|
};
|
||||||
|
let totalDuration = 0;
|
||||||
|
let errorCount = 0;
|
||||||
|
for (const log of this.logs) {
|
||||||
|
// Count by operation
|
||||||
|
stats.operations[log.operation] = (stats.operations[log.operation] || 0) + 1;
|
||||||
|
// Sum duration
|
||||||
|
totalDuration += log.duration;
|
||||||
|
// Count errors
|
||||||
|
if (log.error)
|
||||||
|
errorCount++;
|
||||||
|
}
|
||||||
|
if (this.logs.length > 0) {
|
||||||
|
stats.averageDuration = totalDuration / this.logs.length;
|
||||||
|
stats.errorRate = errorCount / this.logs.length;
|
||||||
|
}
|
||||||
|
return stats;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Export logs for analysis
|
||||||
|
*/
|
||||||
|
exportLogs() {
|
||||||
|
return [...this.logs];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear all logs
|
||||||
|
*/
|
||||||
|
clearLogs() {
|
||||||
|
this.logs = [];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Create audit log augmentation
|
||||||
|
*/
|
||||||
|
export function createAuditLogAugmentation(config) {
|
||||||
|
return new AuditLogAugmentation(config);
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=auditLogAugmentation.js.map
|
||||||
File diff suppressed because one or more lines are too long
97
.recovery-workspace/dist-backup-20250910-141917/augmentations/batchProcessingAugmentation.d.ts
vendored
Normal file
97
.recovery-workspace/dist-backup-20250910-141917/augmentations/batchProcessingAugmentation.d.ts
vendored
Normal file
|
|
@ -0,0 +1,97 @@
|
||||||
|
/**
|
||||||
|
* Batch Processing Augmentation
|
||||||
|
*
|
||||||
|
* Critical for enterprise-scale performance: 500,000+ operations/second
|
||||||
|
* Automatically batches operations for maximum throughput
|
||||||
|
* Handles streaming data, bulk imports, and high-frequency operations
|
||||||
|
*
|
||||||
|
* Performance Impact: 10-50x improvement for bulk operations
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
import { AugmentationManifest } from './manifest.js';
|
||||||
|
interface BatchConfig {
|
||||||
|
enabled?: boolean;
|
||||||
|
adaptiveMode?: boolean;
|
||||||
|
immediateThreshold?: number;
|
||||||
|
batchThreshold?: number;
|
||||||
|
maxBatchSize?: number;
|
||||||
|
maxWaitTime?: number;
|
||||||
|
adaptiveBatching?: boolean;
|
||||||
|
priorityLanes?: number;
|
||||||
|
memoryLimit?: number;
|
||||||
|
}
|
||||||
|
interface BatchMetrics {
|
||||||
|
totalOperations: number;
|
||||||
|
batchesProcessed: number;
|
||||||
|
averageBatchSize: number;
|
||||||
|
averageLatency: number;
|
||||||
|
throughputPerSecond: number;
|
||||||
|
memoryUsage: number;
|
||||||
|
adaptiveAdjustments: number;
|
||||||
|
}
|
||||||
|
export declare class BatchProcessingAugmentation extends BaseAugmentation {
|
||||||
|
readonly metadata: "readonly";
|
||||||
|
name: string;
|
||||||
|
timing: "around";
|
||||||
|
operations: ("add" | "addNoun" | "addVerb" | "saveNoun" | "saveVerb" | "storage")[];
|
||||||
|
priority: number;
|
||||||
|
protected config: Required<BatchConfig>;
|
||||||
|
private batches;
|
||||||
|
private flushTimers;
|
||||||
|
private metrics;
|
||||||
|
private currentMemoryUsage;
|
||||||
|
private performanceHistory;
|
||||||
|
constructor(config?: BatchConfig);
|
||||||
|
getManifest(): AugmentationManifest;
|
||||||
|
protected onInitialize(): Promise<void>;
|
||||||
|
shouldExecute(operation: string, params: any): boolean;
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
private shouldBatch;
|
||||||
|
/**
|
||||||
|
* SMART WORKFLOW DETECTION METHODS
|
||||||
|
* These methods detect critical patterns that must not be batched
|
||||||
|
*/
|
||||||
|
private isEntityRegistryWorkflow;
|
||||||
|
private isDependencyChainStart;
|
||||||
|
private isWriteOnlyMode;
|
||||||
|
private hasEntityRegistryMetadata;
|
||||||
|
private getOperationContext;
|
||||||
|
private getCurrentLoad;
|
||||||
|
private addToBatch;
|
||||||
|
private getOperationPriority;
|
||||||
|
private getBatchKey;
|
||||||
|
private getOperationType;
|
||||||
|
private estimateOperationSize;
|
||||||
|
private shouldFlushBatch;
|
||||||
|
private extractPriorityFromKey;
|
||||||
|
private setFlushTimer;
|
||||||
|
private getAdaptiveWaitTime;
|
||||||
|
private getPerformanceMultiplier;
|
||||||
|
private flushBatch;
|
||||||
|
private processBatch;
|
||||||
|
private processBatchByType;
|
||||||
|
private processBatchSave;
|
||||||
|
private processBatchUpdate;
|
||||||
|
private processBatchDelete;
|
||||||
|
private processIndividually;
|
||||||
|
private processWithConcurrency;
|
||||||
|
private flushOldestBatch;
|
||||||
|
private updateMetrics;
|
||||||
|
private adjustBatchSize;
|
||||||
|
private startMetricsCollection;
|
||||||
|
/**
|
||||||
|
* Get batch processing statistics
|
||||||
|
*/
|
||||||
|
getStats(): BatchMetrics & {
|
||||||
|
pendingBatches: number;
|
||||||
|
pendingOperations: number;
|
||||||
|
currentBatchSize: number;
|
||||||
|
memoryUtilization: string;
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Force flush all pending batches
|
||||||
|
*/
|
||||||
|
flushAll(): Promise<void>;
|
||||||
|
protected onShutdown(): Promise<void>;
|
||||||
|
}
|
||||||
|
export {};
|
||||||
|
|
@ -0,0 +1,669 @@
|
||||||
|
/**
|
||||||
|
* Batch Processing Augmentation
|
||||||
|
*
|
||||||
|
* Critical for enterprise-scale performance: 500,000+ operations/second
|
||||||
|
* Automatically batches operations for maximum throughput
|
||||||
|
* Handles streaming data, bulk imports, and high-frequency operations
|
||||||
|
*
|
||||||
|
* Performance Impact: 10-50x improvement for bulk operations
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
export class BatchProcessingAugmentation extends BaseAugmentation {
|
||||||
|
constructor(config) {
|
||||||
|
super(config);
|
||||||
|
this.metadata = 'readonly'; // Reads metadata for batching decisions
|
||||||
|
this.name = 'BatchProcessing';
|
||||||
|
this.timing = 'around';
|
||||||
|
this.operations = ['add', 'addNoun', 'addVerb', 'saveNoun', 'saveVerb', 'storage'];
|
||||||
|
this.priority = 80; // High priority for performance
|
||||||
|
this.config = {
|
||||||
|
enabled: true,
|
||||||
|
adaptiveMode: true,
|
||||||
|
immediateThreshold: 1,
|
||||||
|
batchThreshold: 5,
|
||||||
|
maxBatchSize: 100,
|
||||||
|
maxWaitTime: 1000,
|
||||||
|
adaptiveBatching: true,
|
||||||
|
priorityLanes: 2,
|
||||||
|
memoryLimit: 100 * 1024 * 1024 // 100MB
|
||||||
|
};
|
||||||
|
this.batches = new Map();
|
||||||
|
this.flushTimers = new Map();
|
||||||
|
this.metrics = {
|
||||||
|
totalOperations: 0,
|
||||||
|
batchesProcessed: 0,
|
||||||
|
averageBatchSize: 0,
|
||||||
|
averageLatency: 0,
|
||||||
|
throughputPerSecond: 0,
|
||||||
|
memoryUsage: 0,
|
||||||
|
adaptiveAdjustments: 0
|
||||||
|
};
|
||||||
|
this.currentMemoryUsage = 0;
|
||||||
|
this.performanceHistory = [];
|
||||||
|
}
|
||||||
|
getManifest() {
|
||||||
|
return {
|
||||||
|
id: 'batch-processing',
|
||||||
|
name: 'Batch Processing',
|
||||||
|
version: '2.0.0',
|
||||||
|
description: 'High-performance batching for bulk operations',
|
||||||
|
longDescription: 'Automatically batches operations for maximum throughput. Essential for enterprise-scale workloads, achieving 500,000+ operations/second. Provides 10-50x performance improvement for bulk operations.',
|
||||||
|
category: 'performance',
|
||||||
|
configSchema: {
|
||||||
|
type: 'object',
|
||||||
|
properties: {
|
||||||
|
enabled: {
|
||||||
|
type: 'boolean',
|
||||||
|
default: true,
|
||||||
|
description: 'Enable batch processing'
|
||||||
|
},
|
||||||
|
adaptiveMode: {
|
||||||
|
type: 'boolean',
|
||||||
|
default: true,
|
||||||
|
description: 'Automatically decide when to batch operations'
|
||||||
|
},
|
||||||
|
immediateThreshold: {
|
||||||
|
type: 'number',
|
||||||
|
default: 1,
|
||||||
|
minimum: 1,
|
||||||
|
maximum: 10,
|
||||||
|
description: 'Operations count below which to execute immediately'
|
||||||
|
},
|
||||||
|
batchThreshold: {
|
||||||
|
type: 'number',
|
||||||
|
default: 5,
|
||||||
|
minimum: 2,
|
||||||
|
maximum: 100,
|
||||||
|
description: 'Queue size at which to start batching'
|
||||||
|
},
|
||||||
|
maxBatchSize: {
|
||||||
|
type: 'number',
|
||||||
|
default: 1000,
|
||||||
|
minimum: 10,
|
||||||
|
maximum: 10000,
|
||||||
|
description: 'Maximum items per batch'
|
||||||
|
},
|
||||||
|
maxWaitTime: {
|
||||||
|
type: 'number',
|
||||||
|
default: 100,
|
||||||
|
minimum: 1,
|
||||||
|
maximum: 5000,
|
||||||
|
description: 'Maximum wait time before flushing batch (ms)'
|
||||||
|
},
|
||||||
|
adaptiveBatching: {
|
||||||
|
type: 'boolean',
|
||||||
|
default: true,
|
||||||
|
description: 'Dynamically adjust batch size based on performance'
|
||||||
|
},
|
||||||
|
priorityLanes: {
|
||||||
|
type: 'number',
|
||||||
|
default: 3,
|
||||||
|
minimum: 1,
|
||||||
|
maximum: 10,
|
||||||
|
description: 'Number of priority processing lanes'
|
||||||
|
},
|
||||||
|
memoryLimit: {
|
||||||
|
type: 'number',
|
||||||
|
default: 104857600, // 100MB
|
||||||
|
minimum: 10485760, // 10MB
|
||||||
|
maximum: 1073741824, // 1GB
|
||||||
|
description: 'Maximum memory for batching in bytes'
|
||||||
|
}
|
||||||
|
},
|
||||||
|
additionalProperties: false
|
||||||
|
},
|
||||||
|
configDefaults: {
|
||||||
|
enabled: true,
|
||||||
|
adaptiveMode: true,
|
||||||
|
immediateThreshold: 1,
|
||||||
|
batchThreshold: 5,
|
||||||
|
maxBatchSize: 1000,
|
||||||
|
maxWaitTime: 100,
|
||||||
|
adaptiveBatching: true,
|
||||||
|
priorityLanes: 3,
|
||||||
|
memoryLimit: 104857600
|
||||||
|
},
|
||||||
|
minBrainyVersion: '2.0.0',
|
||||||
|
keywords: ['batch', 'performance', 'bulk', 'streaming', 'throughput'],
|
||||||
|
documentation: 'https://docs.brainy.dev/augmentations/batch-processing',
|
||||||
|
status: 'stable',
|
||||||
|
performance: {
|
||||||
|
memoryUsage: 'high',
|
||||||
|
cpuUsage: 'medium',
|
||||||
|
networkUsage: 'none'
|
||||||
|
},
|
||||||
|
features: ['auto-batching', 'adaptive-sizing', 'priority-lanes', 'streaming-support'],
|
||||||
|
enhancedOperations: ['add', 'addNoun', 'addVerb', 'saveNoun', 'saveVerb'],
|
||||||
|
ui: {
|
||||||
|
icon: '📦',
|
||||||
|
color: '#9C27B0'
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
async onInitialize() {
|
||||||
|
if (this.config.enabled) {
|
||||||
|
this.startMetricsCollection();
|
||||||
|
this.log(`Batch processing initialized: ${this.config.maxBatchSize} batch size, ${this.config.maxWaitTime}ms max wait`);
|
||||||
|
if (this.config.adaptiveBatching) {
|
||||||
|
this.log('Adaptive batching enabled - will optimize batch size dynamically');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
this.log('Batch processing disabled');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
shouldExecute(operation, params) {
|
||||||
|
if (!this.config.enabled)
|
||||||
|
return false;
|
||||||
|
// Skip batching for single operations or already-batched operations
|
||||||
|
if (params?.batch === false || params?.streaming === false)
|
||||||
|
return false;
|
||||||
|
// Enable for high-volume operations
|
||||||
|
return operation.includes('add') ||
|
||||||
|
operation.includes('save') ||
|
||||||
|
operation.includes('storage');
|
||||||
|
}
|
||||||
|
async execute(operation, params, next) {
|
||||||
|
if (!this.shouldExecute(operation, params)) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
// Check if this should be batched based on system load
|
||||||
|
if (this.shouldBatch(operation, params)) {
|
||||||
|
return this.addToBatch(operation, params, next);
|
||||||
|
}
|
||||||
|
// Execute immediately for low-latency requirements
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
shouldBatch(operation, params) {
|
||||||
|
// ZERO-CONFIG INTELLIGENT ADAPTATION:
|
||||||
|
if (this.config.adaptiveMode) {
|
||||||
|
// CRITICAL WORKFLOW DETECTION: Never batch operations that break critical patterns
|
||||||
|
// 1. ENTITY REGISTRY PATTERN: Never batch when immediate lookup is expected
|
||||||
|
if (this.isEntityRegistryWorkflow(operation, params)) {
|
||||||
|
return false; // Must be immediate for registry lookups to work
|
||||||
|
}
|
||||||
|
// 2. DEPENDENCY CHAIN PATTERN: Never batch when next operation depends on this one
|
||||||
|
if (this.isDependencyChainStart(operation, params)) {
|
||||||
|
return false; // Must be immediate for noun → verb workflows
|
||||||
|
}
|
||||||
|
// Count pending operations in the current operation's batch (needed for write-only mode)
|
||||||
|
const batchKey = this.getBatchKey(operation, params);
|
||||||
|
const currentBatch = this.batches.get(batchKey) || [];
|
||||||
|
const pendingCount = currentBatch.length;
|
||||||
|
// 3. WRITE-ONLY MODE: Special handling for high-speed streaming
|
||||||
|
if (this.isWriteOnlyMode(params)) {
|
||||||
|
// In write-only mode, batch aggressively but ensure entity registry updates immediately
|
||||||
|
if (this.hasEntityRegistryMetadata(params)) {
|
||||||
|
return false; // Entity registry updates must be immediate even in write-only mode
|
||||||
|
}
|
||||||
|
return pendingCount >= 3; // Lower threshold for write-only mode batching
|
||||||
|
}
|
||||||
|
// Apply intelligent thresholds:
|
||||||
|
// 4. Single operations are immediate (responsive user experience)
|
||||||
|
if (pendingCount < this.config.immediateThreshold) {
|
||||||
|
return false; // Execute immediately
|
||||||
|
}
|
||||||
|
// 5. Start batching when multiple operations are queued
|
||||||
|
if (pendingCount >= this.config.batchThreshold) {
|
||||||
|
return true; // Batch for efficiency
|
||||||
|
}
|
||||||
|
// 6. For in-between cases, use smart heuristics
|
||||||
|
const currentLoad = this.getCurrentLoad();
|
||||||
|
if (currentLoad > 0.5)
|
||||||
|
return true; // Higher load = more batching
|
||||||
|
// 7. Batch operations that naturally benefit from grouping
|
||||||
|
if (operation.includes('save') || operation.includes('add')) {
|
||||||
|
return pendingCount > 1; // Batch if others are already waiting
|
||||||
|
}
|
||||||
|
return false; // Default to immediate for best responsiveness
|
||||||
|
}
|
||||||
|
// TRADITIONAL MODE: (for explicit configuration scenarios)
|
||||||
|
// Always batch if explicitly requested
|
||||||
|
if (params?.batch === true || params?.streaming === true)
|
||||||
|
return true;
|
||||||
|
// Batch based on current system load
|
||||||
|
const currentLoad = this.getCurrentLoad();
|
||||||
|
if (currentLoad > 0.7)
|
||||||
|
return true; // High load - batch everything
|
||||||
|
// Batch operations that benefit from grouping
|
||||||
|
return operation.includes('save') ||
|
||||||
|
operation.includes('add') ||
|
||||||
|
operation.includes('update');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* SMART WORKFLOW DETECTION METHODS
|
||||||
|
* These methods detect critical patterns that must not be batched
|
||||||
|
*/
|
||||||
|
isEntityRegistryWorkflow(operation, params) {
|
||||||
|
// Detect operations that will likely be followed by immediate entity registry lookups
|
||||||
|
if (operation === 'addNoun' || operation === 'add') {
|
||||||
|
// Check if metadata contains external identifiers (DID, handle, etc.)
|
||||||
|
const metadata = params?.metadata || params?.data || {};
|
||||||
|
return !!(metadata.did || // Bluesky DID
|
||||||
|
metadata.handle || // Social media handle
|
||||||
|
metadata.uri || // Resource URI
|
||||||
|
metadata.external_id || // External system ID
|
||||||
|
metadata.user_id || // User ID
|
||||||
|
metadata.profile_id || // Profile ID
|
||||||
|
metadata.account_id // Account ID
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
isDependencyChainStart(operation, params) {
|
||||||
|
// Detect operations that are likely to be followed by dependent operations
|
||||||
|
if (operation === 'addNoun' || operation === 'add') {
|
||||||
|
// In interactive workflows, noun creation is often followed by verb creation
|
||||||
|
// Use heuristics to detect this pattern
|
||||||
|
const context = this.getOperationContext();
|
||||||
|
// If we've seen recent addVerb operations, this noun might be for a relationship
|
||||||
|
if (context.recentVerbOperations > 0) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
// If this is part of a rapid sequence of operations, it might be a dependency chain
|
||||||
|
if (context.operationsInLastSecond > 3) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
isWriteOnlyMode(params) {
|
||||||
|
// Detect write-only mode from context or parameters
|
||||||
|
return !!(params?.writeOnlyMode ||
|
||||||
|
params?.streaming ||
|
||||||
|
params?.highThroughput ||
|
||||||
|
this.context?.brain?.writeOnly);
|
||||||
|
}
|
||||||
|
hasEntityRegistryMetadata(params) {
|
||||||
|
// Check if this operation has metadata that needs immediate entity registry updates
|
||||||
|
const metadata = params?.metadata || params?.data || {};
|
||||||
|
return !!(metadata.did ||
|
||||||
|
metadata.handle ||
|
||||||
|
metadata.uri ||
|
||||||
|
metadata.external_id ||
|
||||||
|
// Also check for auto-registration hints
|
||||||
|
params?.autoCreateMissingNouns ||
|
||||||
|
params?.entityRegistry);
|
||||||
|
}
|
||||||
|
getOperationContext() {
|
||||||
|
const now = Date.now();
|
||||||
|
const oneSecondAgo = now - 1000;
|
||||||
|
let recentVerbOperations = 0;
|
||||||
|
let operationsInLastSecond = 0;
|
||||||
|
// Analyze recent operations across all batches
|
||||||
|
for (const batch of this.batches.values()) {
|
||||||
|
for (const op of batch) {
|
||||||
|
if (op.timestamp > oneSecondAgo) {
|
||||||
|
operationsInLastSecond++;
|
||||||
|
if (op.operation.includes('Verb') || op.operation.includes('verb')) {
|
||||||
|
recentVerbOperations++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { recentVerbOperations, operationsInLastSecond };
|
||||||
|
}
|
||||||
|
getCurrentLoad() {
|
||||||
|
// Simple load calculation based on pending operations
|
||||||
|
let totalPending = 0;
|
||||||
|
for (const batch of this.batches.values()) {
|
||||||
|
totalPending += batch.length;
|
||||||
|
}
|
||||||
|
return Math.min(totalPending / 10000, 1.0); // Normalize to 0-1
|
||||||
|
}
|
||||||
|
async addToBatch(operation, params, executor) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const priority = this.getOperationPriority(operation, params);
|
||||||
|
const batchKey = this.getBatchKey(operation, priority);
|
||||||
|
const operationSize = this.estimateOperationSize(params);
|
||||||
|
// Check memory limit
|
||||||
|
if (this.currentMemoryUsage + operationSize > this.config.memoryLimit) {
|
||||||
|
// Memory limit reached - flush oldest batch
|
||||||
|
this.flushOldestBatch();
|
||||||
|
}
|
||||||
|
const batchedOp = {
|
||||||
|
id: `op_${Date.now()}_${Math.random()}`,
|
||||||
|
operation,
|
||||||
|
params,
|
||||||
|
executor, // Store the actual executor
|
||||||
|
resolver: resolve,
|
||||||
|
rejector: reject,
|
||||||
|
timestamp: Date.now(),
|
||||||
|
priority,
|
||||||
|
size: operationSize
|
||||||
|
};
|
||||||
|
// Add to appropriate batch
|
||||||
|
if (!this.batches.has(batchKey)) {
|
||||||
|
this.batches.set(batchKey, []);
|
||||||
|
}
|
||||||
|
const batch = this.batches.get(batchKey);
|
||||||
|
batch.push(batchedOp);
|
||||||
|
this.currentMemoryUsage += operationSize;
|
||||||
|
this.metrics.totalOperations++;
|
||||||
|
// Check if batch should be flushed immediately
|
||||||
|
if (this.shouldFlushBatch(batch, batchKey)) {
|
||||||
|
this.flushBatch(batchKey);
|
||||||
|
}
|
||||||
|
else if (!this.flushTimers.has(batchKey)) {
|
||||||
|
// Set flush timer if not already set
|
||||||
|
this.setFlushTimer(batchKey);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
getOperationPriority(operation, params) {
|
||||||
|
// Explicit priority
|
||||||
|
if (params?.priority !== undefined)
|
||||||
|
return params.priority;
|
||||||
|
// Operation-based priority
|
||||||
|
if (operation.includes('delete'))
|
||||||
|
return 10; // Highest
|
||||||
|
if (operation.includes('update'))
|
||||||
|
return 8;
|
||||||
|
if (operation.includes('save'))
|
||||||
|
return 6;
|
||||||
|
if (operation.includes('add'))
|
||||||
|
return 4;
|
||||||
|
return 1; // Lowest
|
||||||
|
}
|
||||||
|
getBatchKey(operation, priority) {
|
||||||
|
// Group by operation type and priority for optimal batching
|
||||||
|
const opType = this.getOperationType(operation);
|
||||||
|
const priorityLane = Math.min(priority, this.config.priorityLanes - 1);
|
||||||
|
return `${opType}_p${priorityLane}`;
|
||||||
|
}
|
||||||
|
getOperationType(operation) {
|
||||||
|
if (operation.includes('add'))
|
||||||
|
return 'add';
|
||||||
|
if (operation.includes('save'))
|
||||||
|
return 'save';
|
||||||
|
if (operation.includes('update'))
|
||||||
|
return 'update';
|
||||||
|
if (operation.includes('delete'))
|
||||||
|
return 'delete';
|
||||||
|
return 'other';
|
||||||
|
}
|
||||||
|
estimateOperationSize(params) {
|
||||||
|
// Rough estimation of memory usage
|
||||||
|
if (!params)
|
||||||
|
return 100;
|
||||||
|
let size = 0;
|
||||||
|
if (params.vector && Array.isArray(params.vector)) {
|
||||||
|
size += params.vector.length * 8; // 8 bytes per float64
|
||||||
|
}
|
||||||
|
if (params.data) {
|
||||||
|
size += JSON.stringify(params.data).length * 2; // Rough UTF-16 estimate
|
||||||
|
}
|
||||||
|
if (params.metadata) {
|
||||||
|
size += JSON.stringify(params.metadata).length * 2;
|
||||||
|
}
|
||||||
|
return Math.max(size, 100); // Minimum 100 bytes
|
||||||
|
}
|
||||||
|
shouldFlushBatch(batch, batchKey) {
|
||||||
|
// Flush if batch is full
|
||||||
|
if (batch.length >= this.config.maxBatchSize)
|
||||||
|
return true;
|
||||||
|
// Flush if memory limit approaching
|
||||||
|
if (this.currentMemoryUsage > this.config.memoryLimit * 0.9)
|
||||||
|
return true;
|
||||||
|
// Flush high-priority batches more aggressively
|
||||||
|
const priority = this.extractPriorityFromKey(batchKey);
|
||||||
|
if (priority >= 8 && batch.length >= 100)
|
||||||
|
return true;
|
||||||
|
if (priority >= 6 && batch.length >= 500)
|
||||||
|
return true;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
extractPriorityFromKey(batchKey) {
|
||||||
|
const match = batchKey.match(/_p(\d+)$/);
|
||||||
|
return match ? parseInt(match[1]) : 0;
|
||||||
|
}
|
||||||
|
setFlushTimer(batchKey) {
|
||||||
|
const priority = this.extractPriorityFromKey(batchKey);
|
||||||
|
const waitTime = this.getAdaptiveWaitTime(priority);
|
||||||
|
const timer = setTimeout(() => {
|
||||||
|
this.flushBatch(batchKey);
|
||||||
|
}, waitTime);
|
||||||
|
this.flushTimers.set(batchKey, timer);
|
||||||
|
}
|
||||||
|
getAdaptiveWaitTime(priority) {
|
||||||
|
if (!this.config.adaptiveBatching) {
|
||||||
|
return this.config.maxWaitTime;
|
||||||
|
}
|
||||||
|
// Adaptive wait time based on performance and priority
|
||||||
|
const baseWaitTime = this.config.maxWaitTime;
|
||||||
|
const performanceMultiplier = this.getPerformanceMultiplier();
|
||||||
|
const priorityMultiplier = priority >= 8 ? 0.5 : priority >= 6 ? 0.7 : 1.0;
|
||||||
|
return Math.max(baseWaitTime * performanceMultiplier * priorityMultiplier, 10);
|
||||||
|
}
|
||||||
|
getPerformanceMultiplier() {
|
||||||
|
if (this.performanceHistory.length < 10)
|
||||||
|
return 1.0;
|
||||||
|
// Calculate average latency trend
|
||||||
|
const recent = this.performanceHistory.slice(-10);
|
||||||
|
const average = recent.reduce((a, b) => a + b, 0) / recent.length;
|
||||||
|
// If performance is degrading, reduce wait time
|
||||||
|
if (average > this.metrics.averageLatency * 1.2)
|
||||||
|
return 0.7;
|
||||||
|
if (average < this.metrics.averageLatency * 0.8)
|
||||||
|
return 1.3;
|
||||||
|
return 1.0;
|
||||||
|
}
|
||||||
|
async flushBatch(batchKey) {
|
||||||
|
const batch = this.batches.get(batchKey);
|
||||||
|
if (!batch || batch.length === 0)
|
||||||
|
return;
|
||||||
|
// Clear timer
|
||||||
|
const timer = this.flushTimers.get(batchKey);
|
||||||
|
if (timer) {
|
||||||
|
clearTimeout(timer);
|
||||||
|
this.flushTimers.delete(batchKey);
|
||||||
|
}
|
||||||
|
// Remove batch from queue
|
||||||
|
this.batches.delete(batchKey);
|
||||||
|
const startTime = Date.now();
|
||||||
|
try {
|
||||||
|
await this.processBatch(batch);
|
||||||
|
// Update metrics
|
||||||
|
const latency = Date.now() - startTime;
|
||||||
|
this.updateMetrics(batch.length, latency);
|
||||||
|
// Adaptive adjustment
|
||||||
|
if (this.config.adaptiveBatching) {
|
||||||
|
this.adjustBatchSize(latency, batch.length);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log(`Batch processing failed for ${batchKey}: ${error}`, 'error');
|
||||||
|
// Reject all operations in batch
|
||||||
|
batch.forEach(op => {
|
||||||
|
op.rejector(error);
|
||||||
|
this.currentMemoryUsage -= op.size;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async processBatch(batch) {
|
||||||
|
// Group by operation type for efficient processing
|
||||||
|
const operationGroups = new Map();
|
||||||
|
for (const op of batch) {
|
||||||
|
const opType = this.getOperationType(op.operation);
|
||||||
|
if (!operationGroups.has(opType)) {
|
||||||
|
operationGroups.set(opType, []);
|
||||||
|
}
|
||||||
|
operationGroups.get(opType).push(op);
|
||||||
|
}
|
||||||
|
// Process each operation type
|
||||||
|
for (const [opType, operations] of operationGroups) {
|
||||||
|
await this.processBatchByType(opType, operations);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async processBatchByType(opType, operations) {
|
||||||
|
// Execute batch operation based on type
|
||||||
|
try {
|
||||||
|
if (opType === 'add' || opType === 'save') {
|
||||||
|
await this.processBatchSave(operations);
|
||||||
|
}
|
||||||
|
else if (opType === 'update') {
|
||||||
|
await this.processBatchUpdate(operations);
|
||||||
|
}
|
||||||
|
else if (opType === 'delete') {
|
||||||
|
await this.processBatchDelete(operations);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
// Fallback: execute individually
|
||||||
|
await this.processIndividually(operations);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async processBatchSave(operations) {
|
||||||
|
// Try to use storage's bulk save if available
|
||||||
|
const storage = this.context?.storage;
|
||||||
|
if (storage && typeof storage.saveBatch === 'function') {
|
||||||
|
// Use bulk save operation
|
||||||
|
const items = operations.map(op => ({
|
||||||
|
...op.params,
|
||||||
|
_batchId: op.id
|
||||||
|
}));
|
||||||
|
try {
|
||||||
|
const results = await storage.saveBatch(items);
|
||||||
|
// Resolve all operations with actual results
|
||||||
|
operations.forEach((op, index) => {
|
||||||
|
op.resolver(results[index] || op.params.id);
|
||||||
|
this.currentMemoryUsage -= op.size;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
// Reject all operations on batch error
|
||||||
|
operations.forEach(op => {
|
||||||
|
op.rejector(error);
|
||||||
|
this.currentMemoryUsage -= op.size;
|
||||||
|
});
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
// Execute using stored executors with concurrency control
|
||||||
|
await this.processWithConcurrency(operations, 10);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async processBatchUpdate(operations) {
|
||||||
|
await this.processWithConcurrency(operations, 5); // Lower concurrency for updates
|
||||||
|
}
|
||||||
|
async processBatchDelete(operations) {
|
||||||
|
await this.processWithConcurrency(operations, 5); // Lower concurrency for deletes
|
||||||
|
}
|
||||||
|
async processIndividually(operations) {
|
||||||
|
await this.processWithConcurrency(operations, 3); // Conservative concurrency
|
||||||
|
}
|
||||||
|
async processWithConcurrency(operations, concurrency) {
|
||||||
|
const promises = [];
|
||||||
|
for (let i = 0; i < operations.length; i += concurrency) {
|
||||||
|
const chunk = operations.slice(i, i + concurrency);
|
||||||
|
const chunkPromise = Promise.all(chunk.map(async (op) => {
|
||||||
|
try {
|
||||||
|
// Execute using the stored executor function - REAL EXECUTION!
|
||||||
|
const result = await op.executor();
|
||||||
|
op.resolver(result);
|
||||||
|
this.currentMemoryUsage -= op.size;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
op.rejector(error);
|
||||||
|
this.currentMemoryUsage -= op.size;
|
||||||
|
}
|
||||||
|
})).then(() => { }); // Convert to void promise
|
||||||
|
promises.push(chunkPromise);
|
||||||
|
}
|
||||||
|
await Promise.all(promises);
|
||||||
|
}
|
||||||
|
// REMOVED executeOperation - no longer needed since we use stored executors
|
||||||
|
flushOldestBatch() {
|
||||||
|
if (this.batches.size === 0)
|
||||||
|
return;
|
||||||
|
// Find oldest batch
|
||||||
|
let oldestKey = '';
|
||||||
|
let oldestTime = Infinity;
|
||||||
|
for (const [key, batch] of this.batches) {
|
||||||
|
if (batch.length > 0) {
|
||||||
|
const batchAge = Math.min(...batch.map(op => op.timestamp));
|
||||||
|
if (batchAge < oldestTime) {
|
||||||
|
oldestTime = batchAge;
|
||||||
|
oldestKey = key;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (oldestKey) {
|
||||||
|
this.flushBatch(oldestKey);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
updateMetrics(batchSize, latency) {
|
||||||
|
this.metrics.batchesProcessed++;
|
||||||
|
this.metrics.averageBatchSize =
|
||||||
|
(this.metrics.averageBatchSize * (this.metrics.batchesProcessed - 1) + batchSize) /
|
||||||
|
this.metrics.batchesProcessed;
|
||||||
|
// Update latency with exponential moving average
|
||||||
|
this.metrics.averageLatency = this.metrics.averageLatency * 0.9 + latency * 0.1;
|
||||||
|
// Add to performance history
|
||||||
|
this.performanceHistory.push(latency);
|
||||||
|
if (this.performanceHistory.length > 100) {
|
||||||
|
this.performanceHistory.shift();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adjustBatchSize(latency, batchSize) {
|
||||||
|
const targetLatency = this.config.maxWaitTime * 5; // Target: 5x wait time
|
||||||
|
if (latency > targetLatency && batchSize > 100) {
|
||||||
|
// Reduce batch size if latency too high
|
||||||
|
this.config.maxBatchSize = Math.max(this.config.maxBatchSize * 0.9, 100);
|
||||||
|
this.metrics.adaptiveAdjustments++;
|
||||||
|
}
|
||||||
|
else if (latency < targetLatency * 0.5 && batchSize === this.config.maxBatchSize) {
|
||||||
|
// Increase batch size if latency very low
|
||||||
|
this.config.maxBatchSize = Math.min(this.config.maxBatchSize * 1.1, 10000);
|
||||||
|
this.metrics.adaptiveAdjustments++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
startMetricsCollection() {
|
||||||
|
setInterval(() => {
|
||||||
|
// Calculate throughput
|
||||||
|
this.metrics.throughputPerSecond = this.metrics.totalOperations;
|
||||||
|
this.metrics.totalOperations = 0; // Reset for next measurement
|
||||||
|
// Update memory usage
|
||||||
|
this.metrics.memoryUsage = this.currentMemoryUsage;
|
||||||
|
}, 1000);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get batch processing statistics
|
||||||
|
*/
|
||||||
|
getStats() {
|
||||||
|
let pendingOperations = 0;
|
||||||
|
for (const batch of this.batches.values()) {
|
||||||
|
pendingOperations += batch.length;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
...this.metrics,
|
||||||
|
pendingBatches: this.batches.size,
|
||||||
|
pendingOperations,
|
||||||
|
currentBatchSize: this.config.maxBatchSize,
|
||||||
|
memoryUtilization: `${Math.round((this.currentMemoryUsage / this.config.memoryLimit) * 100)}%`
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Force flush all pending batches
|
||||||
|
*/
|
||||||
|
async flushAll() {
|
||||||
|
const batchKeys = Array.from(this.batches.keys());
|
||||||
|
await Promise.all(batchKeys.map(key => this.flushBatch(key)));
|
||||||
|
}
|
||||||
|
async onShutdown() {
|
||||||
|
// Clear all timers
|
||||||
|
for (const timer of this.flushTimers.values()) {
|
||||||
|
clearTimeout(timer);
|
||||||
|
}
|
||||||
|
this.flushTimers.clear();
|
||||||
|
// Flush all pending batches
|
||||||
|
await this.flushAll();
|
||||||
|
const stats = this.getStats();
|
||||||
|
this.log(`Batch processing shutdown: ${this.metrics.batchesProcessed} batches processed, ${stats.memoryUtilization} peak memory usage`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=batchProcessingAugmentation.js.map
|
||||||
File diff suppressed because one or more lines are too long
304
.recovery-workspace/dist-backup-20250910-141917/augmentations/brainyAugmentation.d.ts
vendored
Normal file
304
.recovery-workspace/dist-backup-20250910-141917/augmentations/brainyAugmentation.d.ts
vendored
Normal file
|
|
@ -0,0 +1,304 @@
|
||||||
|
/**
|
||||||
|
* Single BrainyAugmentation Interface
|
||||||
|
*
|
||||||
|
* This replaces the 7 complex interfaces with one elegant, purpose-driven design.
|
||||||
|
* Each augmentation knows its place and when to execute automatically.
|
||||||
|
*
|
||||||
|
* The Vision: Components that enhance Brainy's capabilities seamlessly
|
||||||
|
* - WAL: Adds durability to storage operations
|
||||||
|
* - RequestDeduplicator: Prevents duplicate concurrent requests
|
||||||
|
* - ConnectionPool: Optimizes cloud storage throughput
|
||||||
|
* - IntelligentVerbScoring: Enhances relationship analysis
|
||||||
|
* - StreamingPipeline: Enables unlimited data processing
|
||||||
|
*/
|
||||||
|
import { AugmentationManifest } from './manifest.js';
|
||||||
|
/**
|
||||||
|
* Metadata access declaration for augmentations
|
||||||
|
*/
|
||||||
|
export interface MetadataAccess {
|
||||||
|
reads?: string[] | '*';
|
||||||
|
writes?: string[] | '*';
|
||||||
|
namespace?: string;
|
||||||
|
}
|
||||||
|
export interface BrainyAugmentation {
|
||||||
|
/**
|
||||||
|
* Unique identifier for the augmentation
|
||||||
|
*/
|
||||||
|
name: string;
|
||||||
|
/**
|
||||||
|
* When this augmentation should execute
|
||||||
|
* - 'before': Execute before the main operation
|
||||||
|
* - 'after': Execute after the main operation
|
||||||
|
* - 'around': Wrap the main operation (like middleware)
|
||||||
|
* - 'replace': Replace the main operation entirely
|
||||||
|
*/
|
||||||
|
timing: 'before' | 'after' | 'around' | 'replace';
|
||||||
|
/**
|
||||||
|
* Metadata access contract - REQUIRED
|
||||||
|
* - 'none': No metadata access at all
|
||||||
|
* - 'readonly': Can read any metadata but cannot write
|
||||||
|
* - MetadataAccess: Specific fields to read/write
|
||||||
|
*/
|
||||||
|
metadata: 'none' | 'readonly' | MetadataAccess;
|
||||||
|
/**
|
||||||
|
* Which operations this augmentation applies to
|
||||||
|
* Granular operation matching for precise augmentation targeting
|
||||||
|
*/
|
||||||
|
operations: ('add' | 'addNoun' | 'addVerb' | 'saveNoun' | 'saveVerb' | 'updateMetadata' | 'update' | 'delete' | 'deleteVerb' | 'clear' | 'get' | 'search' | 'searchText' | 'searchByNounTypes' | 'find' | 'findSimilar' | 'searchWithCursor' | 'similar' | 'relate' | 'unrelate' | 'getConnections' | 'getRelations' | 'storage' | 'backup' | 'restore' | 'all')[];
|
||||||
|
/**
|
||||||
|
* Priority for execution order (higher numbers execute first)
|
||||||
|
* - 100: Critical system operations (WAL, ConnectionPool)
|
||||||
|
* - 50: Performance optimizations (RequestDeduplicator, Caching)
|
||||||
|
* - 10: Enhancement features (IntelligentVerbScoring)
|
||||||
|
* - 1: Optional features (Logging, Analytics)
|
||||||
|
*/
|
||||||
|
priority: number;
|
||||||
|
/**
|
||||||
|
* Initialize the augmentation
|
||||||
|
* Called once during Brainy initialization
|
||||||
|
*
|
||||||
|
* @param context - The Brainy instance and storage
|
||||||
|
*/
|
||||||
|
initialize(context: AugmentationContext): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Execute the augmentation
|
||||||
|
*
|
||||||
|
* @param operation - The operation being performed
|
||||||
|
* @param params - Parameters for the operation
|
||||||
|
* @param next - Function to call the next augmentation or main operation
|
||||||
|
* @returns Result of the operation
|
||||||
|
*/
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
/**
|
||||||
|
* Optional: Check if this augmentation should run for the given operation
|
||||||
|
* Return false to skip execution
|
||||||
|
*/
|
||||||
|
shouldExecute?(operation: string, params: any): boolean;
|
||||||
|
/**
|
||||||
|
* Optional: Cleanup when Brainy is destroyed
|
||||||
|
*/
|
||||||
|
shutdown?(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Optional: Computed fields this augmentation provides
|
||||||
|
* Used for discovery, TypeScript support, and API documentation
|
||||||
|
*/
|
||||||
|
computedFields?: {
|
||||||
|
[namespace: string]: {
|
||||||
|
[field: string]: {
|
||||||
|
type: 'string' | 'number' | 'boolean' | 'object' | 'array';
|
||||||
|
description: string;
|
||||||
|
confidence?: number;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Optional: Compute fields for a result entity
|
||||||
|
* Called when user accesses getDisplay(), getSchema(), etc.
|
||||||
|
*
|
||||||
|
* @param result - The result entity (VectorDocument, GraphVerb, etc.)
|
||||||
|
* @param namespace - The namespace being requested ('display', 'schema', etc.)
|
||||||
|
* @returns Computed fields for the namespace
|
||||||
|
*/
|
||||||
|
computeFields?(result: any, namespace: string): Promise<Record<string, any>> | Record<string, any>;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Context provided to augmentations
|
||||||
|
*/
|
||||||
|
export interface AugmentationContext {
|
||||||
|
/**
|
||||||
|
* The Brainy instance (for accessing methods and config)
|
||||||
|
*/
|
||||||
|
brain: any;
|
||||||
|
/**
|
||||||
|
* The storage adapter
|
||||||
|
*/
|
||||||
|
storage: any;
|
||||||
|
/**
|
||||||
|
* Configuration for this augmentation
|
||||||
|
*/
|
||||||
|
config: any;
|
||||||
|
/**
|
||||||
|
* Logging function
|
||||||
|
*/
|
||||||
|
log: (message: string, level?: 'info' | 'warn' | 'error') => void;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Base class for augmentations with common functionality
|
||||||
|
*
|
||||||
|
* This is the unified base class that combines the features of both
|
||||||
|
* BaseAugmentation and ConfigurableAugmentation. All augmentations
|
||||||
|
* should extend this class for consistent configuration support.
|
||||||
|
*/
|
||||||
|
export declare abstract class BaseAugmentation implements BrainyAugmentation {
|
||||||
|
abstract name: string;
|
||||||
|
abstract timing: 'before' | 'after' | 'around' | 'replace';
|
||||||
|
abstract metadata: 'none' | 'readonly' | MetadataAccess;
|
||||||
|
abstract operations: ('add' | 'addNoun' | 'addVerb' | 'saveNoun' | 'saveVerb' | 'updateMetadata' | 'update' | 'delete' | 'deleteVerb' | 'clear' | 'get' | 'search' | 'searchText' | 'searchByNounTypes' | 'find' | 'findSimilar' | 'searchWithCursor' | 'similar' | 'relate' | 'unrelate' | 'getConnections' | 'getRelations' | 'storage' | 'backup' | 'restore' | 'all')[];
|
||||||
|
abstract priority: number;
|
||||||
|
category: 'internal' | 'core' | 'premium' | 'community' | 'external';
|
||||||
|
description: string;
|
||||||
|
enabled: boolean;
|
||||||
|
protected context?: AugmentationContext;
|
||||||
|
protected isInitialized: boolean;
|
||||||
|
protected config: any;
|
||||||
|
private configResolver?;
|
||||||
|
/**
|
||||||
|
* Constructor with optional configuration
|
||||||
|
* @param config Optional configuration to override defaults
|
||||||
|
*/
|
||||||
|
constructor(config?: any);
|
||||||
|
/**
|
||||||
|
* Get the augmentation manifest for discovery
|
||||||
|
* Override this to enable configuration support
|
||||||
|
* CRITICAL: This enables tools to discover parameters and configuration
|
||||||
|
*/
|
||||||
|
getManifest?(): AugmentationManifest;
|
||||||
|
/**
|
||||||
|
* Get parameter schema for operations
|
||||||
|
* Enables tools to know what parameters each operation needs
|
||||||
|
*/
|
||||||
|
getParameterSchema?(operation: string): any;
|
||||||
|
/**
|
||||||
|
* Get operation descriptions
|
||||||
|
* Enables tools to show what each operation does
|
||||||
|
*/
|
||||||
|
getOperationInfo?(): Record<string, {
|
||||||
|
description: string;
|
||||||
|
parameters?: any;
|
||||||
|
returns?: any;
|
||||||
|
examples?: any[];
|
||||||
|
}>;
|
||||||
|
/**
|
||||||
|
* Get current configuration
|
||||||
|
*/
|
||||||
|
getConfig(): any;
|
||||||
|
/**
|
||||||
|
* Update configuration at runtime
|
||||||
|
* @param partial Partial configuration to merge
|
||||||
|
*/
|
||||||
|
updateConfig(partial: any): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Optional: Handle configuration changes
|
||||||
|
* Override this to react to runtime configuration updates
|
||||||
|
*/
|
||||||
|
protected onConfigChange?(newConfig: any, oldConfig: any): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Resolve configuration from all sources
|
||||||
|
* Priority: constructor > env > files > defaults
|
||||||
|
*/
|
||||||
|
private resolveConfiguration;
|
||||||
|
initialize(context: AugmentationContext): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Override this in subclasses for initialization logic
|
||||||
|
*/
|
||||||
|
protected onInitialize(): Promise<void>;
|
||||||
|
abstract execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
shouldExecute(operation: string, params: any): boolean;
|
||||||
|
shutdown(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Override this in subclasses for cleanup logic
|
||||||
|
*/
|
||||||
|
protected onShutdown(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Optional computed fields declaration (override in subclasses)
|
||||||
|
*/
|
||||||
|
computedFields?: {
|
||||||
|
[namespace: string]: {
|
||||||
|
[field: string]: {
|
||||||
|
type: 'string' | 'number' | 'boolean' | 'object' | 'array';
|
||||||
|
description: string;
|
||||||
|
confidence?: number;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Optional computed fields implementation (override in subclasses)
|
||||||
|
* @param result The result entity
|
||||||
|
* @param namespace The requested namespace
|
||||||
|
* @returns Computed fields for the namespace
|
||||||
|
*/
|
||||||
|
computeFields?(result: any, namespace: string): Promise<Record<string, any>> | Record<string, any>;
|
||||||
|
/**
|
||||||
|
* Log a message with the augmentation name
|
||||||
|
*/
|
||||||
|
protected log(message: string, level?: 'info' | 'warn' | 'error'): void;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Alias for backward compatibility
|
||||||
|
* ConfigurableAugmentation is now merged into BaseAugmentation
|
||||||
|
* @deprecated Use BaseAugmentation instead
|
||||||
|
*/
|
||||||
|
export declare const ConfigurableAugmentation: typeof BaseAugmentation;
|
||||||
|
/**
|
||||||
|
* Registry for managing augmentations
|
||||||
|
*/
|
||||||
|
export declare class AugmentationRegistry {
|
||||||
|
private augmentations;
|
||||||
|
private context?;
|
||||||
|
/**
|
||||||
|
* Register an augmentation
|
||||||
|
*/
|
||||||
|
register(augmentation: BrainyAugmentation): void;
|
||||||
|
/**
|
||||||
|
* Find augmentations by operation (before initialization)
|
||||||
|
* Used for two-phase initialization to find storage augmentations
|
||||||
|
*/
|
||||||
|
findByOperation(operation: string): BrainyAugmentation | null;
|
||||||
|
/**
|
||||||
|
* Initialize all augmentations
|
||||||
|
*/
|
||||||
|
initialize(context: AugmentationContext): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Initialize all augmentations (alias for consistency)
|
||||||
|
*/
|
||||||
|
initializeAll(context: AugmentationContext): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Execute augmentations for an operation
|
||||||
|
*/
|
||||||
|
execute<T = any>(operation: string, params: any, mainOperation: () => Promise<T>): Promise<T>;
|
||||||
|
/**
|
||||||
|
* Get all registered augmentations
|
||||||
|
*/
|
||||||
|
getAll(): BrainyAugmentation[];
|
||||||
|
/**
|
||||||
|
* Get augmentation info for listing
|
||||||
|
*/
|
||||||
|
getInfo(): Array<{
|
||||||
|
name: string;
|
||||||
|
type: string;
|
||||||
|
enabled: boolean;
|
||||||
|
description: string;
|
||||||
|
category: string;
|
||||||
|
priority: number;
|
||||||
|
}>;
|
||||||
|
/**
|
||||||
|
* Get augmentations by name
|
||||||
|
*/
|
||||||
|
get(name: string): BrainyAugmentation | undefined;
|
||||||
|
/**
|
||||||
|
* Discover augmentation parameters and schemas
|
||||||
|
* Critical for tools like brain-cloud to generate UIs
|
||||||
|
*/
|
||||||
|
discover(name?: string): any;
|
||||||
|
/**
|
||||||
|
* Get configuration schema for an augmentation
|
||||||
|
* Enables UI generation for configuration
|
||||||
|
*/
|
||||||
|
getConfigSchema(name: string): any;
|
||||||
|
/**
|
||||||
|
* Configure an augmentation at runtime
|
||||||
|
*/
|
||||||
|
configure(name: string, config: any): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Get metrics for an augmentation
|
||||||
|
*/
|
||||||
|
metrics(name?: string): any;
|
||||||
|
/**
|
||||||
|
* Get health status
|
||||||
|
*/
|
||||||
|
health(): any;
|
||||||
|
/**
|
||||||
|
* Shutdown all augmentations
|
||||||
|
*/
|
||||||
|
shutdown(): Promise<void>;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,323 @@
|
||||||
|
/**
|
||||||
|
* Single BrainyAugmentation Interface
|
||||||
|
*
|
||||||
|
* This replaces the 7 complex interfaces with one elegant, purpose-driven design.
|
||||||
|
* Each augmentation knows its place and when to execute automatically.
|
||||||
|
*
|
||||||
|
* The Vision: Components that enhance Brainy's capabilities seamlessly
|
||||||
|
* - WAL: Adds durability to storage operations
|
||||||
|
* - RequestDeduplicator: Prevents duplicate concurrent requests
|
||||||
|
* - ConnectionPool: Optimizes cloud storage throughput
|
||||||
|
* - IntelligentVerbScoring: Enhances relationship analysis
|
||||||
|
* - StreamingPipeline: Enables unlimited data processing
|
||||||
|
*/
|
||||||
|
import { AugmentationConfigResolver } from './configResolver.js';
|
||||||
|
/**
|
||||||
|
* Base class for augmentations with common functionality
|
||||||
|
*
|
||||||
|
* This is the unified base class that combines the features of both
|
||||||
|
* BaseAugmentation and ConfigurableAugmentation. All augmentations
|
||||||
|
* should extend this class for consistent configuration support.
|
||||||
|
*/
|
||||||
|
export class BaseAugmentation {
|
||||||
|
/**
|
||||||
|
* Constructor with optional configuration
|
||||||
|
* @param config Optional configuration to override defaults
|
||||||
|
*/
|
||||||
|
constructor(config) {
|
||||||
|
// Metadata for augmentation listing and management
|
||||||
|
this.category = 'core';
|
||||||
|
this.description = '';
|
||||||
|
this.enabled = true;
|
||||||
|
this.isInitialized = false;
|
||||||
|
this.config = {};
|
||||||
|
// Only resolve configuration if getManifest is implemented
|
||||||
|
if (this.getManifest) {
|
||||||
|
this.config = this.resolveConfiguration(config);
|
||||||
|
}
|
||||||
|
else if (config) {
|
||||||
|
// Legacy support: direct config assignment for augmentations without manifests
|
||||||
|
this.config = config;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get current configuration
|
||||||
|
*/
|
||||||
|
getConfig() {
|
||||||
|
return { ...this.config };
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Update configuration at runtime
|
||||||
|
* @param partial Partial configuration to merge
|
||||||
|
*/
|
||||||
|
async updateConfig(partial) {
|
||||||
|
if (!this.configResolver) {
|
||||||
|
// For legacy augmentations without manifest, just merge config
|
||||||
|
const oldConfig = this.config;
|
||||||
|
this.config = { ...this.config, ...partial };
|
||||||
|
if (this.onConfigChange) {
|
||||||
|
await this.onConfigChange(this.config, oldConfig);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const oldConfig = this.config;
|
||||||
|
try {
|
||||||
|
// Use resolver to update and validate
|
||||||
|
this.config = this.configResolver.updateRuntime(partial);
|
||||||
|
// Call config change handler if implemented
|
||||||
|
if (this.onConfigChange) {
|
||||||
|
await this.onConfigChange(this.config, oldConfig);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
// Revert on error
|
||||||
|
this.config = oldConfig;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Resolve configuration from all sources
|
||||||
|
* Priority: constructor > env > files > defaults
|
||||||
|
*/
|
||||||
|
resolveConfiguration(constructorConfig) {
|
||||||
|
const manifest = this.getManifest();
|
||||||
|
// Create config resolver
|
||||||
|
this.configResolver = new AugmentationConfigResolver({
|
||||||
|
augmentationId: manifest.id,
|
||||||
|
schema: manifest.configSchema,
|
||||||
|
defaults: manifest.configDefaults
|
||||||
|
});
|
||||||
|
// Resolve configuration from all sources
|
||||||
|
return this.configResolver.resolve(constructorConfig);
|
||||||
|
}
|
||||||
|
async initialize(context) {
|
||||||
|
this.context = context;
|
||||||
|
this.isInitialized = true;
|
||||||
|
await this.onInitialize();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Override this in subclasses for initialization logic
|
||||||
|
*/
|
||||||
|
async onInitialize() {
|
||||||
|
// Default: no-op
|
||||||
|
}
|
||||||
|
shouldExecute(operation, params) {
|
||||||
|
// Default: execute if operations match exactly or includes 'all'
|
||||||
|
return this.operations.includes('all') ||
|
||||||
|
this.operations.includes(operation) ||
|
||||||
|
this.operations.some(op => operation.includes(op));
|
||||||
|
}
|
||||||
|
async shutdown() {
|
||||||
|
await this.onShutdown();
|
||||||
|
this.isInitialized = false;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Override this in subclasses for cleanup logic
|
||||||
|
*/
|
||||||
|
async onShutdown() {
|
||||||
|
// Default: no-op
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Log a message with the augmentation name
|
||||||
|
*/
|
||||||
|
log(message, level = 'info') {
|
||||||
|
if (this.context) {
|
||||||
|
this.context.log(`[${this.name}] ${message}`, level);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Alias for backward compatibility
|
||||||
|
* ConfigurableAugmentation is now merged into BaseAugmentation
|
||||||
|
* @deprecated Use BaseAugmentation instead
|
||||||
|
*/
|
||||||
|
export const ConfigurableAugmentation = BaseAugmentation;
|
||||||
|
/**
|
||||||
|
* Registry for managing augmentations
|
||||||
|
*/
|
||||||
|
export class AugmentationRegistry {
|
||||||
|
constructor() {
|
||||||
|
this.augmentations = [];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Register an augmentation
|
||||||
|
*/
|
||||||
|
register(augmentation) {
|
||||||
|
this.augmentations.push(augmentation);
|
||||||
|
// Sort by priority (highest first)
|
||||||
|
this.augmentations.sort((a, b) => b.priority - a.priority);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Find augmentations by operation (before initialization)
|
||||||
|
* Used for two-phase initialization to find storage augmentations
|
||||||
|
*/
|
||||||
|
findByOperation(operation) {
|
||||||
|
return this.augmentations.find(aug => aug.operations.includes(operation) ||
|
||||||
|
aug.operations.includes('all')) || null;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Initialize all augmentations
|
||||||
|
*/
|
||||||
|
async initialize(context) {
|
||||||
|
this.context = context;
|
||||||
|
for (const augmentation of this.augmentations) {
|
||||||
|
await augmentation.initialize(context);
|
||||||
|
}
|
||||||
|
context.log(`Initialized ${this.augmentations.length} augmentations`);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Initialize all augmentations (alias for consistency)
|
||||||
|
*/
|
||||||
|
async initializeAll(context) {
|
||||||
|
return this.initialize(context);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Execute augmentations for an operation
|
||||||
|
*/
|
||||||
|
async execute(operation, params, mainOperation) {
|
||||||
|
// Filter augmentations that should execute for this operation
|
||||||
|
const applicable = this.augmentations.filter(aug => aug.shouldExecute ? aug.shouldExecute(operation, params) :
|
||||||
|
aug.operations.includes('all') ||
|
||||||
|
aug.operations.includes(operation) ||
|
||||||
|
aug.operations.some(op => operation.includes(op)));
|
||||||
|
if (applicable.length === 0) {
|
||||||
|
// No augmentations, execute main operation directly
|
||||||
|
return mainOperation();
|
||||||
|
}
|
||||||
|
// Create a chain of augmentations
|
||||||
|
let index = 0;
|
||||||
|
const executeNext = async () => {
|
||||||
|
if (index >= applicable.length) {
|
||||||
|
// All augmentations processed, execute main operation
|
||||||
|
return mainOperation();
|
||||||
|
}
|
||||||
|
const augmentation = applicable[index++];
|
||||||
|
return augmentation.execute(operation, params, executeNext);
|
||||||
|
};
|
||||||
|
return executeNext();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get all registered augmentations
|
||||||
|
*/
|
||||||
|
getAll() {
|
||||||
|
return [...this.augmentations];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get augmentation info for listing
|
||||||
|
*/
|
||||||
|
getInfo() {
|
||||||
|
return this.augmentations.map(aug => {
|
||||||
|
const baseAug = aug;
|
||||||
|
return {
|
||||||
|
name: aug.name,
|
||||||
|
type: baseAug.category || 'core',
|
||||||
|
enabled: baseAug.enabled !== false,
|
||||||
|
description: baseAug.description || `${aug.name} augmentation`,
|
||||||
|
category: baseAug.category || 'core',
|
||||||
|
priority: aug.priority
|
||||||
|
};
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get augmentations by name
|
||||||
|
*/
|
||||||
|
get(name) {
|
||||||
|
return this.augmentations.find(aug => aug.name === name);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Discover augmentation parameters and schemas
|
||||||
|
* Critical for tools like brain-cloud to generate UIs
|
||||||
|
*/
|
||||||
|
discover(name) {
|
||||||
|
if (name) {
|
||||||
|
const aug = this.get(name);
|
||||||
|
if (!aug)
|
||||||
|
return null;
|
||||||
|
const baseAug = aug;
|
||||||
|
return {
|
||||||
|
name: aug.name,
|
||||||
|
operations: aug.operations,
|
||||||
|
priority: aug.priority,
|
||||||
|
timing: aug.timing,
|
||||||
|
metadata: aug.metadata,
|
||||||
|
manifest: baseAug.getManifest ? baseAug.getManifest() : undefined,
|
||||||
|
parameters: baseAug.getParameterSchema ?
|
||||||
|
aug.operations.reduce((acc, op) => {
|
||||||
|
acc[op] = baseAug.getParameterSchema(op);
|
||||||
|
return acc;
|
||||||
|
}, {}) : undefined,
|
||||||
|
operationInfo: baseAug.getOperationInfo ? baseAug.getOperationInfo() : undefined,
|
||||||
|
config: baseAug.getConfig ? baseAug.getConfig() : undefined
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// Return all augmentations discovery info
|
||||||
|
return this.augmentations.map(aug => this.discover(aug.name));
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get configuration schema for an augmentation
|
||||||
|
* Enables UI generation for configuration
|
||||||
|
*/
|
||||||
|
getConfigSchema(name) {
|
||||||
|
const aug = this.get(name);
|
||||||
|
if (!aug || !aug.getManifest)
|
||||||
|
return null;
|
||||||
|
const manifest = aug.getManifest();
|
||||||
|
return manifest?.configSchema;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Configure an augmentation at runtime
|
||||||
|
*/
|
||||||
|
async configure(name, config) {
|
||||||
|
const aug = this.get(name);
|
||||||
|
if (!aug || !aug.updateConfig) {
|
||||||
|
throw new Error(`Augmentation ${name} does not support configuration`);
|
||||||
|
}
|
||||||
|
await aug.updateConfig(config);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get metrics for an augmentation
|
||||||
|
*/
|
||||||
|
metrics(name) {
|
||||||
|
if (name) {
|
||||||
|
const aug = this.get(name);
|
||||||
|
if (!aug || !aug.metrics)
|
||||||
|
return null;
|
||||||
|
return aug.metrics();
|
||||||
|
}
|
||||||
|
// Return all metrics
|
||||||
|
const allMetrics = {};
|
||||||
|
for (const aug of this.augmentations) {
|
||||||
|
const a = aug;
|
||||||
|
if (a.metrics) {
|
||||||
|
allMetrics[aug.name] = a.metrics();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return allMetrics;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get health status
|
||||||
|
*/
|
||||||
|
health() {
|
||||||
|
const health = {
|
||||||
|
overall: 'healthy',
|
||||||
|
augmentations: {}
|
||||||
|
};
|
||||||
|
for (const aug of this.augmentations) {
|
||||||
|
const a = aug;
|
||||||
|
health.augmentations[aug.name] = a.health ? a.health() : 'unknown';
|
||||||
|
}
|
||||||
|
return health;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Shutdown all augmentations
|
||||||
|
*/
|
||||||
|
async shutdown() {
|
||||||
|
for (const augmentation of this.augmentations) {
|
||||||
|
if (augmentation.shutdown) {
|
||||||
|
await augmentation.shutdown();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this.augmentations = [];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=brainyAugmentation.js.map
|
||||||
File diff suppressed because one or more lines are too long
109
.recovery-workspace/dist-backup-20250910-141917/augmentations/cacheAugmentation.d.ts
vendored
Normal file
109
.recovery-workspace/dist-backup-20250910-141917/augmentations/cacheAugmentation.d.ts
vendored
Normal file
|
|
@ -0,0 +1,109 @@
|
||||||
|
/**
|
||||||
|
* Cache Augmentation - Optional Search Result Caching
|
||||||
|
*
|
||||||
|
* Replaces the hardcoded SearchCache in Brainy with an optional augmentation.
|
||||||
|
* This reduces core size and allows custom cache implementations.
|
||||||
|
*
|
||||||
|
* Zero-config: Automatically enabled with sensible defaults
|
||||||
|
* Can be disabled or customized via augmentation registry
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
import { AugmentationManifest } from './manifest.js';
|
||||||
|
import { SearchCache } from '../utils/searchCache.js';
|
||||||
|
import type { GraphNoun } from '../types/graphTypes.js';
|
||||||
|
export interface CacheConfig {
|
||||||
|
maxSize?: number;
|
||||||
|
ttl?: number;
|
||||||
|
enabled?: boolean;
|
||||||
|
invalidateOnWrite?: boolean;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* CacheAugmentation - Makes search caching optional and pluggable
|
||||||
|
*
|
||||||
|
* Features:
|
||||||
|
* - Transparent search result caching
|
||||||
|
* - Automatic invalidation on data changes
|
||||||
|
* - Memory-aware cache management
|
||||||
|
* - Zero-config with smart defaults
|
||||||
|
*/
|
||||||
|
export declare class CacheAugmentation extends BaseAugmentation {
|
||||||
|
readonly name = "cache";
|
||||||
|
readonly timing: "around";
|
||||||
|
readonly metadata: "none";
|
||||||
|
operations: ("search" | "find" | "similar" | "add" | "update" | "delete" | "clear" | "all")[];
|
||||||
|
readonly priority = 50;
|
||||||
|
readonly category: "core";
|
||||||
|
readonly description = "Transparent search result caching with automatic invalidation";
|
||||||
|
private searchCache;
|
||||||
|
constructor(config?: CacheConfig);
|
||||||
|
getManifest(): AugmentationManifest;
|
||||||
|
protected onInitialize(): Promise<void>;
|
||||||
|
protected onShutdown(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Execute augmentation - wrap operations with caching logic
|
||||||
|
*/
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
/**
|
||||||
|
* Handle search operation with caching
|
||||||
|
*/
|
||||||
|
private handleSearch;
|
||||||
|
/**
|
||||||
|
* Get cache statistics
|
||||||
|
*/
|
||||||
|
getStats(): {
|
||||||
|
enabled: boolean;
|
||||||
|
hits: number;
|
||||||
|
misses: number;
|
||||||
|
size: number;
|
||||||
|
memoryUsage: number;
|
||||||
|
} | {
|
||||||
|
memoryUsage: number;
|
||||||
|
hits: number;
|
||||||
|
misses: number;
|
||||||
|
evictions: number;
|
||||||
|
hitRate: number;
|
||||||
|
size: number;
|
||||||
|
maxSize: number;
|
||||||
|
enabled: boolean;
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Clear the cache manually
|
||||||
|
*/
|
||||||
|
clear(): void;
|
||||||
|
/**
|
||||||
|
* Handle runtime configuration changes
|
||||||
|
*/
|
||||||
|
protected onConfigChange(newConfig: CacheConfig, oldConfig: CacheConfig): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Clean up expired entries
|
||||||
|
*/
|
||||||
|
cleanupExpiredEntries(): number;
|
||||||
|
/**
|
||||||
|
* Invalidate cache when data changes
|
||||||
|
*/
|
||||||
|
invalidateOnDataChange(operation: 'add' | 'update' | 'delete'): void;
|
||||||
|
/**
|
||||||
|
* Get cache key for a query
|
||||||
|
*/
|
||||||
|
getCacheKey(query: any, options?: any): string;
|
||||||
|
/**
|
||||||
|
* Direct cache get
|
||||||
|
*/
|
||||||
|
get(key: string): any;
|
||||||
|
/**
|
||||||
|
* Direct cache set
|
||||||
|
*/
|
||||||
|
set(key: string, value: any): void;
|
||||||
|
/**
|
||||||
|
* Get the underlying SearchCache instance (for compatibility)
|
||||||
|
*/
|
||||||
|
getSearchCache(): SearchCache<GraphNoun> | null;
|
||||||
|
/**
|
||||||
|
* Get memory usage
|
||||||
|
*/
|
||||||
|
getMemoryUsage(): number;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Factory function for zero-config cache augmentation
|
||||||
|
*/
|
||||||
|
export declare function createCacheAugmentation(config?: CacheConfig): CacheAugmentation;
|
||||||
|
|
@ -0,0 +1,338 @@
|
||||||
|
/**
|
||||||
|
* Cache Augmentation - Optional Search Result Caching
|
||||||
|
*
|
||||||
|
* Replaces the hardcoded SearchCache in Brainy with an optional augmentation.
|
||||||
|
* This reduces core size and allows custom cache implementations.
|
||||||
|
*
|
||||||
|
* Zero-config: Automatically enabled with sensible defaults
|
||||||
|
* Can be disabled or customized via augmentation registry
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
import { SearchCache } from '../utils/searchCache.js';
|
||||||
|
/**
|
||||||
|
* CacheAugmentation - Makes search caching optional and pluggable
|
||||||
|
*
|
||||||
|
* Features:
|
||||||
|
* - Transparent search result caching
|
||||||
|
* - Automatic invalidation on data changes
|
||||||
|
* - Memory-aware cache management
|
||||||
|
* - Zero-config with smart defaults
|
||||||
|
*/
|
||||||
|
export class CacheAugmentation extends BaseAugmentation {
|
||||||
|
constructor(config) {
|
||||||
|
super(config);
|
||||||
|
this.name = 'cache';
|
||||||
|
this.timing = 'around';
|
||||||
|
this.metadata = 'none'; // Cache doesn't access metadata
|
||||||
|
this.operations = ['search', 'find', 'similar', 'add', 'update', 'delete', 'clear', 'all'];
|
||||||
|
this.priority = 50; // Mid-priority, runs after data operations
|
||||||
|
// Augmentation metadata
|
||||||
|
this.category = 'core';
|
||||||
|
this.description = 'Transparent search result caching with automatic invalidation';
|
||||||
|
this.searchCache = null;
|
||||||
|
}
|
||||||
|
getManifest() {
|
||||||
|
return {
|
||||||
|
id: 'cache',
|
||||||
|
name: 'Cache',
|
||||||
|
version: '2.0.0',
|
||||||
|
description: 'Intelligent caching for search and query operations',
|
||||||
|
longDescription: 'Provides transparent caching for search results with automatic invalidation on data changes. Significantly improves performance for repeated queries while maintaining data consistency.',
|
||||||
|
category: 'performance',
|
||||||
|
configSchema: {
|
||||||
|
type: 'object',
|
||||||
|
properties: {
|
||||||
|
enabled: {
|
||||||
|
type: 'boolean',
|
||||||
|
default: true,
|
||||||
|
description: 'Enable or disable caching'
|
||||||
|
},
|
||||||
|
maxSize: {
|
||||||
|
type: 'number',
|
||||||
|
default: 1000,
|
||||||
|
minimum: 10,
|
||||||
|
maximum: 100000,
|
||||||
|
description: 'Maximum number of cached entries'
|
||||||
|
},
|
||||||
|
ttl: {
|
||||||
|
type: 'number',
|
||||||
|
default: 300000, // 5 minutes
|
||||||
|
minimum: 1000, // 1 second
|
||||||
|
maximum: 3600000, // 1 hour
|
||||||
|
description: 'Time to live for cache entries in milliseconds'
|
||||||
|
},
|
||||||
|
invalidateOnWrite: {
|
||||||
|
type: 'boolean',
|
||||||
|
default: true,
|
||||||
|
description: 'Automatically invalidate cache on data modifications'
|
||||||
|
}
|
||||||
|
},
|
||||||
|
additionalProperties: false
|
||||||
|
},
|
||||||
|
configDefaults: {
|
||||||
|
enabled: true,
|
||||||
|
maxSize: 1000,
|
||||||
|
ttl: 300000,
|
||||||
|
invalidateOnWrite: true
|
||||||
|
},
|
||||||
|
configExamples: [
|
||||||
|
{
|
||||||
|
name: 'High Performance',
|
||||||
|
description: 'Large cache with longer TTL for read-heavy workloads',
|
||||||
|
config: {
|
||||||
|
enabled: true,
|
||||||
|
maxSize: 10000,
|
||||||
|
ttl: 1800000, // 30 minutes
|
||||||
|
invalidateOnWrite: true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'Conservative',
|
||||||
|
description: 'Small cache with short TTL for frequently changing data',
|
||||||
|
config: {
|
||||||
|
enabled: true,
|
||||||
|
maxSize: 100,
|
||||||
|
ttl: 60000, // 1 minute
|
||||||
|
invalidateOnWrite: true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
minBrainyVersion: '2.0.0',
|
||||||
|
keywords: ['cache', 'performance', 'search', 'optimization'],
|
||||||
|
documentation: 'https://docs.brainy.dev/augmentations/cache',
|
||||||
|
status: 'stable',
|
||||||
|
performance: {
|
||||||
|
memoryUsage: 'medium',
|
||||||
|
cpuUsage: 'low',
|
||||||
|
networkUsage: 'none'
|
||||||
|
},
|
||||||
|
features: ['search-caching', 'auto-invalidation', 'ttl-support', 'memory-management'],
|
||||||
|
enhancedOperations: ['search', 'searchText', 'findSimilar'],
|
||||||
|
metrics: [
|
||||||
|
{
|
||||||
|
name: 'cache_hits',
|
||||||
|
type: 'counter',
|
||||||
|
description: 'Number of cache hits'
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'cache_misses',
|
||||||
|
type: 'counter',
|
||||||
|
description: 'Number of cache misses'
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'cache_size',
|
||||||
|
type: 'gauge',
|
||||||
|
description: 'Current cache size'
|
||||||
|
}
|
||||||
|
],
|
||||||
|
ui: {
|
||||||
|
icon: '⚡',
|
||||||
|
color: '#FFC107'
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
async onInitialize() {
|
||||||
|
if (!this.config.enabled) {
|
||||||
|
this.log('Cache augmentation disabled by configuration');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Initialize search cache with config
|
||||||
|
this.searchCache = new SearchCache({
|
||||||
|
maxSize: this.config.maxSize,
|
||||||
|
maxAge: this.config.ttl, // SearchCache uses maxAge, not ttl
|
||||||
|
enabled: true
|
||||||
|
});
|
||||||
|
this.log(`Cache augmentation initialized (maxSize: ${this.config.maxSize}, ttl: ${this.config.ttl}ms)`);
|
||||||
|
}
|
||||||
|
async onShutdown() {
|
||||||
|
if (this.searchCache) {
|
||||||
|
this.searchCache.clear();
|
||||||
|
this.searchCache = null;
|
||||||
|
}
|
||||||
|
this.log('Cache augmentation shut down');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Execute augmentation - wrap operations with caching logic
|
||||||
|
*/
|
||||||
|
async execute(operation, params, next) {
|
||||||
|
// If cache is disabled, just pass through
|
||||||
|
if (!this.searchCache || !this.config.enabled) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
switch (operation) {
|
||||||
|
case 'search':
|
||||||
|
return this.handleSearch(params, next);
|
||||||
|
case 'add':
|
||||||
|
case 'update':
|
||||||
|
case 'delete':
|
||||||
|
// Invalidate cache on data changes
|
||||||
|
if (this.config.invalidateOnWrite) {
|
||||||
|
const result = await next();
|
||||||
|
this.searchCache.invalidateOnDataChange(operation);
|
||||||
|
this.log(`Cache invalidated due to ${operation} operation`);
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
return next();
|
||||||
|
case 'clear':
|
||||||
|
// Clear cache when all data is cleared
|
||||||
|
const result = await next();
|
||||||
|
this.searchCache.clear();
|
||||||
|
this.log('Cache cleared due to clear operation');
|
||||||
|
return result;
|
||||||
|
default:
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Handle search operation with caching
|
||||||
|
*/
|
||||||
|
async handleSearch(params, next) {
|
||||||
|
if (!this.searchCache)
|
||||||
|
return next();
|
||||||
|
// Extract search parameters
|
||||||
|
const { query, k, options = {} } = params;
|
||||||
|
// Skip cache if explicitly disabled or has complex filters
|
||||||
|
if (options.skipCache || options.metadata) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
// Generate cache key
|
||||||
|
const cacheKey = this.searchCache.getCacheKey(query, k, options);
|
||||||
|
// Check cache
|
||||||
|
const cachedResult = this.searchCache.get(cacheKey);
|
||||||
|
if (cachedResult) {
|
||||||
|
this.log('Cache hit for search query');
|
||||||
|
// Update metrics if available
|
||||||
|
if (this.context?.brain) {
|
||||||
|
const metrics = this.context.brain.augmentations?.get('metrics');
|
||||||
|
if (metrics) {
|
||||||
|
metrics.recordCacheHit?.();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return cachedResult;
|
||||||
|
}
|
||||||
|
// Execute search
|
||||||
|
const result = await next();
|
||||||
|
// Cache the result
|
||||||
|
this.searchCache.set(cacheKey, result);
|
||||||
|
this.log('Search result cached');
|
||||||
|
// Update metrics if available
|
||||||
|
if (this.context?.brain) {
|
||||||
|
const metrics = this.context.brain.augmentations?.get('metrics');
|
||||||
|
if (metrics) {
|
||||||
|
metrics.recordCacheMiss?.();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get cache statistics
|
||||||
|
*/
|
||||||
|
getStats() {
|
||||||
|
if (!this.searchCache) {
|
||||||
|
return {
|
||||||
|
enabled: false,
|
||||||
|
hits: 0,
|
||||||
|
misses: 0,
|
||||||
|
size: 0,
|
||||||
|
memoryUsage: 0
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const stats = this.searchCache.getStats();
|
||||||
|
return {
|
||||||
|
...stats,
|
||||||
|
memoryUsage: this.searchCache.getMemoryUsage()
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear the cache manually
|
||||||
|
*/
|
||||||
|
clear() {
|
||||||
|
if (this.searchCache) {
|
||||||
|
this.searchCache.clear();
|
||||||
|
this.log('Cache manually cleared');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Handle runtime configuration changes
|
||||||
|
*/
|
||||||
|
async onConfigChange(newConfig, oldConfig) {
|
||||||
|
if (this.searchCache && newConfig.enabled) {
|
||||||
|
this.searchCache.updateConfig({
|
||||||
|
maxSize: newConfig.maxSize,
|
||||||
|
maxAge: newConfig.ttl, // SearchCache uses maxAge
|
||||||
|
enabled: newConfig.enabled
|
||||||
|
});
|
||||||
|
this.log('Cache configuration updated');
|
||||||
|
}
|
||||||
|
else if (!newConfig.enabled && this.searchCache) {
|
||||||
|
this.searchCache.clear();
|
||||||
|
this.log('Cache disabled and cleared');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clean up expired entries
|
||||||
|
*/
|
||||||
|
cleanupExpiredEntries() {
|
||||||
|
if (!this.searchCache)
|
||||||
|
return 0;
|
||||||
|
const cleaned = this.searchCache.cleanupExpiredEntries();
|
||||||
|
if (cleaned > 0) {
|
||||||
|
this.log(`Cleaned ${cleaned} expired cache entries`);
|
||||||
|
}
|
||||||
|
return cleaned;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Invalidate cache when data changes
|
||||||
|
*/
|
||||||
|
invalidateOnDataChange(operation) {
|
||||||
|
if (!this.searchCache)
|
||||||
|
return;
|
||||||
|
this.searchCache.invalidateOnDataChange(operation);
|
||||||
|
this.log(`Cache invalidated due to ${operation} operation`);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get cache key for a query
|
||||||
|
*/
|
||||||
|
getCacheKey(query, options) {
|
||||||
|
if (!this.searchCache)
|
||||||
|
return '';
|
||||||
|
return this.searchCache.getCacheKey(query, options);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Direct cache get
|
||||||
|
*/
|
||||||
|
get(key) {
|
||||||
|
if (!this.searchCache)
|
||||||
|
return null;
|
||||||
|
return this.searchCache.get(key);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Direct cache set
|
||||||
|
*/
|
||||||
|
set(key, value) {
|
||||||
|
if (!this.searchCache)
|
||||||
|
return;
|
||||||
|
this.searchCache.set(key, value);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get the underlying SearchCache instance (for compatibility)
|
||||||
|
*/
|
||||||
|
getSearchCache() {
|
||||||
|
return this.searchCache;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get memory usage
|
||||||
|
*/
|
||||||
|
getMemoryUsage() {
|
||||||
|
if (!this.searchCache)
|
||||||
|
return 0;
|
||||||
|
return this.searchCache.getMemoryUsage();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Factory function for zero-config cache augmentation
|
||||||
|
*/
|
||||||
|
export function createCacheAugmentation(config) {
|
||||||
|
return new CacheAugmentation(config);
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=cacheAugmentation.js.map
|
||||||
File diff suppressed because one or more lines are too long
71
.recovery-workspace/dist-backup-20250910-141917/augmentations/conduitAugmentations.d.ts
vendored
Normal file
71
.recovery-workspace/dist-backup-20250910-141917/augmentations/conduitAugmentations.d.ts
vendored
Normal file
|
|
@ -0,0 +1,71 @@
|
||||||
|
/**
|
||||||
|
* Conduit Augmentations - Data Synchronization Bridges
|
||||||
|
*
|
||||||
|
* These augmentations connect and synchronize data between multiple Brainy instances.
|
||||||
|
* Now using the unified BrainyAugmentation interface.
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
export interface WebSocketConnection {
|
||||||
|
connectionId: string;
|
||||||
|
url: string;
|
||||||
|
readyState: number;
|
||||||
|
socket?: any;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Base class for conduit augmentations that sync between Brainy instances
|
||||||
|
* Converted to use the unified BrainyAugmentation interface
|
||||||
|
*/
|
||||||
|
declare abstract class BaseConduitAugmentation extends BaseAugmentation {
|
||||||
|
readonly timing: "after";
|
||||||
|
readonly metadata: "readonly";
|
||||||
|
readonly operations: ("addNoun" | "delete" | "addVerb")[];
|
||||||
|
readonly priority = 20;
|
||||||
|
protected connections: Map<string, any>;
|
||||||
|
protected onShutdown(): Promise<void>;
|
||||||
|
abstract establishConnection(targetSystemId: string, config?: Record<string, unknown>): Promise<WebSocketConnection | null>;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* WebSocket Conduit Augmentation
|
||||||
|
* Syncs data between Brainy instances using WebSockets
|
||||||
|
*/
|
||||||
|
export declare class WebSocketConduitAugmentation extends BaseConduitAugmentation {
|
||||||
|
readonly name = "websocket-conduit";
|
||||||
|
private webSocketConnections;
|
||||||
|
private messageCallbacks;
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
private shouldSync;
|
||||||
|
private syncOperation;
|
||||||
|
establishConnection(url: string, config?: Record<string, unknown>): Promise<WebSocketConnection | null>;
|
||||||
|
private handleMessage;
|
||||||
|
private applySyncOperation;
|
||||||
|
/**
|
||||||
|
* Subscribe to messages from a specific connection
|
||||||
|
*/
|
||||||
|
onMessage(connectionId: string, callback: (data: any) => void): void;
|
||||||
|
/**
|
||||||
|
* Send a message to a specific connection
|
||||||
|
*/
|
||||||
|
sendMessage(connectionId: string, data: any): boolean;
|
||||||
|
}
|
||||||
|
export {};
|
||||||
|
/**
|
||||||
|
* Example usage:
|
||||||
|
*
|
||||||
|
* // Server instance
|
||||||
|
* const serverBrain = new Brainy()
|
||||||
|
* serverBrain.augmentations.register(new APIServerAugmentation())
|
||||||
|
* await serverBrain.init()
|
||||||
|
*
|
||||||
|
* // Client instance
|
||||||
|
* const clientBrain = new Brainy()
|
||||||
|
* const conduit = new WebSocketConduitAugmentation()
|
||||||
|
* clientBrain.augmentations.register(conduit)
|
||||||
|
* await clientBrain.init()
|
||||||
|
*
|
||||||
|
* // Connect client to server
|
||||||
|
* await conduit.establishConnection('ws://localhost:3000/ws')
|
||||||
|
*
|
||||||
|
* // Now operations sync automatically!
|
||||||
|
* await clientBrain.addNoun('synced data', { source: 'client' })
|
||||||
|
* // This will automatically sync to the server
|
||||||
|
*/
|
||||||
|
|
@ -0,0 +1,233 @@
|
||||||
|
/**
|
||||||
|
* Conduit Augmentations - Data Synchronization Bridges
|
||||||
|
*
|
||||||
|
* These augmentations connect and synchronize data between multiple Brainy instances.
|
||||||
|
* Now using the unified BrainyAugmentation interface.
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
import { v4 as uuidv4 } from '../universal/uuid.js';
|
||||||
|
/**
|
||||||
|
* Base class for conduit augmentations that sync between Brainy instances
|
||||||
|
* Converted to use the unified BrainyAugmentation interface
|
||||||
|
*/
|
||||||
|
class BaseConduitAugmentation extends BaseAugmentation {
|
||||||
|
constructor() {
|
||||||
|
super(...arguments);
|
||||||
|
this.timing = 'after'; // Conduits run after operations to sync
|
||||||
|
this.metadata = 'readonly'; // Conduits read metadata to pass to external systems
|
||||||
|
this.operations = ['addNoun', 'delete', 'addVerb'];
|
||||||
|
this.priority = 20; // Medium-low priority
|
||||||
|
this.connections = new Map();
|
||||||
|
}
|
||||||
|
async onShutdown() {
|
||||||
|
// Close all connections
|
||||||
|
for (const [connectionId, connection] of this.connections.entries()) {
|
||||||
|
try {
|
||||||
|
if (connection.close) {
|
||||||
|
await connection.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log(`Failed to close connection ${connectionId}: ${error}`, 'error');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this.connections.clear();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* WebSocket Conduit Augmentation
|
||||||
|
* Syncs data between Brainy instances using WebSockets
|
||||||
|
*/
|
||||||
|
export class WebSocketConduitAugmentation extends BaseConduitAugmentation {
|
||||||
|
constructor() {
|
||||||
|
super(...arguments);
|
||||||
|
this.name = 'websocket-conduit';
|
||||||
|
this.webSocketConnections = new Map();
|
||||||
|
this.messageCallbacks = new Map();
|
||||||
|
}
|
||||||
|
async execute(operation, params, next) {
|
||||||
|
// Execute the operation first
|
||||||
|
const result = await next();
|
||||||
|
// Then sync to connected instances
|
||||||
|
if (this.shouldSync(operation)) {
|
||||||
|
await this.syncOperation(operation, params, result);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
shouldSync(operation) {
|
||||||
|
return ['addNoun', 'deleteNoun', 'addVerb'].includes(operation);
|
||||||
|
}
|
||||||
|
async syncOperation(operation, params, result) {
|
||||||
|
// Broadcast to all connected WebSocket instances
|
||||||
|
for (const [id, connection] of this.webSocketConnections) {
|
||||||
|
if (connection.socket && connection.readyState === 1) { // OPEN state
|
||||||
|
try {
|
||||||
|
const message = JSON.stringify({
|
||||||
|
type: 'sync',
|
||||||
|
operation,
|
||||||
|
params,
|
||||||
|
timestamp: Date.now()
|
||||||
|
});
|
||||||
|
if (typeof connection.socket.send === 'function') {
|
||||||
|
connection.socket.send(message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log(`Failed to sync to ${id}: ${error}`, 'error');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async establishConnection(url, config) {
|
||||||
|
try {
|
||||||
|
const connectionId = uuidv4();
|
||||||
|
const protocols = config?.protocols;
|
||||||
|
// Create WebSocket based on environment
|
||||||
|
let socket;
|
||||||
|
if (typeof WebSocket !== 'undefined') {
|
||||||
|
// Browser environment
|
||||||
|
socket = new WebSocket(url, protocols);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
// Node.js environment - dynamic import
|
||||||
|
try {
|
||||||
|
const ws = await import('ws');
|
||||||
|
socket = new ws.WebSocket(url, protocols);
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
this.log('WebSocket not available in this environment', 'error');
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Setup event handlers
|
||||||
|
socket.onopen = () => {
|
||||||
|
this.log(`Connected to ${url}`);
|
||||||
|
};
|
||||||
|
socket.onmessage = (event) => {
|
||||||
|
this.handleMessage(connectionId, event.data);
|
||||||
|
};
|
||||||
|
socket.onerror = (error) => {
|
||||||
|
this.log(`WebSocket error: ${error}`, 'error');
|
||||||
|
};
|
||||||
|
socket.onclose = () => {
|
||||||
|
this.log(`Disconnected from ${url}`);
|
||||||
|
this.webSocketConnections.delete(connectionId);
|
||||||
|
};
|
||||||
|
// Wait for connection to open
|
||||||
|
await new Promise((resolve, reject) => {
|
||||||
|
const timeout = setTimeout(() => {
|
||||||
|
reject(new Error('Connection timeout'));
|
||||||
|
}, 5000);
|
||||||
|
socket.onopen = () => {
|
||||||
|
clearTimeout(timeout);
|
||||||
|
resolve();
|
||||||
|
};
|
||||||
|
socket.onerror = (error) => {
|
||||||
|
clearTimeout(timeout);
|
||||||
|
reject(error);
|
||||||
|
};
|
||||||
|
});
|
||||||
|
const connection = {
|
||||||
|
connectionId,
|
||||||
|
url,
|
||||||
|
readyState: socket.readyState,
|
||||||
|
socket
|
||||||
|
};
|
||||||
|
this.webSocketConnections.set(connectionId, connection);
|
||||||
|
this.connections.set(connectionId, connection);
|
||||||
|
return connection;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log(`Failed to establish connection to ${url}: ${error}`, 'error');
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
handleMessage(connectionId, data) {
|
||||||
|
try {
|
||||||
|
const message = typeof data === 'string' ? JSON.parse(data) : data;
|
||||||
|
// Handle sync messages from remote instances
|
||||||
|
if (message.type === 'sync') {
|
||||||
|
// Apply the operation to our local instance
|
||||||
|
this.applySyncOperation(message).catch(error => {
|
||||||
|
this.log(`Failed to apply sync operation: ${error}`, 'error');
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// Notify any registered callbacks
|
||||||
|
const callbacks = this.messageCallbacks.get(connectionId);
|
||||||
|
if (callbacks) {
|
||||||
|
callbacks.forEach(callback => callback(message));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log(`Failed to handle message: ${error}`, 'error');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async applySyncOperation(message) {
|
||||||
|
// Apply the synced operation to our local Brainy instance
|
||||||
|
const { operation, params } = message;
|
||||||
|
try {
|
||||||
|
switch (operation) {
|
||||||
|
case 'addNoun':
|
||||||
|
await this.context?.brain.addNoun(params.content, params.metadata);
|
||||||
|
break;
|
||||||
|
case 'deleteNoun':
|
||||||
|
await this.context?.brain.deleteNoun(params.id);
|
||||||
|
break;
|
||||||
|
case 'addVerb':
|
||||||
|
await this.context?.brain.addVerb(params.source, params.target, params.verb, params.metadata);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log(`Failed to apply ${operation}: ${error}`, 'error');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Subscribe to messages from a specific connection
|
||||||
|
*/
|
||||||
|
onMessage(connectionId, callback) {
|
||||||
|
if (!this.messageCallbacks.has(connectionId)) {
|
||||||
|
this.messageCallbacks.set(connectionId, new Set());
|
||||||
|
}
|
||||||
|
this.messageCallbacks.get(connectionId).add(callback);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Send a message to a specific connection
|
||||||
|
*/
|
||||||
|
sendMessage(connectionId, data) {
|
||||||
|
const connection = this.webSocketConnections.get(connectionId);
|
||||||
|
if (connection?.socket && connection.readyState === 1) {
|
||||||
|
try {
|
||||||
|
const message = typeof data === 'string' ? data : JSON.stringify(data);
|
||||||
|
connection.socket.send(message);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log(`Failed to send message: ${error}`, 'error');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Example usage:
|
||||||
|
*
|
||||||
|
* // Server instance
|
||||||
|
* const serverBrain = new Brainy()
|
||||||
|
* serverBrain.augmentations.register(new APIServerAugmentation())
|
||||||
|
* await serverBrain.init()
|
||||||
|
*
|
||||||
|
* // Client instance
|
||||||
|
* const clientBrain = new Brainy()
|
||||||
|
* const conduit = new WebSocketConduitAugmentation()
|
||||||
|
* clientBrain.augmentations.register(conduit)
|
||||||
|
* await clientBrain.init()
|
||||||
|
*
|
||||||
|
* // Connect client to server
|
||||||
|
* await conduit.establishConnection('ws://localhost:3000/ws')
|
||||||
|
*
|
||||||
|
* // Now operations sync automatically!
|
||||||
|
* await clientBrain.addNoun('synced data', { source: 'client' })
|
||||||
|
* // This will automatically sync to the server
|
||||||
|
*/
|
||||||
|
//# sourceMappingURL=conduitAugmentations.js.map
|
||||||
File diff suppressed because one or more lines are too long
122
.recovery-workspace/dist-backup-20250910-141917/augmentations/configResolver.d.ts
vendored
Normal file
122
.recovery-workspace/dist-backup-20250910-141917/augmentations/configResolver.d.ts
vendored
Normal file
|
|
@ -0,0 +1,122 @@
|
||||||
|
/**
|
||||||
|
* Configuration Resolver for Augmentations
|
||||||
|
*
|
||||||
|
* Handles loading and resolving configuration from multiple sources:
|
||||||
|
* - Environment variables
|
||||||
|
* - Configuration files
|
||||||
|
* - Runtime updates
|
||||||
|
* - Default values from schema
|
||||||
|
*/
|
||||||
|
import { JSONSchema } from './manifest.js';
|
||||||
|
/**
|
||||||
|
* Configuration source priority (highest to lowest)
|
||||||
|
*/
|
||||||
|
export declare enum ConfigPriority {
|
||||||
|
RUNTIME = 4,// Runtime updates (highest priority)
|
||||||
|
CONSTRUCTOR = 3,// Constructor parameters
|
||||||
|
ENVIRONMENT = 2,// Environment variables
|
||||||
|
FILE = 1,// Configuration files
|
||||||
|
DEFAULT = 0
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Configuration source information
|
||||||
|
*/
|
||||||
|
export interface ConfigSource {
|
||||||
|
priority: ConfigPriority;
|
||||||
|
source: string;
|
||||||
|
config: any;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Configuration resolution options
|
||||||
|
*/
|
||||||
|
export interface ConfigResolverOptions {
|
||||||
|
augmentationId: string;
|
||||||
|
schema?: JSONSchema;
|
||||||
|
defaults?: Record<string, any>;
|
||||||
|
configPaths?: string[];
|
||||||
|
envPrefix?: string;
|
||||||
|
allowUndefined?: boolean;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Augmentation Configuration Resolver
|
||||||
|
*/
|
||||||
|
export declare class AugmentationConfigResolver {
|
||||||
|
private options;
|
||||||
|
private sources;
|
||||||
|
private resolved;
|
||||||
|
constructor(options: ConfigResolverOptions);
|
||||||
|
/**
|
||||||
|
* Resolve configuration from all sources
|
||||||
|
* @param constructorConfig Optional constructor configuration
|
||||||
|
* @returns Resolved configuration
|
||||||
|
*/
|
||||||
|
resolve(constructorConfig?: any): any;
|
||||||
|
/**
|
||||||
|
* Load default values from schema and defaults
|
||||||
|
*/
|
||||||
|
private loadDefaults;
|
||||||
|
/**
|
||||||
|
* Load configuration from files
|
||||||
|
*/
|
||||||
|
private loadFromFiles;
|
||||||
|
/**
|
||||||
|
* Parse configuration file based on extension
|
||||||
|
*/
|
||||||
|
private parseConfigFile;
|
||||||
|
/**
|
||||||
|
* Extract augmentation-specific configuration from a config object
|
||||||
|
*/
|
||||||
|
private extractAugmentationConfig;
|
||||||
|
/**
|
||||||
|
* Load configuration from environment variables
|
||||||
|
*/
|
||||||
|
private loadFromEnvironment;
|
||||||
|
/**
|
||||||
|
* Convert environment variable key to config key
|
||||||
|
* ENABLED -> enabled
|
||||||
|
* MAX_SIZE -> maxSize
|
||||||
|
*/
|
||||||
|
private envKeyToConfigKey;
|
||||||
|
/**
|
||||||
|
* Parse environment variable value
|
||||||
|
*/
|
||||||
|
private parseEnvValue;
|
||||||
|
/**
|
||||||
|
* Merge configurations by priority
|
||||||
|
*/
|
||||||
|
private mergeConfigurations;
|
||||||
|
/**
|
||||||
|
* Deep merge two objects
|
||||||
|
*/
|
||||||
|
private deepMerge;
|
||||||
|
/**
|
||||||
|
* Validate configuration against schema
|
||||||
|
*/
|
||||||
|
private validateConfiguration;
|
||||||
|
/**
|
||||||
|
* Validate a single property against its schema
|
||||||
|
*/
|
||||||
|
private validateProperty;
|
||||||
|
/**
|
||||||
|
* Get configuration sources for debugging
|
||||||
|
*/
|
||||||
|
getSources(): ConfigSource[];
|
||||||
|
/**
|
||||||
|
* Get resolved configuration
|
||||||
|
*/
|
||||||
|
getResolved(): any;
|
||||||
|
/**
|
||||||
|
* Update configuration at runtime
|
||||||
|
*/
|
||||||
|
updateRuntime(config: any): any;
|
||||||
|
/**
|
||||||
|
* Save configuration to file
|
||||||
|
* @param filepath Path to save configuration
|
||||||
|
* @param format Format to save as (json, etc.)
|
||||||
|
*/
|
||||||
|
saveToFile(filepath?: string, format?: 'json'): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Get environment variable names for this augmentation
|
||||||
|
*/
|
||||||
|
getEnvironmentVariables(): Record<string, any>;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,440 @@
|
||||||
|
/**
|
||||||
|
* Configuration Resolver for Augmentations
|
||||||
|
*
|
||||||
|
* Handles loading and resolving configuration from multiple sources:
|
||||||
|
* - Environment variables
|
||||||
|
* - Configuration files
|
||||||
|
* - Runtime updates
|
||||||
|
* - Default values from schema
|
||||||
|
*/
|
||||||
|
import { existsSync, readFileSync } from 'fs';
|
||||||
|
import { join } from 'path';
|
||||||
|
import { homedir } from 'os';
|
||||||
|
/**
|
||||||
|
* Configuration source priority (highest to lowest)
|
||||||
|
*/
|
||||||
|
export var ConfigPriority;
|
||||||
|
(function (ConfigPriority) {
|
||||||
|
ConfigPriority[ConfigPriority["RUNTIME"] = 4] = "RUNTIME";
|
||||||
|
ConfigPriority[ConfigPriority["CONSTRUCTOR"] = 3] = "CONSTRUCTOR";
|
||||||
|
ConfigPriority[ConfigPriority["ENVIRONMENT"] = 2] = "ENVIRONMENT";
|
||||||
|
ConfigPriority[ConfigPriority["FILE"] = 1] = "FILE";
|
||||||
|
ConfigPriority[ConfigPriority["DEFAULT"] = 0] = "DEFAULT"; // Schema defaults (lowest priority)
|
||||||
|
})(ConfigPriority || (ConfigPriority = {}));
|
||||||
|
/**
|
||||||
|
* Augmentation Configuration Resolver
|
||||||
|
*/
|
||||||
|
export class AugmentationConfigResolver {
|
||||||
|
constructor(options) {
|
||||||
|
this.options = options;
|
||||||
|
this.sources = [];
|
||||||
|
this.resolved = {};
|
||||||
|
this.options = {
|
||||||
|
configPaths: [
|
||||||
|
'.brainyrc',
|
||||||
|
'.brainyrc.json',
|
||||||
|
'brainy.config.json',
|
||||||
|
join(homedir(), '.brainy', 'config.json'),
|
||||||
|
join(homedir(), '.brainyrc')
|
||||||
|
],
|
||||||
|
envPrefix: `BRAINY_AUG_${options.augmentationId.toUpperCase()}_`,
|
||||||
|
allowUndefined: true,
|
||||||
|
...options
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Resolve configuration from all sources
|
||||||
|
* @param constructorConfig Optional constructor configuration
|
||||||
|
* @returns Resolved configuration
|
||||||
|
*/
|
||||||
|
resolve(constructorConfig) {
|
||||||
|
this.sources = [];
|
||||||
|
// Load from all sources in priority order
|
||||||
|
this.loadDefaults();
|
||||||
|
this.loadFromFiles();
|
||||||
|
this.loadFromEnvironment();
|
||||||
|
if (constructorConfig) {
|
||||||
|
this.sources.push({
|
||||||
|
priority: ConfigPriority.CONSTRUCTOR,
|
||||||
|
source: 'constructor',
|
||||||
|
config: constructorConfig
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// Merge configurations by priority
|
||||||
|
this.resolved = this.mergeConfigurations();
|
||||||
|
// Validate against schema if provided
|
||||||
|
if (this.options.schema) {
|
||||||
|
this.validateConfiguration(this.resolved);
|
||||||
|
}
|
||||||
|
return this.resolved;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load default values from schema and defaults
|
||||||
|
*/
|
||||||
|
loadDefaults() {
|
||||||
|
let defaults = {};
|
||||||
|
// Load from provided defaults
|
||||||
|
if (this.options.defaults) {
|
||||||
|
defaults = { ...defaults, ...this.options.defaults };
|
||||||
|
}
|
||||||
|
// Load from schema defaults
|
||||||
|
if (this.options.schema?.properties) {
|
||||||
|
for (const [key, prop] of Object.entries(this.options.schema.properties)) {
|
||||||
|
if (prop.default !== undefined && defaults[key] === undefined) {
|
||||||
|
defaults[key] = prop.default;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (Object.keys(defaults).length > 0) {
|
||||||
|
this.sources.push({
|
||||||
|
priority: ConfigPriority.DEFAULT,
|
||||||
|
source: 'defaults',
|
||||||
|
config: defaults
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load configuration from files
|
||||||
|
*/
|
||||||
|
loadFromFiles() {
|
||||||
|
// Skip in browser environment
|
||||||
|
if (typeof process === 'undefined' || typeof window !== 'undefined') {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
for (const configPath of this.options.configPaths || []) {
|
||||||
|
try {
|
||||||
|
if (existsSync(configPath)) {
|
||||||
|
const content = readFileSync(configPath, 'utf8');
|
||||||
|
const config = this.parseConfigFile(content, configPath);
|
||||||
|
// Extract augmentation-specific configuration
|
||||||
|
const augConfig = this.extractAugmentationConfig(config);
|
||||||
|
if (augConfig && Object.keys(augConfig).length > 0) {
|
||||||
|
this.sources.push({
|
||||||
|
priority: ConfigPriority.FILE,
|
||||||
|
source: `file:${configPath}`,
|
||||||
|
config: augConfig
|
||||||
|
});
|
||||||
|
break; // Use first found config file
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
// Silently ignore file errors
|
||||||
|
console.debug(`Failed to load config from ${configPath}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Parse configuration file based on extension
|
||||||
|
*/
|
||||||
|
parseConfigFile(content, filepath) {
|
||||||
|
try {
|
||||||
|
// Try JSON first
|
||||||
|
return JSON.parse(content);
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
// Try other formats in the future (YAML, TOML, etc.)
|
||||||
|
throw new Error(`Unable to parse config file: ${filepath}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Extract augmentation-specific configuration from a config object
|
||||||
|
*/
|
||||||
|
extractAugmentationConfig(config) {
|
||||||
|
const augId = this.options.augmentationId;
|
||||||
|
// Check for augmentations section
|
||||||
|
if (config.augmentations && config.augmentations[augId]) {
|
||||||
|
return config.augmentations[augId];
|
||||||
|
}
|
||||||
|
// Check for direct augmentation config (prefixed keys)
|
||||||
|
const prefix = `${augId}.`;
|
||||||
|
const augConfig = {};
|
||||||
|
for (const [key, value] of Object.entries(config)) {
|
||||||
|
if (key.startsWith(prefix)) {
|
||||||
|
const configKey = key.slice(prefix.length);
|
||||||
|
augConfig[configKey] = value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return Object.keys(augConfig).length > 0 ? augConfig : null;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load configuration from environment variables
|
||||||
|
*/
|
||||||
|
loadFromEnvironment() {
|
||||||
|
// Skip in browser environment
|
||||||
|
if (typeof process === 'undefined' || !process.env) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const prefix = this.options.envPrefix;
|
||||||
|
const envConfig = {};
|
||||||
|
for (const [key, value] of Object.entries(process.env)) {
|
||||||
|
if (key.startsWith(prefix)) {
|
||||||
|
const configKey = this.envKeyToConfigKey(key.slice(prefix.length));
|
||||||
|
envConfig[configKey] = this.parseEnvValue(value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (Object.keys(envConfig).length > 0) {
|
||||||
|
this.sources.push({
|
||||||
|
priority: ConfigPriority.ENVIRONMENT,
|
||||||
|
source: 'environment',
|
||||||
|
config: envConfig
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Convert environment variable key to config key
|
||||||
|
* ENABLED -> enabled
|
||||||
|
* MAX_SIZE -> maxSize
|
||||||
|
*/
|
||||||
|
envKeyToConfigKey(envKey) {
|
||||||
|
return envKey
|
||||||
|
.toLowerCase()
|
||||||
|
.replace(/_([a-z])/g, (_, letter) => letter.toUpperCase());
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Parse environment variable value
|
||||||
|
*/
|
||||||
|
parseEnvValue(value) {
|
||||||
|
// Handle empty strings
|
||||||
|
if (value === '')
|
||||||
|
return value;
|
||||||
|
// Try to parse as JSON
|
||||||
|
try {
|
||||||
|
return JSON.parse(value);
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
// Check for boolean strings
|
||||||
|
if (value.toLowerCase() === 'true')
|
||||||
|
return true;
|
||||||
|
if (value.toLowerCase() === 'false')
|
||||||
|
return false;
|
||||||
|
// Check for number strings
|
||||||
|
const num = Number(value);
|
||||||
|
if (!isNaN(num) && value.trim() !== '')
|
||||||
|
return num;
|
||||||
|
// Return as string
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Merge configurations by priority
|
||||||
|
*/
|
||||||
|
mergeConfigurations() {
|
||||||
|
// Sort by priority (lowest to highest)
|
||||||
|
const sorted = [...this.sources].sort((a, b) => a.priority - b.priority);
|
||||||
|
// Merge configurations
|
||||||
|
let merged = {};
|
||||||
|
for (const source of sorted) {
|
||||||
|
merged = this.deepMerge(merged, source.config);
|
||||||
|
}
|
||||||
|
return merged;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Deep merge two objects
|
||||||
|
*/
|
||||||
|
deepMerge(target, source) {
|
||||||
|
const output = { ...target };
|
||||||
|
for (const key in source) {
|
||||||
|
if (source.hasOwnProperty(key)) {
|
||||||
|
if (source[key] && typeof source[key] === 'object' && !Array.isArray(source[key])) {
|
||||||
|
if (target[key] && typeof target[key] === 'object' && !Array.isArray(target[key])) {
|
||||||
|
output[key] = this.deepMerge(target[key], source[key]);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
output[key] = source[key];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
output[key] = source[key];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return output;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Validate configuration against schema
|
||||||
|
*/
|
||||||
|
validateConfiguration(config) {
|
||||||
|
if (!this.options.schema)
|
||||||
|
return;
|
||||||
|
const schema = this.options.schema;
|
||||||
|
const errors = [];
|
||||||
|
// Check required fields
|
||||||
|
if (schema.required) {
|
||||||
|
for (const field of schema.required) {
|
||||||
|
if (config[field] === undefined) {
|
||||||
|
errors.push(`Missing required field: ${field}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Validate properties
|
||||||
|
if (schema.properties) {
|
||||||
|
for (const [key, propSchema] of Object.entries(schema.properties)) {
|
||||||
|
const value = config[key];
|
||||||
|
if (value !== undefined) {
|
||||||
|
this.validateProperty(key, value, propSchema, errors);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Check for additional properties
|
||||||
|
if (schema.additionalProperties === false) {
|
||||||
|
const allowedKeys = Object.keys(schema.properties || {});
|
||||||
|
for (const key of Object.keys(config)) {
|
||||||
|
if (!allowedKeys.includes(key)) {
|
||||||
|
errors.push(`Unknown configuration property: ${key}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (errors.length > 0) {
|
||||||
|
throw new Error(`Configuration validation failed for ${this.options.augmentationId}:\n${errors.join('\n')}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Validate a single property against its schema
|
||||||
|
*/
|
||||||
|
validateProperty(key, value, schema, errors) {
|
||||||
|
// Type validation
|
||||||
|
if (schema.type) {
|
||||||
|
const actualType = Array.isArray(value) ? 'array' : typeof value;
|
||||||
|
if (actualType !== schema.type) {
|
||||||
|
errors.push(`${key}: expected ${schema.type}, got ${actualType}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Number validations
|
||||||
|
if (schema.type === 'number') {
|
||||||
|
if (schema.minimum !== undefined && value < schema.minimum) {
|
||||||
|
errors.push(`${key}: value ${value} is less than minimum ${schema.minimum}`);
|
||||||
|
}
|
||||||
|
if (schema.maximum !== undefined && value > schema.maximum) {
|
||||||
|
errors.push(`${key}: value ${value} is greater than maximum ${schema.maximum}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// String validations
|
||||||
|
if (schema.type === 'string') {
|
||||||
|
if (schema.minLength !== undefined && value.length < schema.minLength) {
|
||||||
|
errors.push(`${key}: length ${value.length} is less than minimum ${schema.minLength}`);
|
||||||
|
}
|
||||||
|
if (schema.maxLength !== undefined && value.length > schema.maxLength) {
|
||||||
|
errors.push(`${key}: length ${value.length} is greater than maximum ${schema.maxLength}`);
|
||||||
|
}
|
||||||
|
if (schema.pattern) {
|
||||||
|
const regex = new RegExp(schema.pattern);
|
||||||
|
if (!regex.test(value)) {
|
||||||
|
errors.push(`${key}: value does not match pattern ${schema.pattern}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Enum validation
|
||||||
|
if (schema.enum && !schema.enum.includes(value)) {
|
||||||
|
errors.push(`${key}: value ${value} is not one of allowed values: ${schema.enum.join(', ')}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get configuration sources for debugging
|
||||||
|
*/
|
||||||
|
getSources() {
|
||||||
|
return [...this.sources];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get resolved configuration
|
||||||
|
*/
|
||||||
|
getResolved() {
|
||||||
|
return { ...this.resolved };
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Update configuration at runtime
|
||||||
|
*/
|
||||||
|
updateRuntime(config) {
|
||||||
|
// Add or update runtime source
|
||||||
|
const runtimeIndex = this.sources.findIndex(s => s.priority === ConfigPriority.RUNTIME);
|
||||||
|
if (runtimeIndex >= 0) {
|
||||||
|
this.sources[runtimeIndex].config = {
|
||||||
|
...this.sources[runtimeIndex].config,
|
||||||
|
...config
|
||||||
|
};
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
this.sources.push({
|
||||||
|
priority: ConfigPriority.RUNTIME,
|
||||||
|
source: 'runtime',
|
||||||
|
config
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// Re-merge configurations
|
||||||
|
this.resolved = this.mergeConfigurations();
|
||||||
|
// Validate
|
||||||
|
if (this.options.schema) {
|
||||||
|
this.validateConfiguration(this.resolved);
|
||||||
|
}
|
||||||
|
return this.resolved;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Save configuration to file
|
||||||
|
* @param filepath Path to save configuration
|
||||||
|
* @param format Format to save as (json, etc.)
|
||||||
|
*/
|
||||||
|
async saveToFile(filepath, format = 'json') {
|
||||||
|
// Skip in browser environment
|
||||||
|
if (typeof process === 'undefined' || typeof window !== 'undefined') {
|
||||||
|
throw new Error('Cannot save configuration files in browser environment');
|
||||||
|
}
|
||||||
|
const fs = await import('fs');
|
||||||
|
const path = await import('path');
|
||||||
|
const configPath = filepath || this.options.configPaths?.[0] || '.brainyrc';
|
||||||
|
const augId = this.options.augmentationId;
|
||||||
|
// Load existing config if it exists
|
||||||
|
let fullConfig = {};
|
||||||
|
try {
|
||||||
|
if (fs.existsSync(configPath)) {
|
||||||
|
const content = fs.readFileSync(configPath, 'utf8');
|
||||||
|
fullConfig = JSON.parse(content);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
// Start with empty config
|
||||||
|
}
|
||||||
|
// Ensure augmentations section exists
|
||||||
|
if (!fullConfig.augmentations) {
|
||||||
|
fullConfig.augmentations = {};
|
||||||
|
}
|
||||||
|
// Update augmentation config
|
||||||
|
fullConfig.augmentations[augId] = this.resolved;
|
||||||
|
// Save based on format
|
||||||
|
let content;
|
||||||
|
if (format === 'json') {
|
||||||
|
content = JSON.stringify(fullConfig, null, 2);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
throw new Error(`Unsupported format: ${format}`);
|
||||||
|
}
|
||||||
|
// Ensure directory exists
|
||||||
|
const dir = path.dirname(configPath);
|
||||||
|
if (!fs.existsSync(dir)) {
|
||||||
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
|
}
|
||||||
|
// Write file
|
||||||
|
fs.writeFileSync(configPath, content, 'utf8');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get environment variable names for this augmentation
|
||||||
|
*/
|
||||||
|
getEnvironmentVariables() {
|
||||||
|
const schema = this.options.schema;
|
||||||
|
const prefix = this.options.envPrefix;
|
||||||
|
const vars = {};
|
||||||
|
if (schema?.properties) {
|
||||||
|
for (const [key, prop] of Object.entries(schema.properties)) {
|
||||||
|
const envKey = prefix + key.replace(/([A-Z])/g, '_$1').toUpperCase();
|
||||||
|
vars[envKey] = {
|
||||||
|
description: prop.description,
|
||||||
|
type: prop.type,
|
||||||
|
default: prop.default,
|
||||||
|
currentValue: process.env?.[envKey]
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return vars;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=configResolver.js.map
|
||||||
File diff suppressed because one or more lines are too long
65
.recovery-workspace/dist-backup-20250910-141917/augmentations/connectionPoolAugmentation.d.ts
vendored
Normal file
65
.recovery-workspace/dist-backup-20250910-141917/augmentations/connectionPoolAugmentation.d.ts
vendored
Normal file
|
|
@ -0,0 +1,65 @@
|
||||||
|
/**
|
||||||
|
* Connection Pool Augmentation
|
||||||
|
*
|
||||||
|
* Provides 10-20x throughput improvement for cloud storage (S3, R2, GCS)
|
||||||
|
* Manages connection pooling, request queuing, and parallel processing
|
||||||
|
* Critical for enterprise-scale operations with millions of entries
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
interface ConnectionPoolConfig {
|
||||||
|
enabled?: boolean;
|
||||||
|
maxConnections?: number;
|
||||||
|
minConnections?: number;
|
||||||
|
acquireTimeout?: number;
|
||||||
|
idleTimeout?: number;
|
||||||
|
maxQueueSize?: number;
|
||||||
|
retryAttempts?: number;
|
||||||
|
healthCheckInterval?: number;
|
||||||
|
}
|
||||||
|
export declare class ConnectionPoolAugmentation extends BaseAugmentation {
|
||||||
|
name: string;
|
||||||
|
timing: "around";
|
||||||
|
metadata: "none";
|
||||||
|
operations: ("storage")[];
|
||||||
|
priority: number;
|
||||||
|
protected config: Required<ConnectionPoolConfig>;
|
||||||
|
private connections;
|
||||||
|
private requestQueue;
|
||||||
|
private healthCheckInterval?;
|
||||||
|
private storageType;
|
||||||
|
private stats;
|
||||||
|
constructor(config?: ConnectionPoolConfig);
|
||||||
|
protected onInitialize(): Promise<void>;
|
||||||
|
shouldExecute(operation: string, params: any): boolean;
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
private detectStorageType;
|
||||||
|
private isCloudStorage;
|
||||||
|
private isStorageOperation;
|
||||||
|
private getOperationPriority;
|
||||||
|
private executeWithPool;
|
||||||
|
private getAvailableConnection;
|
||||||
|
private executeWithConnection;
|
||||||
|
private queueRequest;
|
||||||
|
private processQueue;
|
||||||
|
private initializeConnectionPool;
|
||||||
|
private createConnection;
|
||||||
|
private createStorageConnection;
|
||||||
|
private getOrCreateConnection;
|
||||||
|
private startHealthChecks;
|
||||||
|
private performHealthChecks;
|
||||||
|
private startMetricsCollection;
|
||||||
|
private updateLatencyMetrics;
|
||||||
|
private updateThroughputMetrics;
|
||||||
|
/**
|
||||||
|
* Get connection pool statistics
|
||||||
|
*/
|
||||||
|
getStats(): typeof this.stats & {
|
||||||
|
queueSize: number;
|
||||||
|
activeConnections: number;
|
||||||
|
totalConnections: number;
|
||||||
|
poolUtilization: string;
|
||||||
|
storageType: string;
|
||||||
|
};
|
||||||
|
protected onShutdown(): Promise<void>;
|
||||||
|
}
|
||||||
|
export {};
|
||||||
|
|
@ -0,0 +1,342 @@
|
||||||
|
/**
|
||||||
|
* Connection Pool Augmentation
|
||||||
|
*
|
||||||
|
* Provides 10-20x throughput improvement for cloud storage (S3, R2, GCS)
|
||||||
|
* Manages connection pooling, request queuing, and parallel processing
|
||||||
|
* Critical for enterprise-scale operations with millions of entries
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
export class ConnectionPoolAugmentation extends BaseAugmentation {
|
||||||
|
constructor(config = {}) {
|
||||||
|
super();
|
||||||
|
this.name = 'ConnectionPool';
|
||||||
|
this.timing = 'around';
|
||||||
|
this.metadata = 'none'; // Connection pooling doesn't access metadata
|
||||||
|
this.operations = ['storage'];
|
||||||
|
this.priority = 95; // Very high priority for storage operations
|
||||||
|
this.connections = new Map();
|
||||||
|
this.requestQueue = [];
|
||||||
|
this.storageType = 'unknown';
|
||||||
|
this.stats = {
|
||||||
|
totalRequests: 0,
|
||||||
|
queuedRequests: 0,
|
||||||
|
activeConnections: 0,
|
||||||
|
totalConnections: 0,
|
||||||
|
averageLatency: 0,
|
||||||
|
throughputPerSecond: 0
|
||||||
|
};
|
||||||
|
this.config = {
|
||||||
|
enabled: config.enabled ?? true,
|
||||||
|
maxConnections: config.maxConnections ?? 50,
|
||||||
|
minConnections: config.minConnections ?? 5,
|
||||||
|
acquireTimeout: config.acquireTimeout ?? 30000, // 30s
|
||||||
|
idleTimeout: config.idleTimeout ?? 300000, // 5 minutes
|
||||||
|
maxQueueSize: config.maxQueueSize ?? 10000,
|
||||||
|
retryAttempts: config.retryAttempts ?? 3,
|
||||||
|
healthCheckInterval: config.healthCheckInterval ?? 60000 // 1 minute
|
||||||
|
};
|
||||||
|
}
|
||||||
|
async onInitialize() {
|
||||||
|
if (!this.config.enabled) {
|
||||||
|
this.log('Connection pooling disabled');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Detect storage type
|
||||||
|
this.storageType = this.detectStorageType();
|
||||||
|
if (this.isCloudStorage()) {
|
||||||
|
await this.initializeConnectionPool();
|
||||||
|
this.startHealthChecks();
|
||||||
|
this.startMetricsCollection();
|
||||||
|
this.log(`Connection pool initialized for ${this.storageType}: ${this.config.minConnections}-${this.config.maxConnections} connections`);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
this.log(`Connection pooling skipped for ${this.storageType} (local storage)`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
shouldExecute(operation, params) {
|
||||||
|
return this.config.enabled &&
|
||||||
|
this.isCloudStorage() &&
|
||||||
|
this.isStorageOperation(operation);
|
||||||
|
}
|
||||||
|
async execute(operation, params, next) {
|
||||||
|
if (!this.shouldExecute(operation, params)) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
const startTime = Date.now();
|
||||||
|
this.stats.totalRequests++;
|
||||||
|
try {
|
||||||
|
// High priority for critical operations
|
||||||
|
const priority = this.getOperationPriority(operation);
|
||||||
|
// Execute with pooled connection
|
||||||
|
const result = await this.executeWithPool(operation, params, next, priority);
|
||||||
|
// Update metrics
|
||||||
|
const latency = Date.now() - startTime;
|
||||||
|
this.updateLatencyMetrics(latency);
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log(`Connection pool error for ${operation}: ${error}`, 'error');
|
||||||
|
// Fallback to direct execution for reliability
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
detectStorageType() {
|
||||||
|
const storage = this.context?.storage;
|
||||||
|
if (!storage)
|
||||||
|
return 'unknown';
|
||||||
|
const className = storage.constructor.name.toLowerCase();
|
||||||
|
if (className.includes('s3'))
|
||||||
|
return 's3';
|
||||||
|
if (className.includes('r2'))
|
||||||
|
return 'r2';
|
||||||
|
if (className.includes('gcs') || className.includes('google'))
|
||||||
|
return 'gcs';
|
||||||
|
if (className.includes('azure'))
|
||||||
|
return 'azure';
|
||||||
|
if (className.includes('filesystem'))
|
||||||
|
return 'filesystem';
|
||||||
|
if (className.includes('memory'))
|
||||||
|
return 'memory';
|
||||||
|
return 'unknown';
|
||||||
|
}
|
||||||
|
isCloudStorage() {
|
||||||
|
return ['s3', 'r2', 'gcs', 'azure'].includes(this.storageType);
|
||||||
|
}
|
||||||
|
isStorageOperation(operation) {
|
||||||
|
return operation.includes('save') ||
|
||||||
|
operation.includes('get') ||
|
||||||
|
operation.includes('delete') ||
|
||||||
|
operation.includes('list') ||
|
||||||
|
operation.includes('backup') ||
|
||||||
|
operation.includes('restore');
|
||||||
|
}
|
||||||
|
getOperationPriority(operation) {
|
||||||
|
// Critical operations get highest priority
|
||||||
|
if (operation.includes('save') || operation.includes('update'))
|
||||||
|
return 10;
|
||||||
|
if (operation.includes('delete'))
|
||||||
|
return 9;
|
||||||
|
if (operation.includes('get'))
|
||||||
|
return 7;
|
||||||
|
if (operation.includes('list'))
|
||||||
|
return 5;
|
||||||
|
if (operation.includes('backup'))
|
||||||
|
return 3;
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
async executeWithPool(operation, params, executor, priority) {
|
||||||
|
// Check queue size
|
||||||
|
if (this.requestQueue.length >= this.config.maxQueueSize) {
|
||||||
|
throw new Error('Connection pool queue full - system overloaded');
|
||||||
|
}
|
||||||
|
// Try to get available connection immediately
|
||||||
|
const connection = await this.getOrCreateConnection();
|
||||||
|
if (connection && connection.isIdle) {
|
||||||
|
return this.executeWithConnection(connection, operation, executor);
|
||||||
|
}
|
||||||
|
// Queue the request with the actual executor
|
||||||
|
return this.queueRequest(operation, params, executor, priority);
|
||||||
|
}
|
||||||
|
getAvailableConnection() {
|
||||||
|
// Find idle connection with best health score
|
||||||
|
let bestConnection = null;
|
||||||
|
let bestScore = -1;
|
||||||
|
for (const connection of this.connections.values()) {
|
||||||
|
if (connection.isIdle && connection.healthScore > bestScore) {
|
||||||
|
bestConnection = connection;
|
||||||
|
bestScore = connection.healthScore;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return bestConnection;
|
||||||
|
}
|
||||||
|
async executeWithConnection(connection, operation, executor) {
|
||||||
|
// Mark connection as active
|
||||||
|
connection.isIdle = false;
|
||||||
|
connection.activeRequests++;
|
||||||
|
connection.lastUsed = Date.now();
|
||||||
|
this.stats.activeConnections++;
|
||||||
|
try {
|
||||||
|
const result = await executor();
|
||||||
|
// Update connection health on success
|
||||||
|
connection.healthScore = Math.min(connection.healthScore + 1, 100);
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
// Decrease health on failure
|
||||||
|
connection.healthScore = Math.max(connection.healthScore - 5, 0);
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
finally {
|
||||||
|
// Release connection
|
||||||
|
connection.isIdle = true;
|
||||||
|
connection.activeRequests--;
|
||||||
|
this.stats.activeConnections--;
|
||||||
|
// Process next queued request
|
||||||
|
this.processQueue();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async queueRequest(operation, params, executor, priority) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const request = {
|
||||||
|
id: `req_${Date.now()}_${Math.random()}`,
|
||||||
|
operation,
|
||||||
|
params,
|
||||||
|
executor, // Store the actual executor function
|
||||||
|
resolver: resolve,
|
||||||
|
rejector: reject,
|
||||||
|
timestamp: Date.now(),
|
||||||
|
priority
|
||||||
|
};
|
||||||
|
// Insert by priority (higher priority first)
|
||||||
|
const insertIndex = this.requestQueue.findIndex(r => r.priority < priority);
|
||||||
|
if (insertIndex === -1) {
|
||||||
|
this.requestQueue.push(request);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
this.requestQueue.splice(insertIndex, 0, request);
|
||||||
|
}
|
||||||
|
this.stats.queuedRequests++;
|
||||||
|
// Set timeout
|
||||||
|
setTimeout(() => {
|
||||||
|
const index = this.requestQueue.findIndex(r => r.id === request.id);
|
||||||
|
if (index !== -1) {
|
||||||
|
this.requestQueue.splice(index, 1);
|
||||||
|
this.stats.queuedRequests--;
|
||||||
|
reject(new Error(`Connection pool timeout: ${this.config.acquireTimeout}ms`));
|
||||||
|
}
|
||||||
|
}, this.config.acquireTimeout);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
processQueue() {
|
||||||
|
if (this.requestQueue.length === 0)
|
||||||
|
return;
|
||||||
|
const connection = this.getAvailableConnection();
|
||||||
|
if (!connection)
|
||||||
|
return;
|
||||||
|
const request = this.requestQueue.shift();
|
||||||
|
this.stats.queuedRequests--;
|
||||||
|
// Execute queued request with the REAL executor
|
||||||
|
this.executeWithConnection(connection, request.operation, request.executor)
|
||||||
|
.then(request.resolver)
|
||||||
|
.catch(request.rejector);
|
||||||
|
}
|
||||||
|
async initializeConnectionPool() {
|
||||||
|
// Create minimum connections
|
||||||
|
for (let i = 0; i < this.config.minConnections; i++) {
|
||||||
|
await this.createConnection();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async createConnection() {
|
||||||
|
const connectionId = `conn_${Date.now()}_${Math.random()}`;
|
||||||
|
// Create actual connection based on storage type
|
||||||
|
const actualConnection = await this.createStorageConnection();
|
||||||
|
const connection = {
|
||||||
|
id: connectionId,
|
||||||
|
connection: actualConnection,
|
||||||
|
isIdle: true,
|
||||||
|
lastUsed: Date.now(),
|
||||||
|
healthScore: 100,
|
||||||
|
activeRequests: 0,
|
||||||
|
requestCount: 0
|
||||||
|
};
|
||||||
|
this.connections.set(connectionId, connection);
|
||||||
|
this.stats.totalConnections++;
|
||||||
|
return connection;
|
||||||
|
}
|
||||||
|
async createStorageConnection() {
|
||||||
|
// For cloud storage, reuse the existing storage instance
|
||||||
|
// Connection pooling in this context means managing concurrent requests
|
||||||
|
// not creating multiple storage instances (which would be wasteful)
|
||||||
|
const storage = this.context?.storage;
|
||||||
|
if (!storage) {
|
||||||
|
throw new Error('Storage not available for connection pooling');
|
||||||
|
}
|
||||||
|
// Return a connection wrapper that tracks usage
|
||||||
|
return {
|
||||||
|
storage,
|
||||||
|
created: Date.now(),
|
||||||
|
requestCount: 0
|
||||||
|
};
|
||||||
|
}
|
||||||
|
async getOrCreateConnection() {
|
||||||
|
// Try to get an available connection
|
||||||
|
let connection = this.getAvailableConnection();
|
||||||
|
// If no connection available and under max, create new one
|
||||||
|
if (!connection && this.connections.size < this.config.maxConnections) {
|
||||||
|
connection = await this.createConnection();
|
||||||
|
}
|
||||||
|
return connection;
|
||||||
|
}
|
||||||
|
startHealthChecks() {
|
||||||
|
this.healthCheckInterval = setInterval(() => {
|
||||||
|
this.performHealthChecks();
|
||||||
|
}, this.config.healthCheckInterval);
|
||||||
|
}
|
||||||
|
performHealthChecks() {
|
||||||
|
const now = Date.now();
|
||||||
|
const toRemove = [];
|
||||||
|
for (const [id, connection] of this.connections) {
|
||||||
|
// Remove idle connections that are too old
|
||||||
|
if (connection.isIdle &&
|
||||||
|
now - connection.lastUsed > this.config.idleTimeout &&
|
||||||
|
this.connections.size > this.config.minConnections) {
|
||||||
|
toRemove.push(id);
|
||||||
|
}
|
||||||
|
// Remove unhealthy connections
|
||||||
|
if (connection.healthScore < 20) {
|
||||||
|
toRemove.push(id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Remove unhealthy/old connections
|
||||||
|
for (const id of toRemove) {
|
||||||
|
this.connections.delete(id);
|
||||||
|
this.stats.totalConnections--;
|
||||||
|
}
|
||||||
|
// Ensure minimum connections
|
||||||
|
while (this.connections.size < this.config.minConnections) {
|
||||||
|
this.createConnection();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
startMetricsCollection() {
|
||||||
|
setInterval(() => {
|
||||||
|
this.updateThroughputMetrics();
|
||||||
|
}, 1000); // Update every second
|
||||||
|
}
|
||||||
|
updateLatencyMetrics(latency) {
|
||||||
|
// Simple moving average
|
||||||
|
this.stats.averageLatency = (this.stats.averageLatency * 0.9) + (latency * 0.1);
|
||||||
|
}
|
||||||
|
updateThroughputMetrics() {
|
||||||
|
// Reset counter for next second
|
||||||
|
this.stats.throughputPerSecond = this.stats.totalRequests;
|
||||||
|
// Reset total for next measurement (in practice, use sliding window)
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get connection pool statistics
|
||||||
|
*/
|
||||||
|
getStats() {
|
||||||
|
return {
|
||||||
|
...this.stats,
|
||||||
|
queueSize: this.requestQueue.length,
|
||||||
|
activeConnections: this.stats.activeConnections,
|
||||||
|
totalConnections: this.connections.size,
|
||||||
|
poolUtilization: `${Math.round((this.stats.activeConnections / this.connections.size) * 100)}%`,
|
||||||
|
storageType: this.storageType
|
||||||
|
};
|
||||||
|
}
|
||||||
|
async onShutdown() {
|
||||||
|
if (this.healthCheckInterval) {
|
||||||
|
clearInterval(this.healthCheckInterval);
|
||||||
|
}
|
||||||
|
// Close all connections
|
||||||
|
this.connections.clear();
|
||||||
|
// Reject all queued requests
|
||||||
|
this.requestQueue.forEach(request => {
|
||||||
|
request.rejector(new Error('Connection pool shutting down'));
|
||||||
|
});
|
||||||
|
this.requestQueue = [];
|
||||||
|
const stats = this.getStats();
|
||||||
|
this.log(`Connection pool shutdown: ${stats.totalRequests} requests processed, ${stats.poolUtilization} avg utilization`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=connectionPoolAugmentation.js.map
|
||||||
File diff suppressed because one or more lines are too long
59
.recovery-workspace/dist-backup-20250910-141917/augmentations/defaultAugmentations.d.ts
vendored
Normal file
59
.recovery-workspace/dist-backup-20250910-141917/augmentations/defaultAugmentations.d.ts
vendored
Normal file
|
|
@ -0,0 +1,59 @@
|
||||||
|
/**
|
||||||
|
* Default Augmentations Registration
|
||||||
|
*
|
||||||
|
* Maintains zero-config philosophy by automatically registering
|
||||||
|
* core augmentations that were previously hardcoded in Brainy.
|
||||||
|
*
|
||||||
|
* These augmentations are optional but enabled by default for
|
||||||
|
* backward compatibility and optimal performance.
|
||||||
|
*/
|
||||||
|
import { Brainy } from '../brainy.js';
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
import { CacheAugmentation } from './cacheAugmentation.js';
|
||||||
|
import { IndexAugmentation } from './indexAugmentation.js';
|
||||||
|
import { MetricsAugmentation } from './metricsAugmentation.js';
|
||||||
|
import { MonitoringAugmentation } from './monitoringAugmentation.js';
|
||||||
|
import { UniversalDisplayAugmentation } from './universalDisplayAugmentation.js';
|
||||||
|
/**
|
||||||
|
* Create default augmentations for zero-config operation
|
||||||
|
* Returns an array of augmentations to be registered
|
||||||
|
*
|
||||||
|
* @param config - Configuration options
|
||||||
|
* @returns Array of augmentations to register
|
||||||
|
*/
|
||||||
|
export declare function createDefaultAugmentations(config?: {
|
||||||
|
cache?: boolean | Record<string, any>;
|
||||||
|
index?: boolean | Record<string, any>;
|
||||||
|
metrics?: boolean | Record<string, any>;
|
||||||
|
monitoring?: boolean | Record<string, any>;
|
||||||
|
display?: boolean | Record<string, any>;
|
||||||
|
}): BaseAugmentation[];
|
||||||
|
/**
|
||||||
|
* Get augmentation by name with type safety
|
||||||
|
*/
|
||||||
|
export declare function getAugmentation<T>(brain: Brainy, name: string): T | null;
|
||||||
|
/**
|
||||||
|
* Compatibility helpers for migrating from hardcoded features
|
||||||
|
*/
|
||||||
|
export declare const AugmentationHelpers: {
|
||||||
|
/**
|
||||||
|
* Get cache augmentation
|
||||||
|
*/
|
||||||
|
getCache(brain: Brainy): CacheAugmentation | null;
|
||||||
|
/**
|
||||||
|
* Get index augmentation
|
||||||
|
*/
|
||||||
|
getIndex(brain: Brainy): IndexAugmentation | null;
|
||||||
|
/**
|
||||||
|
* Get metrics augmentation
|
||||||
|
*/
|
||||||
|
getMetrics(brain: Brainy): MetricsAugmentation | null;
|
||||||
|
/**
|
||||||
|
* Get monitoring augmentation
|
||||||
|
*/
|
||||||
|
getMonitoring(brain: Brainy): MonitoringAugmentation | null;
|
||||||
|
/**
|
||||||
|
* Get display augmentation
|
||||||
|
*/
|
||||||
|
getDisplay(brain: Brainy): UniversalDisplayAugmentation | null;
|
||||||
|
};
|
||||||
|
|
@ -0,0 +1,100 @@
|
||||||
|
/**
|
||||||
|
* Default Augmentations Registration
|
||||||
|
*
|
||||||
|
* Maintains zero-config philosophy by automatically registering
|
||||||
|
* core augmentations that were previously hardcoded in Brainy.
|
||||||
|
*
|
||||||
|
* These augmentations are optional but enabled by default for
|
||||||
|
* backward compatibility and optimal performance.
|
||||||
|
*/
|
||||||
|
import { CacheAugmentation } from './cacheAugmentation.js';
|
||||||
|
import { IndexAugmentation } from './indexAugmentation.js';
|
||||||
|
import { MetricsAugmentation } from './metricsAugmentation.js';
|
||||||
|
import { MonitoringAugmentation } from './monitoringAugmentation.js';
|
||||||
|
import { UniversalDisplayAugmentation } from './universalDisplayAugmentation.js';
|
||||||
|
/**
|
||||||
|
* Create default augmentations for zero-config operation
|
||||||
|
* Returns an array of augmentations to be registered
|
||||||
|
*
|
||||||
|
* @param config - Configuration options
|
||||||
|
* @returns Array of augmentations to register
|
||||||
|
*/
|
||||||
|
export function createDefaultAugmentations(config = {}) {
|
||||||
|
const augmentations = [];
|
||||||
|
// Cache augmentation (was SearchCache)
|
||||||
|
if (config.cache !== false) {
|
||||||
|
const cacheConfig = typeof config.cache === 'object' ? config.cache : {};
|
||||||
|
augmentations.push(new CacheAugmentation(cacheConfig));
|
||||||
|
}
|
||||||
|
// Index augmentation (was MetadataIndex)
|
||||||
|
if (config.index !== false) {
|
||||||
|
const indexConfig = typeof config.index === 'object' ? config.index : {};
|
||||||
|
augmentations.push(new IndexAugmentation(indexConfig));
|
||||||
|
}
|
||||||
|
// Metrics augmentation (was StatisticsCollector)
|
||||||
|
if (config.metrics !== false) {
|
||||||
|
const metricsConfig = typeof config.metrics === 'object' ? config.metrics : {};
|
||||||
|
augmentations.push(new MetricsAugmentation(metricsConfig));
|
||||||
|
}
|
||||||
|
// Display augmentation (AI-powered intelligent display fields)
|
||||||
|
if (config.display !== false) {
|
||||||
|
const displayConfig = typeof config.display === 'object' ? config.display : {};
|
||||||
|
augmentations.push(new UniversalDisplayAugmentation(displayConfig));
|
||||||
|
}
|
||||||
|
// Monitoring augmentation (was HealthMonitor)
|
||||||
|
// Only enable by default in distributed mode
|
||||||
|
const isDistributed = process.env.BRAINY_MODE === 'distributed' ||
|
||||||
|
process.env.BRAINY_DISTRIBUTED === 'true';
|
||||||
|
if (config.monitoring !== false && (config.monitoring || isDistributed)) {
|
||||||
|
const monitoringConfig = typeof config.monitoring === 'object' ? config.monitoring : {};
|
||||||
|
augmentations.push(new MonitoringAugmentation(monitoringConfig));
|
||||||
|
}
|
||||||
|
return augmentations;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get augmentation by name with type safety
|
||||||
|
*/
|
||||||
|
export function getAugmentation(brain, name) {
|
||||||
|
// Access augmentations through a public method or property
|
||||||
|
const augmentations = brain.augmentations;
|
||||||
|
if (!augmentations)
|
||||||
|
return null;
|
||||||
|
const aug = augmentations.get(name);
|
||||||
|
return aug;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Compatibility helpers for migrating from hardcoded features
|
||||||
|
*/
|
||||||
|
export const AugmentationHelpers = {
|
||||||
|
/**
|
||||||
|
* Get cache augmentation
|
||||||
|
*/
|
||||||
|
getCache(brain) {
|
||||||
|
return getAugmentation(brain, 'cache');
|
||||||
|
},
|
||||||
|
/**
|
||||||
|
* Get index augmentation
|
||||||
|
*/
|
||||||
|
getIndex(brain) {
|
||||||
|
return getAugmentation(brain, 'index');
|
||||||
|
},
|
||||||
|
/**
|
||||||
|
* Get metrics augmentation
|
||||||
|
*/
|
||||||
|
getMetrics(brain) {
|
||||||
|
return getAugmentation(brain, 'metrics');
|
||||||
|
},
|
||||||
|
/**
|
||||||
|
* Get monitoring augmentation
|
||||||
|
*/
|
||||||
|
getMonitoring(brain) {
|
||||||
|
return getAugmentation(brain, 'monitoring');
|
||||||
|
},
|
||||||
|
/**
|
||||||
|
* Get display augmentation
|
||||||
|
*/
|
||||||
|
getDisplay(brain) {
|
||||||
|
return getAugmentation(brain, 'display');
|
||||||
|
}
|
||||||
|
};
|
||||||
|
//# sourceMappingURL=defaultAugmentations.js.map
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
{"version":3,"file":"defaultAugmentations.js","sourceRoot":"","sources":["../../src/augmentations/defaultAugmentations.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAIH,OAAO,EAAE,iBAAiB,EAAE,MAAM,wBAAwB,CAAA;AAC1D,OAAO,EAAE,iBAAiB,EAAE,MAAM,wBAAwB,CAAA;AAC1D,OAAO,EAAE,mBAAmB,EAAE,MAAM,0BAA0B,CAAA;AAC9D,OAAO,EAAE,sBAAsB,EAAE,MAAM,6BAA6B,CAAA;AACpE,OAAO,EAAE,4BAA4B,EAAE,MAAM,mCAAmC,CAAA;AAEhF;;;;;;GAMG;AACH,MAAM,UAAU,0BAA0B,CACxC,SAMI,EAAE;IAEN,MAAM,aAAa,GAAuB,EAAE,CAAA;IAE5C,uCAAuC;IACvC,IAAI,MAAM,CAAC,KAAK,KAAK,KAAK,EAAE,CAAC;QAC3B,MAAM,WAAW,GAAG,OAAO,MAAM,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAA;QACxE,aAAa,CAAC,IAAI,CAAC,IAAI,iBAAiB,CAAC,WAAW,CAAC,CAAC,CAAA;IACxD,CAAC;IAED,yCAAyC;IACzC,IAAI,MAAM,CAAC,KAAK,KAAK,KAAK,EAAE,CAAC;QAC3B,MAAM,WAAW,GAAG,OAAO,MAAM,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAA;QACxE,aAAa,CAAC,IAAI,CAAC,IAAI,iBAAiB,CAAC,WAAW,CAAC,CAAC,CAAA;IACxD,CAAC;IAED,iDAAiD;IACjD,IAAI,MAAM,CAAC,OAAO,KAAK,KAAK,EAAE,CAAC;QAC7B,MAAM,aAAa,GAAG,OAAO,MAAM,CAAC,OAAO,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAA;QAC9E,aAAa,CAAC,IAAI,CAAC,IAAI,mBAAmB,CAAC,aAAa,CAAC,CAAC,CAAA;IAC5D,CAAC;IAED,+DAA+D;IAC/D,IAAI,MAAM,CAAC,OAAO,KAAK,KAAK,EAAE,CAAC;QAC7B,MAAM,aAAa,GAAG,OAAO,MAAM,CAAC,OAAO,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAA;QAC9E,aAAa,CAAC,IAAI,CAAC,IAAI,4BAA4B,CAAC,aAAa,CAAC,CAAC,CAAA;IACrE,CAAC;IAED,8CAA8C;IAC9C,6CAA6C;IAC7C,MAAM,aAAa,GAAG,OAAO,CAAC,GAAG,CAAC,WAAW,KAAK,aAAa;QACzC,OAAO,CAAC,GAAG,CAAC,kBAAkB,KAAK,MAAM,CAAA;IAE/D,IAAI,MAAM,CAAC,UAAU,KAAK,KAAK,IAAI,CAAC,MAAM,CAAC,UAAU,IAAI,aAAa,CAAC,EAAE,CAAC;QACxE,MAAM,gBAAgB,GAAG,OAAO,MAAM,CAAC,UAAU,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,CAAA;QACvF,aAAa,CAAC,IAAI,CAAC,IAAI,sBAAsB,CAAC,gBAAgB,CAAC,CAAC,CAAA;IAClE,CAAC;IAED,OAAO,aAAa,CAAA;AACtB,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,eAAe,CAAI,KAAa,EAAE,IAAY;IAC5D,2DAA2D;IAC3D,MAAM,aAAa,GAAI,KAAa,CAAC,aAAa,CAAA;IAClD,IAAI,CAAC,aAAa;QAAE,OAAO,IAAI,CAAA;IAC/B,MAAM,GAAG,GAAG,aAAa,CAAC,GAAG,CAAC,IAAI,CAAC,CAAA;IACnC,OAAO,GAAe,CAAA;AACxB,CAAC;AAED;;GAEG;AACH,MAAM,CAAC,MAAM,mBAAmB,GAAG;IACjC;;OAEG;IACH,QAAQ,CAAC,KAAa;QACpB,OAAO,eAAe,CAAoB,KAAK,EAAE,OAAO,CAAC,CAAA;IAC3D,CAAC;IAED;;OAEG;IACH,QAAQ,CAAC,KAAa;QACpB,OAAO,eAAe,CAAoB,KAAK,EAAE,OAAO,CAAC,CAAA;IAC3D,CAAC;IAED;;OAEG;IACH,UAAU,CAAC,KAAa;QACtB,OAAO,eAAe,CAAsB,KAAK,EAAE,SAAS,CAAC,CAAA;IAC/D,CAAC;IAED;;OAEG;IACH,aAAa,CAAC,KAAa;QACzB,OAAO,eAAe,CAAyB,KAAK,EAAE,YAAY,CAAC,CAAA;IACrE,CAAC;IAED;;OAEG;IACH,UAAU,CAAC,KAAa;QACtB,OAAO,eAAe,CAA+B,KAAK,EAAE,SAAS,CAAC,CAAA;IACxE,CAAC;CACF,CAAA"}
|
||||||
152
.recovery-workspace/dist-backup-20250910-141917/augmentations/discovery.d.ts
vendored
Normal file
152
.recovery-workspace/dist-backup-20250910-141917/augmentations/discovery.d.ts
vendored
Normal file
|
|
@ -0,0 +1,152 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Discovery API
|
||||||
|
*
|
||||||
|
* Provides discovery and configuration capabilities for augmentations
|
||||||
|
* Enables tools like brain-cloud to dynamically discover, configure, and manage augmentations
|
||||||
|
*/
|
||||||
|
import { AugmentationRegistry } from './brainyAugmentation.js';
|
||||||
|
import { AugmentationManifest, JSONSchema } from './manifest.js';
|
||||||
|
/**
|
||||||
|
* Augmentation listing with manifest and status
|
||||||
|
*/
|
||||||
|
export interface AugmentationListing {
|
||||||
|
id: string;
|
||||||
|
name: string;
|
||||||
|
manifest: AugmentationManifest;
|
||||||
|
status: {
|
||||||
|
enabled: boolean;
|
||||||
|
initialized: boolean;
|
||||||
|
category: string;
|
||||||
|
priority: number;
|
||||||
|
};
|
||||||
|
config?: {
|
||||||
|
current: any;
|
||||||
|
schema?: JSONSchema;
|
||||||
|
sources?: any[];
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Configuration validation result
|
||||||
|
*/
|
||||||
|
export interface ConfigValidationResult {
|
||||||
|
valid: boolean;
|
||||||
|
errors?: string[];
|
||||||
|
warnings?: string[];
|
||||||
|
suggestions?: string[];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Discovery API options
|
||||||
|
*/
|
||||||
|
export interface DiscoveryOptions {
|
||||||
|
includeConfig?: boolean;
|
||||||
|
includeSchema?: boolean;
|
||||||
|
includeSources?: boolean;
|
||||||
|
category?: string;
|
||||||
|
enabled?: boolean;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Augmentation Discovery API
|
||||||
|
*
|
||||||
|
* Provides a unified interface for discovering and managing augmentations
|
||||||
|
*/
|
||||||
|
export declare class AugmentationDiscovery {
|
||||||
|
private registry;
|
||||||
|
constructor(registry: AugmentationRegistry);
|
||||||
|
/**
|
||||||
|
* Discover all registered augmentations with manifests
|
||||||
|
* @param options Discovery options
|
||||||
|
* @returns List of augmentation listings
|
||||||
|
*/
|
||||||
|
discover(options?: DiscoveryOptions): Promise<AugmentationListing[]>;
|
||||||
|
/**
|
||||||
|
* Get a specific augmentation's manifest
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Augmentation manifest or null if not found
|
||||||
|
*/
|
||||||
|
getManifest(augId: string): Promise<AugmentationManifest | null>;
|
||||||
|
/**
|
||||||
|
* Get configuration schema for an augmentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Configuration schema or null
|
||||||
|
*/
|
||||||
|
getConfigSchema(augId: string): Promise<JSONSchema | null>;
|
||||||
|
/**
|
||||||
|
* Get current configuration for an augmentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Current configuration or null
|
||||||
|
*/
|
||||||
|
getConfig(augId: string): Promise<any | null>;
|
||||||
|
/**
|
||||||
|
* Update configuration for an augmentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @param config New configuration
|
||||||
|
* @returns Updated configuration or null on failure
|
||||||
|
*/
|
||||||
|
updateConfig(augId: string, config: any): Promise<any | null>;
|
||||||
|
/**
|
||||||
|
* Validate configuration against schema
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @param config Configuration to validate
|
||||||
|
* @returns Validation result
|
||||||
|
*/
|
||||||
|
validateConfig(augId: string, config: any): Promise<ConfigValidationResult>;
|
||||||
|
/**
|
||||||
|
* Validate a property value against its schema
|
||||||
|
*/
|
||||||
|
private validatePropertyValue;
|
||||||
|
/**
|
||||||
|
* Get environment variables for an augmentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Map of environment variable names to descriptions
|
||||||
|
*/
|
||||||
|
getEnvironmentVariables(augId: string): Promise<Record<string, any> | null>;
|
||||||
|
/**
|
||||||
|
* Get configuration examples for an augmentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Configuration examples or empty array
|
||||||
|
*/
|
||||||
|
getConfigExamples(augId: string): Promise<any[]>;
|
||||||
|
/**
|
||||||
|
* Check if an augmentation supports configuration
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns True if augmentation supports configuration
|
||||||
|
*/
|
||||||
|
supportsConfiguration(augId: string): Promise<boolean>;
|
||||||
|
/**
|
||||||
|
* Get augmentations by category
|
||||||
|
* @param category Category to filter by
|
||||||
|
* @returns List of augmentations in the category
|
||||||
|
*/
|
||||||
|
getByCategory(category: string): Promise<AugmentationListing[]>;
|
||||||
|
/**
|
||||||
|
* Get enabled augmentations
|
||||||
|
* @returns List of enabled augmentations
|
||||||
|
*/
|
||||||
|
getEnabled(): Promise<AugmentationListing[]>;
|
||||||
|
/**
|
||||||
|
* Search augmentations by keyword
|
||||||
|
* @param query Search query
|
||||||
|
* @returns Matching augmentations
|
||||||
|
*/
|
||||||
|
search(query: string): Promise<AugmentationListing[]>;
|
||||||
|
/**
|
||||||
|
* Export configuration for all augmentations
|
||||||
|
* @returns Map of augmentation IDs to configurations
|
||||||
|
*/
|
||||||
|
exportConfigurations(): Promise<Record<string, any>>;
|
||||||
|
/**
|
||||||
|
* Import configurations for multiple augmentations
|
||||||
|
* @param configs Map of augmentation IDs to configurations
|
||||||
|
* @returns Results of import operation
|
||||||
|
*/
|
||||||
|
importConfigurations(configs: Record<string, any>): Promise<Record<string, {
|
||||||
|
success: boolean;
|
||||||
|
error?: string;
|
||||||
|
}>>;
|
||||||
|
/**
|
||||||
|
* Generate configuration documentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Markdown documentation
|
||||||
|
*/
|
||||||
|
generateConfigDocs(augId: string): Promise<string | null>;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,441 @@
|
||||||
|
/**
|
||||||
|
* Augmentation Discovery API
|
||||||
|
*
|
||||||
|
* Provides discovery and configuration capabilities for augmentations
|
||||||
|
* Enables tools like brain-cloud to dynamically discover, configure, and manage augmentations
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* Augmentation Discovery API
|
||||||
|
*
|
||||||
|
* Provides a unified interface for discovering and managing augmentations
|
||||||
|
*/
|
||||||
|
export class AugmentationDiscovery {
|
||||||
|
constructor(registry) {
|
||||||
|
this.registry = registry;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Discover all registered augmentations with manifests
|
||||||
|
* @param options Discovery options
|
||||||
|
* @returns List of augmentation listings
|
||||||
|
*/
|
||||||
|
async discover(options = {}) {
|
||||||
|
const augmentations = this.registry.getAll();
|
||||||
|
const listings = [];
|
||||||
|
for (const aug of augmentations) {
|
||||||
|
// Check if augmentation has manifest support
|
||||||
|
const hasManifest = 'getManifest' in aug && typeof aug.getManifest === 'function';
|
||||||
|
if (!hasManifest) {
|
||||||
|
// Skip augmentations without manifest support (legacy)
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
// Check if augmentation has manifest method
|
||||||
|
if (!('getManifest' in aug) || typeof aug.getManifest !== 'function') {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const getManifestFn = aug.getManifest;
|
||||||
|
const manifest = getManifestFn();
|
||||||
|
// Apply filters
|
||||||
|
if (options.category && manifest.category !== options.category) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (options.enabled !== undefined) {
|
||||||
|
const isEnabled = aug.enabled !== false;
|
||||||
|
if (isEnabled !== options.enabled) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Build listing
|
||||||
|
const listing = {
|
||||||
|
id: manifest.id,
|
||||||
|
name: manifest.name,
|
||||||
|
manifest,
|
||||||
|
status: {
|
||||||
|
enabled: aug.enabled !== false,
|
||||||
|
initialized: aug.isInitialized || false,
|
||||||
|
category: aug.category || manifest.category,
|
||||||
|
priority: aug.priority
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// Include configuration if requested
|
||||||
|
if (options.includeConfig && 'getConfig' in aug) {
|
||||||
|
const getConfigFn = aug.getConfig;
|
||||||
|
listing.config = {
|
||||||
|
current: getConfigFn()
|
||||||
|
};
|
||||||
|
if (options.includeSchema) {
|
||||||
|
listing.config.schema = manifest.configSchema;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
listings.push(listing);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.warn(`Failed to get manifest for augmentation ${aug.name}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Sort by priority (highest first) then by name
|
||||||
|
listings.sort((a, b) => {
|
||||||
|
const priorityDiff = b.status.priority - a.status.priority;
|
||||||
|
if (priorityDiff !== 0)
|
||||||
|
return priorityDiff;
|
||||||
|
return a.name.localeCompare(b.name);
|
||||||
|
});
|
||||||
|
return listings;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get a specific augmentation's manifest
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Augmentation manifest or null if not found
|
||||||
|
*/
|
||||||
|
async getManifest(augId) {
|
||||||
|
const aug = this.registry.get(augId);
|
||||||
|
if (!aug || !('getManifest' in aug)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const getManifestFn = aug.getManifest;
|
||||||
|
return getManifestFn();
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.error(`Failed to get manifest for ${augId}:`, error);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get configuration schema for an augmentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Configuration schema or null
|
||||||
|
*/
|
||||||
|
async getConfigSchema(augId) {
|
||||||
|
const manifest = await this.getManifest(augId);
|
||||||
|
return manifest?.configSchema || null;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get current configuration for an augmentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Current configuration or null
|
||||||
|
*/
|
||||||
|
async getConfig(augId) {
|
||||||
|
const aug = this.registry.get(augId);
|
||||||
|
if (!aug || !('getConfig' in aug)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const getConfigFn = aug.getConfig;
|
||||||
|
return getConfigFn();
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.error(`Failed to get config for ${augId}:`, error);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Update configuration for an augmentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @param config New configuration
|
||||||
|
* @returns Updated configuration or null on failure
|
||||||
|
*/
|
||||||
|
async updateConfig(augId, config) {
|
||||||
|
const aug = this.registry.get(augId);
|
||||||
|
if (!aug || !('updateConfig' in aug) || !('getConfig' in aug)) {
|
||||||
|
throw new Error(`Augmentation ${augId} does not support configuration updates`);
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const updateConfigFn = aug.updateConfig;
|
||||||
|
await updateConfigFn(config);
|
||||||
|
const getConfigFn = aug.getConfig;
|
||||||
|
return getConfigFn();
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
throw new Error(`Failed to update config for ${augId}: ${error}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Validate configuration against schema
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @param config Configuration to validate
|
||||||
|
* @returns Validation result
|
||||||
|
*/
|
||||||
|
async validateConfig(augId, config) {
|
||||||
|
const schema = await this.getConfigSchema(augId);
|
||||||
|
if (!schema) {
|
||||||
|
return {
|
||||||
|
valid: true,
|
||||||
|
warnings: ['No schema available for validation']
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const errors = [];
|
||||||
|
const warnings = [];
|
||||||
|
const suggestions = [];
|
||||||
|
// Check required fields
|
||||||
|
if (schema.required) {
|
||||||
|
for (const field of schema.required) {
|
||||||
|
if (config[field] === undefined) {
|
||||||
|
errors.push(`Missing required field: ${field}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Validate properties
|
||||||
|
if (schema.properties) {
|
||||||
|
for (const [key, propSchema] of Object.entries(schema.properties)) {
|
||||||
|
const value = config[key];
|
||||||
|
if (value === undefined) {
|
||||||
|
// Check if there's a default
|
||||||
|
if (propSchema.default !== undefined) {
|
||||||
|
suggestions.push(`Field '${key}' not provided, will use default: ${JSON.stringify(propSchema.default)}`);
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Type validation
|
||||||
|
if (propSchema.type) {
|
||||||
|
const actualType = Array.isArray(value) ? 'array' : typeof value;
|
||||||
|
if (actualType !== propSchema.type) {
|
||||||
|
errors.push(`${key}: expected ${propSchema.type}, got ${actualType}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Additional validations for specific types
|
||||||
|
this.validatePropertyValue(key, value, propSchema, errors, warnings);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Check for unknown properties
|
||||||
|
if (schema.additionalProperties === false && schema.properties) {
|
||||||
|
const allowedKeys = Object.keys(schema.properties);
|
||||||
|
for (const key of Object.keys(config)) {
|
||||||
|
if (!allowedKeys.includes(key)) {
|
||||||
|
warnings.push(`Unknown property: ${key}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
valid: errors.length === 0,
|
||||||
|
errors: errors.length > 0 ? errors : undefined,
|
||||||
|
warnings: warnings.length > 0 ? warnings : undefined,
|
||||||
|
suggestions: suggestions.length > 0 ? suggestions : undefined
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Validate a property value against its schema
|
||||||
|
*/
|
||||||
|
validatePropertyValue(key, value, schema, errors, warnings) {
|
||||||
|
// Number validations
|
||||||
|
if (schema.type === 'number') {
|
||||||
|
if (schema.minimum !== undefined && value < schema.minimum) {
|
||||||
|
errors.push(`${key}: value ${value} is less than minimum ${schema.minimum}`);
|
||||||
|
}
|
||||||
|
if (schema.maximum !== undefined && value > schema.maximum) {
|
||||||
|
errors.push(`${key}: value ${value} is greater than maximum ${schema.maximum}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// String validations
|
||||||
|
if (schema.type === 'string') {
|
||||||
|
if (schema.minLength !== undefined && value.length < schema.minLength) {
|
||||||
|
errors.push(`${key}: length ${value.length} is less than minimum ${schema.minLength}`);
|
||||||
|
}
|
||||||
|
if (schema.maxLength !== undefined && value.length > schema.maxLength) {
|
||||||
|
errors.push(`${key}: length ${value.length} is greater than maximum ${schema.maxLength}`);
|
||||||
|
}
|
||||||
|
if (schema.pattern) {
|
||||||
|
const regex = new RegExp(schema.pattern);
|
||||||
|
if (!regex.test(value)) {
|
||||||
|
errors.push(`${key}: value does not match pattern ${schema.pattern}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Enum validation
|
||||||
|
if (schema.enum && !schema.enum.includes(value)) {
|
||||||
|
errors.push(`${key}: value '${value}' is not one of allowed values: ${schema.enum.join(', ')}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get environment variables for an augmentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Map of environment variable names to descriptions
|
||||||
|
*/
|
||||||
|
async getEnvironmentVariables(augId) {
|
||||||
|
const manifest = await this.getManifest(augId);
|
||||||
|
if (!manifest?.configSchema?.properties) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const prefix = `BRAINY_AUG_${augId.toUpperCase()}_`;
|
||||||
|
const vars = {};
|
||||||
|
for (const [key, prop] of Object.entries(manifest.configSchema.properties)) {
|
||||||
|
const envKey = prefix + key.replace(/([A-Z])/g, '_$1').toUpperCase();
|
||||||
|
vars[envKey] = {
|
||||||
|
configKey: key,
|
||||||
|
description: prop.description,
|
||||||
|
type: prop.type,
|
||||||
|
default: prop.default,
|
||||||
|
required: manifest.configSchema.required?.includes(key),
|
||||||
|
currentValue: typeof process !== 'undefined' ? process.env?.[envKey] : undefined
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return vars;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get configuration examples for an augmentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Configuration examples or empty array
|
||||||
|
*/
|
||||||
|
async getConfigExamples(augId) {
|
||||||
|
const manifest = await this.getManifest(augId);
|
||||||
|
return manifest?.configExamples || [];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if an augmentation supports configuration
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns True if augmentation supports configuration
|
||||||
|
*/
|
||||||
|
async supportsConfiguration(augId) {
|
||||||
|
const aug = this.registry.get(augId);
|
||||||
|
return !!(aug && 'getConfig' in aug && 'updateConfig' in aug);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get augmentations by category
|
||||||
|
* @param category Category to filter by
|
||||||
|
* @returns List of augmentations in the category
|
||||||
|
*/
|
||||||
|
async getByCategory(category) {
|
||||||
|
return this.discover({ category });
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get enabled augmentations
|
||||||
|
* @returns List of enabled augmentations
|
||||||
|
*/
|
||||||
|
async getEnabled() {
|
||||||
|
return this.discover({ enabled: true });
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Search augmentations by keyword
|
||||||
|
* @param query Search query
|
||||||
|
* @returns Matching augmentations
|
||||||
|
*/
|
||||||
|
async search(query) {
|
||||||
|
const all = await this.discover();
|
||||||
|
const queryLower = query.toLowerCase();
|
||||||
|
return all.filter(listing => {
|
||||||
|
const manifest = listing.manifest;
|
||||||
|
// Search in various fields
|
||||||
|
const searchFields = [
|
||||||
|
manifest.name,
|
||||||
|
manifest.description,
|
||||||
|
manifest.longDescription,
|
||||||
|
...(manifest.keywords || []),
|
||||||
|
manifest.category
|
||||||
|
].filter(Boolean).map(s => s.toLowerCase());
|
||||||
|
return searchFields.some(field => field.includes(queryLower));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Export configuration for all augmentations
|
||||||
|
* @returns Map of augmentation IDs to configurations
|
||||||
|
*/
|
||||||
|
async exportConfigurations() {
|
||||||
|
const configs = {};
|
||||||
|
const listings = await this.discover({ includeConfig: true });
|
||||||
|
for (const listing of listings) {
|
||||||
|
if (listing.config?.current) {
|
||||||
|
configs[listing.id] = listing.config.current;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return configs;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Import configurations for multiple augmentations
|
||||||
|
* @param configs Map of augmentation IDs to configurations
|
||||||
|
* @returns Results of import operation
|
||||||
|
*/
|
||||||
|
async importConfigurations(configs) {
|
||||||
|
const results = {};
|
||||||
|
for (const [augId, config] of Object.entries(configs)) {
|
||||||
|
try {
|
||||||
|
// Validate before applying
|
||||||
|
const validation = await this.validateConfig(augId, config);
|
||||||
|
if (!validation.valid) {
|
||||||
|
results[augId] = {
|
||||||
|
success: false,
|
||||||
|
error: `Validation failed: ${validation.errors?.join(', ')}`
|
||||||
|
};
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Apply configuration
|
||||||
|
await this.updateConfig(augId, config);
|
||||||
|
results[augId] = { success: true };
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
results[augId] = {
|
||||||
|
success: false,
|
||||||
|
error: error instanceof Error ? error.message : String(error)
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return results;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Generate configuration documentation
|
||||||
|
* @param augId Augmentation ID
|
||||||
|
* @returns Markdown documentation
|
||||||
|
*/
|
||||||
|
async generateConfigDocs(augId) {
|
||||||
|
const manifest = await this.getManifest(augId);
|
||||||
|
if (!manifest)
|
||||||
|
return null;
|
||||||
|
const schema = manifest.configSchema;
|
||||||
|
const examples = manifest.configExamples || [];
|
||||||
|
const envVars = await this.getEnvironmentVariables(augId);
|
||||||
|
let docs = `# ${manifest.name} Configuration\n\n`;
|
||||||
|
docs += `${manifest.description}\n\n`;
|
||||||
|
if (manifest.longDescription) {
|
||||||
|
docs += `## Overview\n\n${manifest.longDescription}\n\n`;
|
||||||
|
}
|
||||||
|
// Configuration options
|
||||||
|
if (schema?.properties) {
|
||||||
|
docs += `## Configuration Options\n\n`;
|
||||||
|
for (const [key, prop] of Object.entries(schema.properties)) {
|
||||||
|
const required = schema.required?.includes(key) ? ' *(required)*' : '';
|
||||||
|
docs += `### \`${key}\`${required}\n\n`;
|
||||||
|
if (prop.description) {
|
||||||
|
docs += `${prop.description}\n\n`;
|
||||||
|
}
|
||||||
|
docs += `- **Type**: ${prop.type}\n`;
|
||||||
|
if (prop.default !== undefined) {
|
||||||
|
docs += `- **Default**: \`${JSON.stringify(prop.default)}\`\n`;
|
||||||
|
}
|
||||||
|
if (prop.minimum !== undefined) {
|
||||||
|
docs += `- **Minimum**: ${prop.minimum}\n`;
|
||||||
|
}
|
||||||
|
if (prop.maximum !== undefined) {
|
||||||
|
docs += `- **Maximum**: ${prop.maximum}\n`;
|
||||||
|
}
|
||||||
|
if (prop.enum) {
|
||||||
|
docs += `- **Allowed values**: ${prop.enum.map(v => `\`${v}\``).join(', ')}\n`;
|
||||||
|
}
|
||||||
|
docs += '\n';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Environment variables
|
||||||
|
if (envVars && Object.keys(envVars).length > 0) {
|
||||||
|
docs += `## Environment Variables\n\n`;
|
||||||
|
docs += `| Variable | Config Key | Type | Required | Default |\n`;
|
||||||
|
docs += `|----------|------------|------|----------|----------|\n`;
|
||||||
|
for (const [envKey, info] of Object.entries(envVars)) {
|
||||||
|
docs += `| \`${envKey}\` | ${info.configKey} | ${info.type} | ${info.required ? 'Yes' : 'No'} | ${info.default !== undefined ? `\`${info.default}\`` : '-'} |\n`;
|
||||||
|
}
|
||||||
|
docs += '\n';
|
||||||
|
}
|
||||||
|
// Examples
|
||||||
|
if (examples.length > 0) {
|
||||||
|
docs += `## Examples\n\n`;
|
||||||
|
for (const example of examples) {
|
||||||
|
docs += `### ${example.name}\n\n`;
|
||||||
|
if (example.description) {
|
||||||
|
docs += `${example.description}\n\n`;
|
||||||
|
}
|
||||||
|
docs += '```json\n';
|
||||||
|
docs += JSON.stringify(example.config, null, 2);
|
||||||
|
docs += '\n```\n\n';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return docs;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=discovery.js.map
|
||||||
File diff suppressed because one or more lines are too long
142
.recovery-workspace/dist-backup-20250910-141917/augmentations/discovery/catalogDiscovery.d.ts
vendored
Normal file
142
.recovery-workspace/dist-backup-20250910-141917/augmentations/discovery/catalogDiscovery.d.ts
vendored
Normal file
|
|
@ -0,0 +1,142 @@
|
||||||
|
/**
|
||||||
|
* Brain-Cloud Catalog Discovery
|
||||||
|
*
|
||||||
|
* Discovers augmentations available in the brain-cloud catalog
|
||||||
|
* Handles free, premium, and community augmentations
|
||||||
|
*/
|
||||||
|
import { AugmentationManifest } from '../manifest.js';
|
||||||
|
export interface CatalogAugmentation {
|
||||||
|
id: string;
|
||||||
|
name: string;
|
||||||
|
description: string;
|
||||||
|
longDescription?: string;
|
||||||
|
category: string;
|
||||||
|
status: 'available' | 'coming_soon' | 'deprecated';
|
||||||
|
tier: 'free' | 'premium' | 'enterprise';
|
||||||
|
price?: {
|
||||||
|
monthly?: number;
|
||||||
|
yearly?: number;
|
||||||
|
oneTime?: number;
|
||||||
|
};
|
||||||
|
manifest?: AugmentationManifest;
|
||||||
|
source: 'catalog';
|
||||||
|
cdnUrl?: string;
|
||||||
|
npmPackage?: string;
|
||||||
|
githubRepo?: string;
|
||||||
|
author?: {
|
||||||
|
name: string;
|
||||||
|
url?: string;
|
||||||
|
};
|
||||||
|
metrics?: {
|
||||||
|
installations: number;
|
||||||
|
rating: number;
|
||||||
|
reviews: number;
|
||||||
|
};
|
||||||
|
requirements?: {
|
||||||
|
minBrainyVersion?: string;
|
||||||
|
maxBrainyVersion?: string;
|
||||||
|
dependencies?: string[];
|
||||||
|
};
|
||||||
|
}
|
||||||
|
export interface CatalogOptions {
|
||||||
|
apiUrl?: string;
|
||||||
|
apiKey?: string;
|
||||||
|
cache?: boolean;
|
||||||
|
cacheTimeout?: number;
|
||||||
|
}
|
||||||
|
export interface CatalogFilters {
|
||||||
|
category?: string;
|
||||||
|
tier?: 'free' | 'premium' | 'enterprise';
|
||||||
|
status?: 'available' | 'coming_soon' | 'deprecated';
|
||||||
|
search?: string;
|
||||||
|
installed?: boolean;
|
||||||
|
minRating?: number;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Brain-Cloud Catalog Discovery
|
||||||
|
*/
|
||||||
|
export declare class CatalogDiscovery {
|
||||||
|
private apiUrl;
|
||||||
|
private apiKey?;
|
||||||
|
private cache;
|
||||||
|
private cacheTimeout;
|
||||||
|
constructor(options?: CatalogOptions);
|
||||||
|
/**
|
||||||
|
* Discover augmentations from catalog
|
||||||
|
*/
|
||||||
|
discover(filters?: CatalogFilters): Promise<CatalogAugmentation[]>;
|
||||||
|
/**
|
||||||
|
* Get specific augmentation details
|
||||||
|
*/
|
||||||
|
getAugmentation(id: string): Promise<CatalogAugmentation | null>;
|
||||||
|
/**
|
||||||
|
* Get augmentation manifest
|
||||||
|
*/
|
||||||
|
getManifest(id: string): Promise<AugmentationManifest | null>;
|
||||||
|
/**
|
||||||
|
* Get CDN URL for dynamic loading
|
||||||
|
*/
|
||||||
|
getCDNUrl(id: string): Promise<string | null>;
|
||||||
|
/**
|
||||||
|
* Check if user has access to augmentation
|
||||||
|
*/
|
||||||
|
checkAccess(id: string): Promise<{
|
||||||
|
hasAccess: boolean;
|
||||||
|
requiresPurchase?: boolean;
|
||||||
|
requiredTier?: string;
|
||||||
|
}>;
|
||||||
|
/**
|
||||||
|
* Purchase/activate augmentation
|
||||||
|
*/
|
||||||
|
purchase(id: string, licenseKey?: string): Promise<{
|
||||||
|
success: boolean;
|
||||||
|
cdnUrl?: string;
|
||||||
|
npmPackage?: string;
|
||||||
|
licenseKey?: string;
|
||||||
|
}>;
|
||||||
|
/**
|
||||||
|
* Get user's purchased augmentations
|
||||||
|
*/
|
||||||
|
getPurchased(): Promise<CatalogAugmentation[]>;
|
||||||
|
/**
|
||||||
|
* Get categories
|
||||||
|
*/
|
||||||
|
getCategories(): Promise<Array<{
|
||||||
|
id: string;
|
||||||
|
name: string;
|
||||||
|
description: string;
|
||||||
|
icon?: string;
|
||||||
|
}>>;
|
||||||
|
/**
|
||||||
|
* Search augmentations
|
||||||
|
*/
|
||||||
|
search(query: string): Promise<CatalogAugmentation[]>;
|
||||||
|
/**
|
||||||
|
* Get trending augmentations
|
||||||
|
*/
|
||||||
|
getTrending(limit?: number): Promise<CatalogAugmentation[]>;
|
||||||
|
/**
|
||||||
|
* Get recommended augmentations
|
||||||
|
*/
|
||||||
|
getRecommended(): Promise<CatalogAugmentation[]>;
|
||||||
|
/**
|
||||||
|
* Transform catalog data
|
||||||
|
*/
|
||||||
|
private transformCatalogData;
|
||||||
|
/**
|
||||||
|
* Transform single augmentation
|
||||||
|
*/
|
||||||
|
private transformAugmentation;
|
||||||
|
/**
|
||||||
|
* Get request headers
|
||||||
|
*/
|
||||||
|
private getHeaders;
|
||||||
|
/**
|
||||||
|
* Clear cache
|
||||||
|
*/
|
||||||
|
clearCache(): void;
|
||||||
|
/**
|
||||||
|
* Set API key
|
||||||
|
*/
|
||||||
|
setApiKey(apiKey: string): void;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,249 @@
|
||||||
|
/**
|
||||||
|
* Brain-Cloud Catalog Discovery
|
||||||
|
*
|
||||||
|
* Discovers augmentations available in the brain-cloud catalog
|
||||||
|
* Handles free, premium, and community augmentations
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* Brain-Cloud Catalog Discovery
|
||||||
|
*/
|
||||||
|
export class CatalogDiscovery {
|
||||||
|
constructor(options = {}) {
|
||||||
|
this.cache = new Map();
|
||||||
|
this.apiUrl = options.apiUrl || 'https://api.soulcraft.com/brain-cloud';
|
||||||
|
this.apiKey = options.apiKey;
|
||||||
|
this.cacheTimeout = options.cacheTimeout || 5 * 60 * 1000; // 5 minutes
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Discover augmentations from catalog
|
||||||
|
*/
|
||||||
|
async discover(filters = {}) {
|
||||||
|
const cacheKey = JSON.stringify(filters);
|
||||||
|
// Check cache
|
||||||
|
if (this.cache.has(cacheKey)) {
|
||||||
|
const cached = this.cache.get(cacheKey);
|
||||||
|
if (Date.now() - cached.timestamp < this.cacheTimeout) {
|
||||||
|
return cached.data;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Build query parameters
|
||||||
|
const params = new URLSearchParams();
|
||||||
|
if (filters.category)
|
||||||
|
params.append('category', filters.category);
|
||||||
|
if (filters.tier)
|
||||||
|
params.append('tier', filters.tier);
|
||||||
|
if (filters.status)
|
||||||
|
params.append('status', filters.status);
|
||||||
|
if (filters.search)
|
||||||
|
params.append('q', filters.search);
|
||||||
|
if (filters.minRating)
|
||||||
|
params.append('minRating', filters.minRating.toString());
|
||||||
|
// Fetch from API
|
||||||
|
const response = await fetch(`${this.apiUrl}/augmentations/discover?${params}`, {
|
||||||
|
headers: this.getHeaders()
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Failed to fetch catalog: ${response.statusText}`);
|
||||||
|
}
|
||||||
|
const data = await response.json();
|
||||||
|
const augmentations = this.transformCatalogData(data);
|
||||||
|
// Cache result
|
||||||
|
this.cache.set(cacheKey, {
|
||||||
|
data: augmentations,
|
||||||
|
timestamp: Date.now()
|
||||||
|
});
|
||||||
|
return augmentations;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get specific augmentation details
|
||||||
|
*/
|
||||||
|
async getAugmentation(id) {
|
||||||
|
const response = await fetch(`${this.apiUrl}/augmentations/${id}`, {
|
||||||
|
headers: this.getHeaders()
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
if (response.status === 404)
|
||||||
|
return null;
|
||||||
|
throw new Error(`Failed to fetch augmentation: ${response.statusText}`);
|
||||||
|
}
|
||||||
|
const data = await response.json();
|
||||||
|
return this.transformAugmentation(data);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get augmentation manifest
|
||||||
|
*/
|
||||||
|
async getManifest(id) {
|
||||||
|
const response = await fetch(`${this.apiUrl}/augmentations/${id}/manifest`, {
|
||||||
|
headers: this.getHeaders()
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
if (response.status === 404)
|
||||||
|
return null;
|
||||||
|
throw new Error(`Failed to fetch manifest: ${response.statusText}`);
|
||||||
|
}
|
||||||
|
return response.json();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get CDN URL for dynamic loading
|
||||||
|
*/
|
||||||
|
async getCDNUrl(id) {
|
||||||
|
const aug = await this.getAugmentation(id);
|
||||||
|
return aug?.cdnUrl || null;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if user has access to augmentation
|
||||||
|
*/
|
||||||
|
async checkAccess(id) {
|
||||||
|
if (!this.apiKey) {
|
||||||
|
// No API key, only free augmentations
|
||||||
|
const aug = await this.getAugmentation(id);
|
||||||
|
return {
|
||||||
|
hasAccess: aug?.tier === 'free',
|
||||||
|
requiresPurchase: aug?.tier !== 'free',
|
||||||
|
requiredTier: aug?.tier
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const response = await fetch(`${this.apiUrl}/augmentations/${id}/access`, {
|
||||||
|
headers: this.getHeaders()
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Failed to check access: ${response.statusText}`);
|
||||||
|
}
|
||||||
|
return response.json();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Purchase/activate augmentation
|
||||||
|
*/
|
||||||
|
async purchase(id, licenseKey) {
|
||||||
|
const body = licenseKey ? { licenseKey } : {};
|
||||||
|
const response = await fetch(`${this.apiUrl}/augmentations/${id}/purchase`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {
|
||||||
|
...this.getHeaders(),
|
||||||
|
'Content-Type': 'application/json'
|
||||||
|
},
|
||||||
|
body: JSON.stringify(body)
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Failed to purchase: ${response.statusText}`);
|
||||||
|
}
|
||||||
|
return response.json();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get user's purchased augmentations
|
||||||
|
*/
|
||||||
|
async getPurchased() {
|
||||||
|
if (!this.apiKey) {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
const response = await fetch(`${this.apiUrl}/user/augmentations`, {
|
||||||
|
headers: this.getHeaders()
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Failed to fetch purchased: ${response.statusText}`);
|
||||||
|
}
|
||||||
|
const data = await response.json();
|
||||||
|
return this.transformCatalogData(data);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get categories
|
||||||
|
*/
|
||||||
|
async getCategories() {
|
||||||
|
const response = await fetch(`${this.apiUrl}/augmentations/categories`, {
|
||||||
|
headers: this.getHeaders()
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Failed to fetch categories: ${response.statusText}`);
|
||||||
|
}
|
||||||
|
return response.json();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Search augmentations
|
||||||
|
*/
|
||||||
|
async search(query) {
|
||||||
|
return this.discover({ search: query });
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get trending augmentations
|
||||||
|
*/
|
||||||
|
async getTrending(limit = 10) {
|
||||||
|
const response = await fetch(`${this.apiUrl}/augmentations/trending?limit=${limit}`, {
|
||||||
|
headers: this.getHeaders()
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Failed to fetch trending: ${response.statusText}`);
|
||||||
|
}
|
||||||
|
const data = await response.json();
|
||||||
|
return this.transformCatalogData(data);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get recommended augmentations
|
||||||
|
*/
|
||||||
|
async getRecommended() {
|
||||||
|
if (!this.apiKey) {
|
||||||
|
// Return popular free augmentations
|
||||||
|
return this.discover({ tier: 'free', minRating: 4 });
|
||||||
|
}
|
||||||
|
const response = await fetch(`${this.apiUrl}/augmentations/recommended`, {
|
||||||
|
headers: this.getHeaders()
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Failed to fetch recommended: ${response.statusText}`);
|
||||||
|
}
|
||||||
|
const data = await response.json();
|
||||||
|
return this.transformCatalogData(data);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Transform catalog data
|
||||||
|
*/
|
||||||
|
transformCatalogData(data) {
|
||||||
|
return data.map(item => this.transformAugmentation(item));
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Transform single augmentation
|
||||||
|
*/
|
||||||
|
transformAugmentation(item) {
|
||||||
|
return {
|
||||||
|
id: item.id,
|
||||||
|
name: item.name,
|
||||||
|
description: item.description,
|
||||||
|
longDescription: item.longDescription,
|
||||||
|
category: item.category,
|
||||||
|
status: item.status || 'available',
|
||||||
|
tier: item.tier || 'free',
|
||||||
|
price: item.price,
|
||||||
|
manifest: item.manifest,
|
||||||
|
source: 'catalog',
|
||||||
|
cdnUrl: item.cdnUrl || `https://cdn.soulcraft.com/augmentations/${item.id}@latest`,
|
||||||
|
npmPackage: item.npmPackage,
|
||||||
|
githubRepo: item.githubRepo,
|
||||||
|
author: item.author,
|
||||||
|
metrics: item.metrics,
|
||||||
|
requirements: item.requirements
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get request headers
|
||||||
|
*/
|
||||||
|
getHeaders() {
|
||||||
|
const headers = {};
|
||||||
|
if (this.apiKey) {
|
||||||
|
headers['Authorization'] = `Bearer ${this.apiKey}`;
|
||||||
|
}
|
||||||
|
return headers;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear cache
|
||||||
|
*/
|
||||||
|
clearCache() {
|
||||||
|
this.cache.clear();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Set API key
|
||||||
|
*/
|
||||||
|
setApiKey(apiKey) {
|
||||||
|
this.apiKey = apiKey;
|
||||||
|
this.clearCache(); // Clear cache when API key changes
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=catalogDiscovery.js.map
|
||||||
File diff suppressed because one or more lines are too long
84
.recovery-workspace/dist-backup-20250910-141917/augmentations/discovery/localDiscovery.d.ts
vendored
Normal file
84
.recovery-workspace/dist-backup-20250910-141917/augmentations/discovery/localDiscovery.d.ts
vendored
Normal file
|
|
@ -0,0 +1,84 @@
|
||||||
|
/**
|
||||||
|
* Local Augmentation Discovery
|
||||||
|
*
|
||||||
|
* Discovers augmentations installed locally in node_modules
|
||||||
|
* and built-in augmentations that ship with Brainy
|
||||||
|
*/
|
||||||
|
import { AugmentationManifest } from '../manifest.js';
|
||||||
|
export interface LocalAugmentation {
|
||||||
|
id: string;
|
||||||
|
name: string;
|
||||||
|
source: 'builtin' | 'npm' | 'local';
|
||||||
|
path: string;
|
||||||
|
manifest?: AugmentationManifest;
|
||||||
|
package?: {
|
||||||
|
name: string;
|
||||||
|
version: string;
|
||||||
|
description?: string;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Discovers augmentations installed locally
|
||||||
|
*/
|
||||||
|
export declare class LocalAugmentationDiscovery {
|
||||||
|
private options;
|
||||||
|
private builtInAugmentations;
|
||||||
|
private installedAugmentations;
|
||||||
|
constructor(options?: {
|
||||||
|
brainyPath?: string;
|
||||||
|
projectPath?: string;
|
||||||
|
scanNodeModules?: boolean;
|
||||||
|
});
|
||||||
|
/**
|
||||||
|
* Register built-in augmentations that ship with Brainy
|
||||||
|
*/
|
||||||
|
private registerBuiltIn;
|
||||||
|
/**
|
||||||
|
* Find Brainy installation path
|
||||||
|
*/
|
||||||
|
private findBrainyPath;
|
||||||
|
/**
|
||||||
|
* Discover all augmentations
|
||||||
|
*/
|
||||||
|
discoverAll(): Promise<LocalAugmentation[]>;
|
||||||
|
/**
|
||||||
|
* Scan node_modules for installed augmentations
|
||||||
|
*/
|
||||||
|
private scanNodeModules;
|
||||||
|
/**
|
||||||
|
* Scan local project for augmentations
|
||||||
|
*/
|
||||||
|
private scanLocalProject;
|
||||||
|
/**
|
||||||
|
* Load augmentation from package
|
||||||
|
*/
|
||||||
|
private loadPackageAugmentation;
|
||||||
|
/**
|
||||||
|
* Load package.json
|
||||||
|
*/
|
||||||
|
private loadPackageJson;
|
||||||
|
/**
|
||||||
|
* Convert name to human-readable format
|
||||||
|
*/
|
||||||
|
private humanizeName;
|
||||||
|
/**
|
||||||
|
* Get built-in augmentations
|
||||||
|
*/
|
||||||
|
getBuiltIn(): LocalAugmentation[];
|
||||||
|
/**
|
||||||
|
* Get installed augmentations
|
||||||
|
*/
|
||||||
|
getInstalled(): LocalAugmentation[];
|
||||||
|
/**
|
||||||
|
* Check if augmentation is installed
|
||||||
|
*/
|
||||||
|
isInstalled(id: string): boolean;
|
||||||
|
/**
|
||||||
|
* Get import path for augmentation
|
||||||
|
*/
|
||||||
|
getImportPath(id: string): string | null;
|
||||||
|
/**
|
||||||
|
* Load augmentation module dynamically
|
||||||
|
*/
|
||||||
|
loadAugmentation(id: string): Promise<any>;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,247 @@
|
||||||
|
/**
|
||||||
|
* Local Augmentation Discovery
|
||||||
|
*
|
||||||
|
* Discovers augmentations installed locally in node_modules
|
||||||
|
* and built-in augmentations that ship with Brainy
|
||||||
|
*/
|
||||||
|
import { existsSync, readdirSync, readFileSync } from 'fs';
|
||||||
|
import { join } from 'path';
|
||||||
|
/**
|
||||||
|
* Discovers augmentations installed locally
|
||||||
|
*/
|
||||||
|
export class LocalAugmentationDiscovery {
|
||||||
|
constructor(options = {}) {
|
||||||
|
this.options = options;
|
||||||
|
this.builtInAugmentations = new Map();
|
||||||
|
this.installedAugmentations = new Map();
|
||||||
|
this.options = {
|
||||||
|
brainyPath: this.options.brainyPath || this.findBrainyPath(),
|
||||||
|
projectPath: this.options.projectPath || process.cwd(),
|
||||||
|
scanNodeModules: this.options.scanNodeModules ?? true
|
||||||
|
};
|
||||||
|
// Register built-in augmentations
|
||||||
|
this.registerBuiltIn();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Register built-in augmentations that ship with Brainy
|
||||||
|
*/
|
||||||
|
registerBuiltIn() {
|
||||||
|
const builtIn = [
|
||||||
|
{ id: 'wal', name: 'Write-Ahead Log', path: 'walAugmentation' },
|
||||||
|
{ id: 'cache', name: 'Cache', path: 'cacheAugmentation' },
|
||||||
|
{ id: 'batch', name: 'Batch Processing', path: 'batchProcessingAugmentation' },
|
||||||
|
{ id: 'entity-registry', name: 'Entity Registry', path: 'entityRegistryAugmentation' },
|
||||||
|
{ id: 'index', name: 'Index', path: 'indexAugmentation' },
|
||||||
|
{ id: 'metrics', name: 'Metrics', path: 'metricsAugmentation' },
|
||||||
|
{ id: 'monitoring', name: 'Monitoring', path: 'monitoringAugmentation' },
|
||||||
|
{ id: 'connection-pool', name: 'Connection Pool', path: 'connectionPoolAugmentation' },
|
||||||
|
{ id: 'request-deduplicator', name: 'Request Deduplicator', path: 'requestDeduplicatorAugmentation' },
|
||||||
|
{ id: 'api-server', name: 'API Server', path: 'apiServerAugmentation' },
|
||||||
|
{ id: 'neural-import', name: 'Neural Import', path: 'neuralImport' },
|
||||||
|
{ id: 'intelligent-verb-scoring', name: 'Intelligent Verb Scoring', path: 'intelligentVerbScoringAugmentation' },
|
||||||
|
{ id: 'universal-display', name: 'Universal Display', path: 'universalDisplayAugmentation' },
|
||||||
|
// Storage augmentations
|
||||||
|
{ id: 'memory-storage', name: 'Memory Storage', path: 'storageAugmentations', export: 'MemoryStorageAugmentation' },
|
||||||
|
{ id: 'filesystem-storage', name: 'FileSystem Storage', path: 'storageAugmentations', export: 'FileSystemStorageAugmentation' },
|
||||||
|
{ id: 'opfs-storage', name: 'OPFS Storage', path: 'storageAugmentations', export: 'OPFSStorageAugmentation' },
|
||||||
|
{ id: 's3-storage', name: 'S3 Storage', path: 'storageAugmentations', export: 'S3StorageAugmentation' },
|
||||||
|
];
|
||||||
|
for (const aug of builtIn) {
|
||||||
|
this.builtInAugmentations.set(aug.id, {
|
||||||
|
id: aug.id,
|
||||||
|
name: aug.name,
|
||||||
|
source: 'builtin',
|
||||||
|
path: `@soulcraft/brainy/augmentations/${aug.path}`,
|
||||||
|
package: {
|
||||||
|
name: '@soulcraft/brainy',
|
||||||
|
version: 'builtin',
|
||||||
|
description: `Built-in ${aug.name} augmentation`
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Find Brainy installation path
|
||||||
|
*/
|
||||||
|
findBrainyPath() {
|
||||||
|
// Try to find brainy in node_modules
|
||||||
|
const possiblePaths = [
|
||||||
|
join(process.cwd(), 'node_modules', '@soulcraft', 'brainy'),
|
||||||
|
join(process.cwd(), 'node_modules', 'brainy'),
|
||||||
|
join(process.cwd(), '..', 'node_modules', '@soulcraft', 'brainy'),
|
||||||
|
];
|
||||||
|
for (const path of possiblePaths) {
|
||||||
|
if (existsSync(path)) {
|
||||||
|
return path;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Fallback to current directory
|
||||||
|
return process.cwd();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Discover all augmentations
|
||||||
|
*/
|
||||||
|
async discoverAll() {
|
||||||
|
const augmentations = [];
|
||||||
|
// Add built-in augmentations
|
||||||
|
augmentations.push(...this.builtInAugmentations.values());
|
||||||
|
// Scan node_modules if enabled
|
||||||
|
if (this.options.scanNodeModules) {
|
||||||
|
const installed = await this.scanNodeModules();
|
||||||
|
augmentations.push(...installed);
|
||||||
|
}
|
||||||
|
// Scan local project
|
||||||
|
const local = await this.scanLocalProject();
|
||||||
|
augmentations.push(...local);
|
||||||
|
return augmentations;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Scan node_modules for installed augmentations
|
||||||
|
*/
|
||||||
|
async scanNodeModules() {
|
||||||
|
const augmentations = [];
|
||||||
|
const nodeModulesPath = join(this.options.projectPath, 'node_modules');
|
||||||
|
if (!existsSync(nodeModulesPath)) {
|
||||||
|
return augmentations;
|
||||||
|
}
|
||||||
|
// Scan @brainy/* packages
|
||||||
|
const brainyPath = join(nodeModulesPath, '@brainy');
|
||||||
|
if (existsSync(brainyPath)) {
|
||||||
|
const packages = readdirSync(brainyPath);
|
||||||
|
for (const pkg of packages) {
|
||||||
|
const augmentation = await this.loadPackageAugmentation(join(brainyPath, pkg));
|
||||||
|
if (augmentation) {
|
||||||
|
augmentations.push(augmentation);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Scan packages with brainy-augmentation keyword
|
||||||
|
const packages = readdirSync(nodeModulesPath);
|
||||||
|
for (const pkg of packages) {
|
||||||
|
if (pkg.startsWith('@') || pkg.startsWith('.'))
|
||||||
|
continue;
|
||||||
|
const pkgPath = join(nodeModulesPath, pkg);
|
||||||
|
const packageJson = this.loadPackageJson(pkgPath);
|
||||||
|
if (packageJson?.keywords?.includes('brainy-augmentation')) {
|
||||||
|
const augmentation = await this.loadPackageAugmentation(pkgPath);
|
||||||
|
if (augmentation) {
|
||||||
|
augmentations.push(augmentation);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return augmentations;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Scan local project for augmentations
|
||||||
|
*/
|
||||||
|
async scanLocalProject() {
|
||||||
|
const augmentations = [];
|
||||||
|
// Check for augmentations directory
|
||||||
|
const augPath = join(this.options.projectPath, 'augmentations');
|
||||||
|
if (existsSync(augPath)) {
|
||||||
|
const files = readdirSync(augPath);
|
||||||
|
for (const file of files) {
|
||||||
|
if (file.endsWith('.ts') || file.endsWith('.js')) {
|
||||||
|
const name = file.replace(/\.(ts|js)$/, '');
|
||||||
|
augmentations.push({
|
||||||
|
id: name,
|
||||||
|
name: this.humanizeName(name),
|
||||||
|
source: 'local',
|
||||||
|
path: join(augPath, file)
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return augmentations;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load augmentation from package
|
||||||
|
*/
|
||||||
|
async loadPackageAugmentation(pkgPath) {
|
||||||
|
const packageJson = this.loadPackageJson(pkgPath);
|
||||||
|
if (!packageJson)
|
||||||
|
return null;
|
||||||
|
// Check if it's a brainy augmentation
|
||||||
|
const isBrainyAug = packageJson.keywords?.includes('brainy-augmentation') ||
|
||||||
|
packageJson.brainy?.type === 'augmentation';
|
||||||
|
if (!isBrainyAug)
|
||||||
|
return null;
|
||||||
|
const manifest = packageJson.brainy?.manifest || null;
|
||||||
|
return {
|
||||||
|
id: packageJson.brainy?.id || packageJson.name.replace('@brainy/', ''),
|
||||||
|
name: packageJson.brainy?.name || packageJson.name,
|
||||||
|
source: 'npm',
|
||||||
|
path: pkgPath,
|
||||||
|
manifest,
|
||||||
|
package: {
|
||||||
|
name: packageJson.name,
|
||||||
|
version: packageJson.version,
|
||||||
|
description: packageJson.description
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load package.json
|
||||||
|
*/
|
||||||
|
loadPackageJson(pkgPath) {
|
||||||
|
const packageJsonPath = join(pkgPath, 'package.json');
|
||||||
|
if (!existsSync(packageJsonPath))
|
||||||
|
return null;
|
||||||
|
try {
|
||||||
|
return JSON.parse(readFileSync(packageJsonPath, 'utf8'));
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Convert name to human-readable format
|
||||||
|
*/
|
||||||
|
humanizeName(name) {
|
||||||
|
return name
|
||||||
|
.replace(/[-_]/g, ' ')
|
||||||
|
.replace(/augmentation/gi, '')
|
||||||
|
.replace(/\b\w/g, l => l.toUpperCase())
|
||||||
|
.trim();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get built-in augmentations
|
||||||
|
*/
|
||||||
|
getBuiltIn() {
|
||||||
|
return Array.from(this.builtInAugmentations.values());
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get installed augmentations
|
||||||
|
*/
|
||||||
|
getInstalled() {
|
||||||
|
return Array.from(this.installedAugmentations.values());
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if augmentation is installed
|
||||||
|
*/
|
||||||
|
isInstalled(id) {
|
||||||
|
return this.builtInAugmentations.has(id) ||
|
||||||
|
this.installedAugmentations.has(id);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get import path for augmentation
|
||||||
|
*/
|
||||||
|
getImportPath(id) {
|
||||||
|
const aug = this.builtInAugmentations.get(id) ||
|
||||||
|
this.installedAugmentations.get(id);
|
||||||
|
return aug?.path || null;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load augmentation module dynamically
|
||||||
|
*/
|
||||||
|
async loadAugmentation(id) {
|
||||||
|
const path = this.getImportPath(id);
|
||||||
|
if (!path) {
|
||||||
|
throw new Error(`Augmentation ${id} not found`);
|
||||||
|
}
|
||||||
|
// Dynamic import
|
||||||
|
const module = await import(path);
|
||||||
|
return module.default || module;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=localDiscovery.js.map
|
||||||
File diff suppressed because one or more lines are too long
97
.recovery-workspace/dist-backup-20250910-141917/augmentations/discovery/runtimeLoader.d.ts
vendored
Normal file
97
.recovery-workspace/dist-backup-20250910-141917/augmentations/discovery/runtimeLoader.d.ts
vendored
Normal file
|
|
@ -0,0 +1,97 @@
|
||||||
|
/**
|
||||||
|
* Runtime Augmentation Loader
|
||||||
|
*
|
||||||
|
* Dynamically loads and registers augmentations at runtime
|
||||||
|
* Supports CDN loading for browser environments and npm imports for Node.js
|
||||||
|
*/
|
||||||
|
import { BrainyAugmentation, AugmentationRegistry } from '../brainyAugmentation.js';
|
||||||
|
import { AugmentationManifest } from '../manifest.js';
|
||||||
|
export interface LoaderOptions {
|
||||||
|
cdnUrl?: string;
|
||||||
|
allowUnsafe?: boolean;
|
||||||
|
sandbox?: boolean;
|
||||||
|
timeout?: number;
|
||||||
|
cache?: boolean;
|
||||||
|
}
|
||||||
|
export interface LoadedAugmentation {
|
||||||
|
id: string;
|
||||||
|
instance: BrainyAugmentation;
|
||||||
|
manifest: AugmentationManifest;
|
||||||
|
source: 'cdn' | 'npm' | 'local';
|
||||||
|
loadTime: number;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Runtime Augmentation Loader
|
||||||
|
*
|
||||||
|
* Enables dynamic loading of augmentations from various sources
|
||||||
|
*/
|
||||||
|
export declare class RuntimeAugmentationLoader {
|
||||||
|
private options;
|
||||||
|
private loaded;
|
||||||
|
private cdnCache;
|
||||||
|
private registry?;
|
||||||
|
constructor(options?: LoaderOptions);
|
||||||
|
/**
|
||||||
|
* Set the augmentation registry
|
||||||
|
*/
|
||||||
|
setRegistry(registry: AugmentationRegistry): void;
|
||||||
|
/**
|
||||||
|
* Load augmentation from CDN (browser)
|
||||||
|
*/
|
||||||
|
loadFromCDN(id: string, version?: string, config?: any): Promise<LoadedAugmentation>;
|
||||||
|
/**
|
||||||
|
* Load augmentation from NPM (Node.js)
|
||||||
|
*/
|
||||||
|
loadFromNPM(packageName: string, config?: any): Promise<LoadedAugmentation>;
|
||||||
|
/**
|
||||||
|
* Load augmentation from local file
|
||||||
|
*/
|
||||||
|
loadFromFile(path: string, config?: any): Promise<LoadedAugmentation>;
|
||||||
|
/**
|
||||||
|
* Load multiple augmentations
|
||||||
|
*/
|
||||||
|
loadBatch(augmentations: Array<{
|
||||||
|
source: 'cdn' | 'npm' | 'local';
|
||||||
|
id: string;
|
||||||
|
version?: string;
|
||||||
|
path?: string;
|
||||||
|
config?: any;
|
||||||
|
}>): Promise<LoadedAugmentation[]>;
|
||||||
|
/**
|
||||||
|
* Unload augmentation
|
||||||
|
*/
|
||||||
|
unload(id: string): boolean;
|
||||||
|
/**
|
||||||
|
* Get loaded augmentations
|
||||||
|
*/
|
||||||
|
getLoaded(): LoadedAugmentation[];
|
||||||
|
/**
|
||||||
|
* Check if augmentation is loaded
|
||||||
|
*/
|
||||||
|
isLoaded(id: string): boolean;
|
||||||
|
/**
|
||||||
|
* Get loaded augmentation
|
||||||
|
*/
|
||||||
|
getAugmentation(id: string): BrainyAugmentation | null;
|
||||||
|
/**
|
||||||
|
* Load CDN module (browser-specific)
|
||||||
|
*/
|
||||||
|
private loadCDNModule;
|
||||||
|
/**
|
||||||
|
* Validate augmentation instance
|
||||||
|
*/
|
||||||
|
private isValidAugmentation;
|
||||||
|
/**
|
||||||
|
* Clear all caches
|
||||||
|
*/
|
||||||
|
clearCache(): void;
|
||||||
|
/**
|
||||||
|
* Get load statistics
|
||||||
|
*/
|
||||||
|
getStats(): {
|
||||||
|
loaded: number;
|
||||||
|
totalLoadTime: number;
|
||||||
|
averageLoadTime: number;
|
||||||
|
sources: Record<string, number>;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,337 @@
|
||||||
|
/**
|
||||||
|
* Runtime Augmentation Loader
|
||||||
|
*
|
||||||
|
* Dynamically loads and registers augmentations at runtime
|
||||||
|
* Supports CDN loading for browser environments and npm imports for Node.js
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* Runtime Augmentation Loader
|
||||||
|
*
|
||||||
|
* Enables dynamic loading of augmentations from various sources
|
||||||
|
*/
|
||||||
|
export class RuntimeAugmentationLoader {
|
||||||
|
constructor(options = {}) {
|
||||||
|
this.options = options;
|
||||||
|
this.loaded = new Map();
|
||||||
|
this.cdnCache = new Map();
|
||||||
|
this.options = {
|
||||||
|
cdnUrl: options.cdnUrl || 'https://cdn.soulcraft.com/augmentations',
|
||||||
|
allowUnsafe: options.allowUnsafe || false,
|
||||||
|
sandbox: options.sandbox || true,
|
||||||
|
timeout: options.timeout || 30000,
|
||||||
|
cache: options.cache ?? true
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Set the augmentation registry
|
||||||
|
*/
|
||||||
|
setRegistry(registry) {
|
||||||
|
this.registry = registry;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load augmentation from CDN (browser)
|
||||||
|
*/
|
||||||
|
async loadFromCDN(id, version = 'latest', config) {
|
||||||
|
// Check if already loaded
|
||||||
|
if (this.loaded.has(id)) {
|
||||||
|
return this.loaded.get(id);
|
||||||
|
}
|
||||||
|
const url = `${this.options.cdnUrl}/${id}@${version}/index.js`;
|
||||||
|
const startTime = Date.now();
|
||||||
|
try {
|
||||||
|
// Load module from CDN
|
||||||
|
const module = await this.loadCDNModule(url);
|
||||||
|
// Extract augmentation class
|
||||||
|
const AugmentationClass = module.default || module[Object.keys(module)[0]];
|
||||||
|
if (!AugmentationClass) {
|
||||||
|
throw new Error(`No augmentation class found in module ${id}`);
|
||||||
|
}
|
||||||
|
// Create instance
|
||||||
|
const instance = new AugmentationClass(config);
|
||||||
|
// Validate it's a proper augmentation
|
||||||
|
if (!this.isValidAugmentation(instance)) {
|
||||||
|
throw new Error(`Invalid augmentation: ${id}`);
|
||||||
|
}
|
||||||
|
// Get manifest
|
||||||
|
const manifest = instance.getManifest ? instance.getManifest() : {
|
||||||
|
id,
|
||||||
|
name: id,
|
||||||
|
version: version,
|
||||||
|
description: `Dynamically loaded ${id}`,
|
||||||
|
category: 'external'
|
||||||
|
};
|
||||||
|
const loaded = {
|
||||||
|
id,
|
||||||
|
instance,
|
||||||
|
manifest,
|
||||||
|
source: 'cdn',
|
||||||
|
loadTime: Date.now() - startTime
|
||||||
|
};
|
||||||
|
// Cache
|
||||||
|
this.loaded.set(id, loaded);
|
||||||
|
// Auto-register if registry is set
|
||||||
|
if (this.registry) {
|
||||||
|
this.registry.register(instance);
|
||||||
|
}
|
||||||
|
return loaded;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
throw new Error(`Failed to load augmentation ${id} from CDN: ${error}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load augmentation from NPM (Node.js)
|
||||||
|
*/
|
||||||
|
async loadFromNPM(packageName, config) {
|
||||||
|
// Check if already loaded
|
||||||
|
const id = packageName.replace('@', '').replace('/', '-');
|
||||||
|
if (this.loaded.has(id)) {
|
||||||
|
return this.loaded.get(id);
|
||||||
|
}
|
||||||
|
const startTime = Date.now();
|
||||||
|
try {
|
||||||
|
// Dynamic import
|
||||||
|
const module = await import(packageName);
|
||||||
|
// Extract augmentation class
|
||||||
|
const AugmentationClass = module.default || module[Object.keys(module)[0]];
|
||||||
|
if (!AugmentationClass) {
|
||||||
|
throw new Error(`No augmentation class found in package ${packageName}`);
|
||||||
|
}
|
||||||
|
// Create instance
|
||||||
|
const instance = new AugmentationClass(config);
|
||||||
|
// Validate
|
||||||
|
if (!this.isValidAugmentation(instance)) {
|
||||||
|
throw new Error(`Invalid augmentation in package: ${packageName}`);
|
||||||
|
}
|
||||||
|
// Get manifest
|
||||||
|
const manifest = instance.getManifest ? instance.getManifest() : {
|
||||||
|
id,
|
||||||
|
name: packageName,
|
||||||
|
version: 'unknown',
|
||||||
|
description: `Loaded from ${packageName}`,
|
||||||
|
category: 'external'
|
||||||
|
};
|
||||||
|
const loaded = {
|
||||||
|
id,
|
||||||
|
instance,
|
||||||
|
manifest,
|
||||||
|
source: 'npm',
|
||||||
|
loadTime: Date.now() - startTime
|
||||||
|
};
|
||||||
|
// Cache
|
||||||
|
this.loaded.set(id, loaded);
|
||||||
|
// Auto-register
|
||||||
|
if (this.registry) {
|
||||||
|
this.registry.register(instance);
|
||||||
|
}
|
||||||
|
return loaded;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
throw new Error(`Failed to load augmentation from NPM ${packageName}: ${error}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load augmentation from local file
|
||||||
|
*/
|
||||||
|
async loadFromFile(path, config) {
|
||||||
|
const startTime = Date.now();
|
||||||
|
try {
|
||||||
|
// Dynamic import
|
||||||
|
const module = await import(path);
|
||||||
|
// Extract augmentation class
|
||||||
|
const AugmentationClass = module.default || module[Object.keys(module)[0]];
|
||||||
|
if (!AugmentationClass) {
|
||||||
|
throw new Error(`No augmentation class found in file ${path}`);
|
||||||
|
}
|
||||||
|
// Create instance
|
||||||
|
const instance = new AugmentationClass(config);
|
||||||
|
// Validate
|
||||||
|
if (!this.isValidAugmentation(instance)) {
|
||||||
|
throw new Error(`Invalid augmentation in file: ${path}`);
|
||||||
|
}
|
||||||
|
// Extract ID from path
|
||||||
|
const id = path.split('/').pop()?.replace(/\.(js|ts)$/, '') || 'unknown';
|
||||||
|
// Get manifest
|
||||||
|
const manifest = instance.getManifest ? instance.getManifest() : {
|
||||||
|
id,
|
||||||
|
name: id,
|
||||||
|
version: 'local',
|
||||||
|
description: `Loaded from ${path}`,
|
||||||
|
category: 'local'
|
||||||
|
};
|
||||||
|
const loaded = {
|
||||||
|
id,
|
||||||
|
instance,
|
||||||
|
manifest,
|
||||||
|
source: 'local',
|
||||||
|
loadTime: Date.now() - startTime
|
||||||
|
};
|
||||||
|
// Cache
|
||||||
|
this.loaded.set(id, loaded);
|
||||||
|
// Auto-register
|
||||||
|
if (this.registry) {
|
||||||
|
this.registry.register(instance);
|
||||||
|
}
|
||||||
|
return loaded;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
throw new Error(`Failed to load augmentation from file ${path}: ${error}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load multiple augmentations
|
||||||
|
*/
|
||||||
|
async loadBatch(augmentations) {
|
||||||
|
const results = await Promise.allSettled(augmentations.map(aug => {
|
||||||
|
switch (aug.source) {
|
||||||
|
case 'cdn':
|
||||||
|
return this.loadFromCDN(aug.id, aug.version, aug.config);
|
||||||
|
case 'npm':
|
||||||
|
return this.loadFromNPM(aug.id, aug.config);
|
||||||
|
case 'local':
|
||||||
|
return this.loadFromFile(aug.path || aug.id, aug.config);
|
||||||
|
default:
|
||||||
|
return Promise.reject(new Error(`Unknown source: ${aug.source}`));
|
||||||
|
}
|
||||||
|
}));
|
||||||
|
const loaded = [];
|
||||||
|
const errors = [];
|
||||||
|
for (const result of results) {
|
||||||
|
if (result.status === 'fulfilled') {
|
||||||
|
loaded.push(result.value);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
errors.push(result.reason.message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (errors.length > 0) {
|
||||||
|
console.warn('Some augmentations failed to load:', errors);
|
||||||
|
}
|
||||||
|
return loaded;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Unload augmentation
|
||||||
|
*/
|
||||||
|
unload(id) {
|
||||||
|
const loaded = this.loaded.get(id);
|
||||||
|
if (!loaded)
|
||||||
|
return false;
|
||||||
|
// Shutdown if possible
|
||||||
|
if (loaded.instance.shutdown) {
|
||||||
|
loaded.instance.shutdown();
|
||||||
|
}
|
||||||
|
// Remove from registry if set
|
||||||
|
// Note: Registry doesn't have unregister yet, would need to add
|
||||||
|
// Remove from cache
|
||||||
|
this.loaded.delete(id);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get loaded augmentations
|
||||||
|
*/
|
||||||
|
getLoaded() {
|
||||||
|
return Array.from(this.loaded.values());
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if augmentation is loaded
|
||||||
|
*/
|
||||||
|
isLoaded(id) {
|
||||||
|
return this.loaded.has(id);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get loaded augmentation
|
||||||
|
*/
|
||||||
|
getAugmentation(id) {
|
||||||
|
return this.loaded.get(id)?.instance || null;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Load CDN module (browser-specific)
|
||||||
|
*/
|
||||||
|
async loadCDNModule(url) {
|
||||||
|
// Check cache
|
||||||
|
if (this.options.cache && this.cdnCache.has(url)) {
|
||||||
|
return this.cdnCache.get(url);
|
||||||
|
}
|
||||||
|
// In browser environment
|
||||||
|
if (typeof window !== 'undefined') {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const timeout = setTimeout(() => {
|
||||||
|
reject(new Error(`Timeout loading ${url}`));
|
||||||
|
}, this.options.timeout);
|
||||||
|
// Create script element
|
||||||
|
const script = document.createElement('script');
|
||||||
|
script.type = 'module';
|
||||||
|
script.src = url;
|
||||||
|
// Handle load
|
||||||
|
script.onload = async () => {
|
||||||
|
clearTimeout(timeout);
|
||||||
|
// The module should register itself on window
|
||||||
|
const moduleId = url.split('/').pop()?.split('@')[0];
|
||||||
|
if (moduleId && window[moduleId]) {
|
||||||
|
const module = window[moduleId];
|
||||||
|
// Cache
|
||||||
|
if (this.options.cache) {
|
||||||
|
this.cdnCache.set(url, module);
|
||||||
|
}
|
||||||
|
resolve(module);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
reject(new Error(`Module not found on window: ${moduleId}`));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// Handle error
|
||||||
|
script.onerror = () => {
|
||||||
|
clearTimeout(timeout);
|
||||||
|
reject(new Error(`Failed to load script: ${url}`));
|
||||||
|
};
|
||||||
|
// Add to document
|
||||||
|
document.head.appendChild(script);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
// In Node.js, use dynamic import
|
||||||
|
const module = await import(url);
|
||||||
|
// Cache
|
||||||
|
if (this.options.cache) {
|
||||||
|
this.cdnCache.set(url, module);
|
||||||
|
}
|
||||||
|
return module;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Validate augmentation instance
|
||||||
|
*/
|
||||||
|
isValidAugmentation(instance) {
|
||||||
|
// Check required properties
|
||||||
|
return !!(instance.name &&
|
||||||
|
instance.timing &&
|
||||||
|
instance.operations &&
|
||||||
|
instance.priority !== undefined &&
|
||||||
|
typeof instance.execute === 'function' &&
|
||||||
|
typeof instance.initialize === 'function');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear all caches
|
||||||
|
*/
|
||||||
|
clearCache() {
|
||||||
|
this.cdnCache.clear();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get load statistics
|
||||||
|
*/
|
||||||
|
getStats() {
|
||||||
|
const loaded = Array.from(this.loaded.values());
|
||||||
|
const totalLoadTime = loaded.reduce((sum, aug) => sum + aug.loadTime, 0);
|
||||||
|
const sources = loaded.reduce((acc, aug) => {
|
||||||
|
acc[aug.source] = (acc[aug.source] || 0) + 1;
|
||||||
|
return acc;
|
||||||
|
}, {});
|
||||||
|
return {
|
||||||
|
loaded: loaded.length,
|
||||||
|
totalLoadTime,
|
||||||
|
averageLoadTime: loaded.length > 0 ? totalLoadTime / loaded.length : 0,
|
||||||
|
sources
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=runtimeLoader.js.map
|
||||||
File diff suppressed because one or more lines are too long
130
.recovery-workspace/dist-backup-20250910-141917/augmentations/display/cache.d.ts
vendored
Normal file
130
.recovery-workspace/dist-backup-20250910-141917/augmentations/display/cache.d.ts
vendored
Normal file
|
|
@ -0,0 +1,130 @@
|
||||||
|
/**
|
||||||
|
* Universal Display Augmentation - Intelligent Caching System
|
||||||
|
*
|
||||||
|
* High-performance LRU cache with smart eviction and batch optimization
|
||||||
|
* Designed for minimal memory footprint and maximum hit ratio
|
||||||
|
*/
|
||||||
|
import type { ComputedDisplayFields, DisplayAugmentationStats } from './types.js';
|
||||||
|
/**
|
||||||
|
* LRU (Least Recently Used) Cache for computed display fields
|
||||||
|
* Optimized for the display augmentation use case
|
||||||
|
*/
|
||||||
|
export declare class DisplayCache {
|
||||||
|
private cache;
|
||||||
|
private readonly maxSize;
|
||||||
|
private stats;
|
||||||
|
constructor(maxSize?: number);
|
||||||
|
/**
|
||||||
|
* Get cached display fields with LRU update
|
||||||
|
* @param key Cache key
|
||||||
|
* @returns Cached fields or null if not found
|
||||||
|
*/
|
||||||
|
get(key: string): ComputedDisplayFields | null;
|
||||||
|
/**
|
||||||
|
* Store computed display fields in cache
|
||||||
|
* @param key Cache key
|
||||||
|
* @param fields Computed display fields
|
||||||
|
* @param computationTime Time taken to compute (for stats)
|
||||||
|
*/
|
||||||
|
set(key: string, fields: ComputedDisplayFields, computationTime?: number): void;
|
||||||
|
/**
|
||||||
|
* Check if a key exists in cache without affecting LRU order
|
||||||
|
* @param key Cache key
|
||||||
|
* @returns True if key exists
|
||||||
|
*/
|
||||||
|
has(key: string): boolean;
|
||||||
|
/**
|
||||||
|
* Generate cache key from data
|
||||||
|
* @param id Entity ID (preferred)
|
||||||
|
* @param data Fallback data for key generation
|
||||||
|
* @param entityType Type of entity (noun/verb)
|
||||||
|
* @returns Cache key string
|
||||||
|
*/
|
||||||
|
generateKey(id?: string, data?: any, entityType?: 'noun' | 'verb'): string;
|
||||||
|
/**
|
||||||
|
* Clear all cached entries
|
||||||
|
*/
|
||||||
|
clear(): void;
|
||||||
|
/**
|
||||||
|
* Get cache statistics
|
||||||
|
* @returns Cache performance statistics
|
||||||
|
*/
|
||||||
|
getStats(): DisplayAugmentationStats;
|
||||||
|
/**
|
||||||
|
* Get current cache size
|
||||||
|
* @returns Number of cached entries
|
||||||
|
*/
|
||||||
|
size(): number;
|
||||||
|
/**
|
||||||
|
* Get cache capacity
|
||||||
|
* @returns Maximum cache size
|
||||||
|
*/
|
||||||
|
capacity(): number;
|
||||||
|
/**
|
||||||
|
* Evict least recently used entry
|
||||||
|
*/
|
||||||
|
private evictOldest;
|
||||||
|
/**
|
||||||
|
* Simple hash function for cache keys
|
||||||
|
* @param str String to hash
|
||||||
|
* @returns Simple hash number
|
||||||
|
*/
|
||||||
|
private simpleHash;
|
||||||
|
/**
|
||||||
|
* Optimize cache by removing stale entries
|
||||||
|
* Called periodically to maintain cache health
|
||||||
|
*/
|
||||||
|
optimizeCache(): void;
|
||||||
|
/**
|
||||||
|
* Precompute display fields for a batch of entities
|
||||||
|
* @param entities Array of entities with their compute functions
|
||||||
|
* @returns Promise resolving when batch is complete
|
||||||
|
*/
|
||||||
|
batchPrecompute<T>(entities: Array<{
|
||||||
|
key: string;
|
||||||
|
computeFn: () => Promise<ComputedDisplayFields>;
|
||||||
|
}>): Promise<void>;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Request deduplicator for batch processing
|
||||||
|
* Prevents duplicate computations for the same data
|
||||||
|
*/
|
||||||
|
export declare class RequestDeduplicator {
|
||||||
|
private pendingRequests;
|
||||||
|
private readonly batchSize;
|
||||||
|
constructor(batchSize?: number);
|
||||||
|
/**
|
||||||
|
* Deduplicate computation request
|
||||||
|
* @param key Unique key for the computation
|
||||||
|
* @param computeFn Function to compute the result
|
||||||
|
* @returns Promise that resolves to the computed fields
|
||||||
|
*/
|
||||||
|
deduplicate(key: string, computeFn: () => Promise<ComputedDisplayFields>): Promise<ComputedDisplayFields>;
|
||||||
|
/**
|
||||||
|
* Get number of pending requests
|
||||||
|
* @returns Number of pending computations
|
||||||
|
*/
|
||||||
|
getPendingCount(): number;
|
||||||
|
/**
|
||||||
|
* Clear all pending requests
|
||||||
|
*/
|
||||||
|
clear(): void;
|
||||||
|
/**
|
||||||
|
* Shutdown the deduplicator
|
||||||
|
*/
|
||||||
|
shutdown(): void;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get global display cache instance
|
||||||
|
* @param maxSize Optional cache size (only used on first call)
|
||||||
|
* @returns Shared display cache instance
|
||||||
|
*/
|
||||||
|
export declare function getGlobalDisplayCache(maxSize?: number): DisplayCache;
|
||||||
|
/**
|
||||||
|
* Clear global cache (for testing or memory management)
|
||||||
|
*/
|
||||||
|
export declare function clearGlobalDisplayCache(): void;
|
||||||
|
/**
|
||||||
|
* Shutdown global cache and cleanup
|
||||||
|
*/
|
||||||
|
export declare function shutdownGlobalDisplayCache(): void;
|
||||||
|
|
@ -0,0 +1,319 @@
|
||||||
|
/**
|
||||||
|
* Universal Display Augmentation - Intelligent Caching System
|
||||||
|
*
|
||||||
|
* High-performance LRU cache with smart eviction and batch optimization
|
||||||
|
* Designed for minimal memory footprint and maximum hit ratio
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* LRU (Least Recently Used) Cache for computed display fields
|
||||||
|
* Optimized for the display augmentation use case
|
||||||
|
*/
|
||||||
|
export class DisplayCache {
|
||||||
|
constructor(maxSize = 1000) {
|
||||||
|
this.cache = new Map();
|
||||||
|
this.stats = {
|
||||||
|
hits: 0,
|
||||||
|
misses: 0,
|
||||||
|
evictions: 0,
|
||||||
|
totalComputations: 0,
|
||||||
|
totalComputationTime: 0
|
||||||
|
};
|
||||||
|
this.maxSize = maxSize;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get cached display fields with LRU update
|
||||||
|
* @param key Cache key
|
||||||
|
* @returns Cached fields or null if not found
|
||||||
|
*/
|
||||||
|
get(key) {
|
||||||
|
const entry = this.cache.get(key);
|
||||||
|
if (!entry) {
|
||||||
|
this.stats.misses++;
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
// Update LRU - move to end
|
||||||
|
this.cache.delete(key);
|
||||||
|
entry.lastAccessed = Date.now();
|
||||||
|
entry.accessCount++;
|
||||||
|
this.cache.set(key, entry);
|
||||||
|
this.stats.hits++;
|
||||||
|
return entry.fields;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Store computed display fields in cache
|
||||||
|
* @param key Cache key
|
||||||
|
* @param fields Computed display fields
|
||||||
|
* @param computationTime Time taken to compute (for stats)
|
||||||
|
*/
|
||||||
|
set(key, fields, computationTime) {
|
||||||
|
// Remove if already exists (for LRU update)
|
||||||
|
if (this.cache.has(key)) {
|
||||||
|
this.cache.delete(key);
|
||||||
|
}
|
||||||
|
// Create cache entry
|
||||||
|
const entry = {
|
||||||
|
fields,
|
||||||
|
lastAccessed: Date.now(),
|
||||||
|
accessCount: 1
|
||||||
|
};
|
||||||
|
// Add to end (most recently used)
|
||||||
|
this.cache.set(key, entry);
|
||||||
|
// Update stats
|
||||||
|
this.stats.totalComputations++;
|
||||||
|
if (computationTime) {
|
||||||
|
this.stats.totalComputationTime += computationTime;
|
||||||
|
}
|
||||||
|
// Evict oldest if over capacity
|
||||||
|
if (this.cache.size > this.maxSize) {
|
||||||
|
this.evictOldest();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if a key exists in cache without affecting LRU order
|
||||||
|
* @param key Cache key
|
||||||
|
* @returns True if key exists
|
||||||
|
*/
|
||||||
|
has(key) {
|
||||||
|
return this.cache.has(key);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Generate cache key from data
|
||||||
|
* @param id Entity ID (preferred)
|
||||||
|
* @param data Fallback data for key generation
|
||||||
|
* @param entityType Type of entity (noun/verb)
|
||||||
|
* @returns Cache key string
|
||||||
|
*/
|
||||||
|
generateKey(id, data, entityType = 'noun') {
|
||||||
|
// Use ID if available (most reliable)
|
||||||
|
if (id) {
|
||||||
|
return `${entityType}:${id}`;
|
||||||
|
}
|
||||||
|
// Generate hash from data
|
||||||
|
if (data) {
|
||||||
|
const dataString = JSON.stringify(data, Object.keys(data).sort());
|
||||||
|
const hash = this.simpleHash(dataString);
|
||||||
|
return `${entityType}:hash:${hash}`;
|
||||||
|
}
|
||||||
|
// Fallback to timestamp (not ideal but prevents crashes)
|
||||||
|
return `${entityType}:temp:${Date.now()}:${Math.random()}`;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear all cached entries
|
||||||
|
*/
|
||||||
|
clear() {
|
||||||
|
this.cache.clear();
|
||||||
|
this.stats = {
|
||||||
|
hits: 0,
|
||||||
|
misses: 0,
|
||||||
|
evictions: 0,
|
||||||
|
totalComputations: 0,
|
||||||
|
totalComputationTime: 0
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get cache statistics
|
||||||
|
* @returns Cache performance statistics
|
||||||
|
*/
|
||||||
|
getStats() {
|
||||||
|
const hitRatio = this.stats.hits + this.stats.misses > 0
|
||||||
|
? this.stats.hits / (this.stats.hits + this.stats.misses)
|
||||||
|
: 0;
|
||||||
|
const avgComputationTime = this.stats.totalComputations > 0
|
||||||
|
? this.stats.totalComputationTime / this.stats.totalComputations
|
||||||
|
: 0;
|
||||||
|
// Analyze cached types for common types statistics
|
||||||
|
const typeCount = new Map();
|
||||||
|
let fastestComputation = Infinity;
|
||||||
|
let slowestComputation = 0;
|
||||||
|
for (const entry of this.cache.values()) {
|
||||||
|
const type = entry.fields.type;
|
||||||
|
typeCount.set(type, (typeCount.get(type) || 0) + 1);
|
||||||
|
}
|
||||||
|
const commonTypes = Array.from(typeCount.entries())
|
||||||
|
.sort(([, a], [, b]) => b - a)
|
||||||
|
.slice(0, 10)
|
||||||
|
.map(([type, count]) => ({
|
||||||
|
type,
|
||||||
|
count,
|
||||||
|
percentage: Math.round((count / this.cache.size) * 100)
|
||||||
|
}));
|
||||||
|
return {
|
||||||
|
totalComputations: this.stats.totalComputations,
|
||||||
|
cacheHitRatio: Math.round(hitRatio * 100) / 100,
|
||||||
|
averageComputationTime: Math.round(avgComputationTime * 100) / 100,
|
||||||
|
commonTypes,
|
||||||
|
performance: {
|
||||||
|
fastestComputation,
|
||||||
|
slowestComputation,
|
||||||
|
totalComputationTime: this.stats.totalComputationTime
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get current cache size
|
||||||
|
* @returns Number of cached entries
|
||||||
|
*/
|
||||||
|
size() {
|
||||||
|
return this.cache.size;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get cache capacity
|
||||||
|
* @returns Maximum cache size
|
||||||
|
*/
|
||||||
|
capacity() {
|
||||||
|
return this.maxSize;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Evict least recently used entry
|
||||||
|
*/
|
||||||
|
evictOldest() {
|
||||||
|
// First entry is oldest (LRU)
|
||||||
|
const firstKey = this.cache.keys().next().value;
|
||||||
|
if (firstKey) {
|
||||||
|
this.cache.delete(firstKey);
|
||||||
|
this.stats.evictions++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Simple hash function for cache keys
|
||||||
|
* @param str String to hash
|
||||||
|
* @returns Simple hash number
|
||||||
|
*/
|
||||||
|
simpleHash(str) {
|
||||||
|
let hash = 0;
|
||||||
|
for (let i = 0; i < str.length; i++) {
|
||||||
|
const char = str.charCodeAt(i);
|
||||||
|
hash = ((hash << 5) - hash) + char;
|
||||||
|
hash = hash & hash; // Convert to 32-bit integer
|
||||||
|
}
|
||||||
|
return Math.abs(hash);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Optimize cache by removing stale entries
|
||||||
|
* Called periodically to maintain cache health
|
||||||
|
*/
|
||||||
|
optimizeCache() {
|
||||||
|
const now = Date.now();
|
||||||
|
const maxAge = 24 * 60 * 60 * 1000; // 24 hours
|
||||||
|
const minAccessCount = 2; // Minimum access count to keep
|
||||||
|
const toDelete = [];
|
||||||
|
for (const [key, entry] of this.cache.entries()) {
|
||||||
|
// Remove very old entries with low access count
|
||||||
|
if (now - entry.lastAccessed > maxAge && entry.accessCount < minAccessCount) {
|
||||||
|
toDelete.push(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Remove stale entries
|
||||||
|
for (const key of toDelete) {
|
||||||
|
this.cache.delete(key);
|
||||||
|
this.stats.evictions++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Precompute display fields for a batch of entities
|
||||||
|
* @param entities Array of entities with their compute functions
|
||||||
|
* @returns Promise resolving when batch is complete
|
||||||
|
*/
|
||||||
|
async batchPrecompute(entities) {
|
||||||
|
const promises = entities.map(async ({ key, computeFn }) => {
|
||||||
|
if (!this.has(key)) {
|
||||||
|
const startTime = Date.now();
|
||||||
|
try {
|
||||||
|
const fields = await computeFn();
|
||||||
|
const computationTime = Date.now() - startTime;
|
||||||
|
this.set(key, fields, computationTime);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.warn(`Batch precompute failed for key ${key}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
await Promise.all(promises);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Request deduplicator for batch processing
|
||||||
|
* Prevents duplicate computations for the same data
|
||||||
|
*/
|
||||||
|
export class RequestDeduplicator {
|
||||||
|
constructor(batchSize = 50) {
|
||||||
|
this.pendingRequests = new Map();
|
||||||
|
this.batchSize = batchSize;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Deduplicate computation request
|
||||||
|
* @param key Unique key for the computation
|
||||||
|
* @param computeFn Function to compute the result
|
||||||
|
* @returns Promise that resolves to the computed fields
|
||||||
|
*/
|
||||||
|
async deduplicate(key, computeFn) {
|
||||||
|
// Return existing promise if already pending
|
||||||
|
if (this.pendingRequests.has(key)) {
|
||||||
|
return this.pendingRequests.get(key);
|
||||||
|
}
|
||||||
|
// Create new computation promise
|
||||||
|
const promise = computeFn().finally(() => {
|
||||||
|
// Remove from pending when complete
|
||||||
|
this.pendingRequests.delete(key);
|
||||||
|
});
|
||||||
|
this.pendingRequests.set(key, promise);
|
||||||
|
return promise;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get number of pending requests
|
||||||
|
* @returns Number of pending computations
|
||||||
|
*/
|
||||||
|
getPendingCount() {
|
||||||
|
return this.pendingRequests.size;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear all pending requests
|
||||||
|
*/
|
||||||
|
clear() {
|
||||||
|
this.pendingRequests.clear();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Shutdown the deduplicator
|
||||||
|
*/
|
||||||
|
shutdown() {
|
||||||
|
this.clear();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Global cache instance management
|
||||||
|
* Provides singleton access to display cache
|
||||||
|
*/
|
||||||
|
let globalDisplayCache = null;
|
||||||
|
/**
|
||||||
|
* Get global display cache instance
|
||||||
|
* @param maxSize Optional cache size (only used on first call)
|
||||||
|
* @returns Shared display cache instance
|
||||||
|
*/
|
||||||
|
export function getGlobalDisplayCache(maxSize) {
|
||||||
|
if (!globalDisplayCache) {
|
||||||
|
globalDisplayCache = new DisplayCache(maxSize);
|
||||||
|
// Set up periodic optimization
|
||||||
|
setInterval(() => {
|
||||||
|
globalDisplayCache?.optimizeCache();
|
||||||
|
}, 60 * 60 * 1000); // Every hour
|
||||||
|
}
|
||||||
|
return globalDisplayCache;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear global cache (for testing or memory management)
|
||||||
|
*/
|
||||||
|
export function clearGlobalDisplayCache() {
|
||||||
|
if (globalDisplayCache) {
|
||||||
|
globalDisplayCache.clear();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Shutdown global cache and cleanup
|
||||||
|
*/
|
||||||
|
export function shutdownGlobalDisplayCache() {
|
||||||
|
if (globalDisplayCache) {
|
||||||
|
globalDisplayCache.clear();
|
||||||
|
globalDisplayCache = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=cache.js.map
|
||||||
File diff suppressed because one or more lines are too long
52
.recovery-workspace/dist-backup-20250910-141917/augmentations/display/fieldPatterns.d.ts
vendored
Normal file
52
.recovery-workspace/dist-backup-20250910-141917/augmentations/display/fieldPatterns.d.ts
vendored
Normal file
|
|
@ -0,0 +1,52 @@
|
||||||
|
/**
|
||||||
|
* Universal Display Augmentation - Smart Field Patterns
|
||||||
|
*
|
||||||
|
* Intelligent field detection patterns for mapping user data to display fields
|
||||||
|
* Uses semantic understanding and common naming conventions
|
||||||
|
*/
|
||||||
|
import type { FieldPattern, FieldComputationContext } from './types.js';
|
||||||
|
/**
|
||||||
|
* Universal field patterns that work across all data types
|
||||||
|
* Ordered by confidence level (highest first)
|
||||||
|
*/
|
||||||
|
export declare const UNIVERSAL_FIELD_PATTERNS: FieldPattern[];
|
||||||
|
/**
|
||||||
|
* Type-specific field patterns for enhanced detection
|
||||||
|
* Used when we know the specific type of the entity
|
||||||
|
*/
|
||||||
|
export declare const TYPE_SPECIFIC_PATTERNS: Record<string, FieldPattern[]>;
|
||||||
|
/**
|
||||||
|
* Get field patterns for a specific entity type
|
||||||
|
* @param entityType The type of entity (noun or verb)
|
||||||
|
* @param specificType Optional specific noun/verb type
|
||||||
|
* @returns Array of applicable field patterns
|
||||||
|
*/
|
||||||
|
export declare function getFieldPatterns(entityType: 'noun' | 'verb', specificType?: string): FieldPattern[];
|
||||||
|
/**
|
||||||
|
* Priority fields for different entity types (for AI analysis)
|
||||||
|
* Used by the BrainyTypes and neural processing
|
||||||
|
*/
|
||||||
|
export declare const TYPE_PRIORITY_FIELDS: Record<string, string[]>;
|
||||||
|
/**
|
||||||
|
* Get priority fields for intelligent analysis
|
||||||
|
* @param entityType The type of entity
|
||||||
|
* @param specificType Optional specific type
|
||||||
|
* @returns Array of priority field names
|
||||||
|
*/
|
||||||
|
export declare function getPriorityFields(entityType: 'noun' | 'verb', specificType?: string): string[];
|
||||||
|
/**
|
||||||
|
* Smart field value extraction with type-aware processing
|
||||||
|
* @param data The data object to extract from
|
||||||
|
* @param pattern The field pattern to apply
|
||||||
|
* @param context The computation context
|
||||||
|
* @returns The extracted and processed field value
|
||||||
|
*/
|
||||||
|
export declare function extractFieldValue(data: any, pattern: FieldPattern, context: FieldComputationContext): any;
|
||||||
|
/**
|
||||||
|
* Calculate confidence score for field detection
|
||||||
|
* @param pattern The field pattern
|
||||||
|
* @param context The computation context
|
||||||
|
* @param value The extracted value
|
||||||
|
* @returns Confidence score (0-1)
|
||||||
|
*/
|
||||||
|
export declare function calculateFieldConfidence(pattern: FieldPattern, context: FieldComputationContext, value: any): number;
|
||||||
|
|
@ -0,0 +1,393 @@
|
||||||
|
/**
|
||||||
|
* Universal Display Augmentation - Smart Field Patterns
|
||||||
|
*
|
||||||
|
* Intelligent field detection patterns for mapping user data to display fields
|
||||||
|
* Uses semantic understanding and common naming conventions
|
||||||
|
*/
|
||||||
|
import { NounType } from '../../types/graphTypes.js';
|
||||||
|
/**
|
||||||
|
* Universal field patterns that work across all data types
|
||||||
|
* Ordered by confidence level (highest first)
|
||||||
|
*/
|
||||||
|
export const UNIVERSAL_FIELD_PATTERNS = [
|
||||||
|
// Title/Name Patterns (Highest Priority)
|
||||||
|
{
|
||||||
|
fields: ['name', 'title', 'displayName', 'label', 'heading'],
|
||||||
|
displayField: 'title',
|
||||||
|
confidence: 0.95
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['firstName', 'lastName', 'fullName', 'realName'],
|
||||||
|
displayField: 'title',
|
||||||
|
confidence: 0.9,
|
||||||
|
applicableTypes: [NounType.Person, NounType.User],
|
||||||
|
transform: (value, context) => {
|
||||||
|
const { metadata } = context;
|
||||||
|
if (metadata.firstName && metadata.lastName) {
|
||||||
|
return `${metadata.firstName} ${metadata.lastName}`.trim();
|
||||||
|
}
|
||||||
|
return String(value || '');
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['companyName', 'organizationName', 'orgName', 'businessName'],
|
||||||
|
displayField: 'title',
|
||||||
|
confidence: 0.9,
|
||||||
|
applicableTypes: [NounType.Organization]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['filename', 'fileName', 'documentTitle', 'docName'],
|
||||||
|
displayField: 'title',
|
||||||
|
confidence: 0.85,
|
||||||
|
applicableTypes: [NounType.Document, NounType.File, NounType.Media]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['projectName', 'projectTitle', 'initiative'],
|
||||||
|
displayField: 'title',
|
||||||
|
confidence: 0.9,
|
||||||
|
applicableTypes: [NounType.Project]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['taskName', 'taskTitle', 'action', 'todo'],
|
||||||
|
displayField: 'title',
|
||||||
|
confidence: 0.85,
|
||||||
|
applicableTypes: [NounType.Task]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['subject', 'topic', 'headline', 'caption'],
|
||||||
|
displayField: 'title',
|
||||||
|
confidence: 0.8
|
||||||
|
},
|
||||||
|
// Description Patterns (High Priority)
|
||||||
|
{
|
||||||
|
fields: ['description', 'summary', 'overview', 'details'],
|
||||||
|
displayField: 'description',
|
||||||
|
confidence: 0.9
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['bio', 'biography', 'profile', 'about'],
|
||||||
|
displayField: 'description',
|
||||||
|
confidence: 0.85,
|
||||||
|
applicableTypes: [NounType.Person, NounType.User]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['content', 'text', 'body', 'message'],
|
||||||
|
displayField: 'description',
|
||||||
|
confidence: 0.8
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['abstract', 'excerpt', 'snippet', 'preview'],
|
||||||
|
displayField: 'description',
|
||||||
|
confidence: 0.75
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['notes', 'comments', 'remarks', 'observations'],
|
||||||
|
displayField: 'description',
|
||||||
|
confidence: 0.7
|
||||||
|
},
|
||||||
|
// Type Patterns (Medium Priority)
|
||||||
|
{
|
||||||
|
fields: ['type', 'category', 'classification', 'kind'],
|
||||||
|
displayField: 'type',
|
||||||
|
confidence: 0.9
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['nounType', 'entityType', 'objectType'],
|
||||||
|
displayField: 'type',
|
||||||
|
confidence: 0.95
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['role', 'position', 'jobTitle', 'occupation'],
|
||||||
|
displayField: 'type',
|
||||||
|
confidence: 0.8,
|
||||||
|
applicableTypes: [NounType.Person, NounType.User],
|
||||||
|
transform: (value) => String(value || 'Person')
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['industry', 'sector', 'domain', 'field'],
|
||||||
|
displayField: 'type',
|
||||||
|
confidence: 0.7,
|
||||||
|
applicableTypes: [NounType.Organization]
|
||||||
|
},
|
||||||
|
// Tag Patterns (Medium Priority)
|
||||||
|
{
|
||||||
|
fields: ['tags', 'keywords', 'labels', 'categories'],
|
||||||
|
displayField: 'tags',
|
||||||
|
confidence: 0.85
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['topics', 'subjects', 'themes'],
|
||||||
|
displayField: 'tags',
|
||||||
|
confidence: 0.8
|
||||||
|
}
|
||||||
|
];
|
||||||
|
/**
|
||||||
|
* Type-specific field patterns for enhanced detection
|
||||||
|
* Used when we know the specific type of the entity
|
||||||
|
*/
|
||||||
|
export const TYPE_SPECIFIC_PATTERNS = {
|
||||||
|
[NounType.Person]: [
|
||||||
|
{
|
||||||
|
fields: ['email', 'emailAddress', 'contactEmail'],
|
||||||
|
displayField: 'description',
|
||||||
|
confidence: 0.7,
|
||||||
|
transform: (value, context) => {
|
||||||
|
const { metadata } = context;
|
||||||
|
const role = metadata.role || metadata.jobTitle || metadata.position;
|
||||||
|
const company = metadata.company || metadata.organization || metadata.employer;
|
||||||
|
const parts = [];
|
||||||
|
if (role)
|
||||||
|
parts.push(role);
|
||||||
|
if (company)
|
||||||
|
parts.push(`at ${company}`);
|
||||||
|
if (parts.length === 0 && value)
|
||||||
|
parts.push(`Contact: ${value}`);
|
||||||
|
return parts.join(' ') || 'Person';
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['phone', 'phoneNumber', 'mobile', 'cell'],
|
||||||
|
displayField: 'tags',
|
||||||
|
confidence: 0.6
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[NounType.Organization]: [
|
||||||
|
{
|
||||||
|
fields: ['website', 'url', 'homepage', 'domain'],
|
||||||
|
displayField: 'description',
|
||||||
|
confidence: 0.7,
|
||||||
|
transform: (value, context) => {
|
||||||
|
const { metadata } = context;
|
||||||
|
const industry = metadata.industry || metadata.sector;
|
||||||
|
const location = metadata.location || metadata.city || metadata.country;
|
||||||
|
const parts = [];
|
||||||
|
if (industry)
|
||||||
|
parts.push(industry);
|
||||||
|
parts.push('organization');
|
||||||
|
if (location)
|
||||||
|
parts.push(`in ${location}`);
|
||||||
|
return parts.join(' ');
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
fields: ['employees', 'size', 'headcount'],
|
||||||
|
displayField: 'tags',
|
||||||
|
confidence: 0.6
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[NounType.Project]: [
|
||||||
|
{
|
||||||
|
fields: ['status', 'phase', 'stage', 'state'],
|
||||||
|
displayField: 'description',
|
||||||
|
confidence: 0.8,
|
||||||
|
transform: (value, context) => {
|
||||||
|
const { metadata } = context;
|
||||||
|
const status = String(value || 'active').toLowerCase();
|
||||||
|
const budget = metadata.budget || metadata.cost;
|
||||||
|
const lead = metadata.lead || metadata.manager || metadata.owner;
|
||||||
|
const parts = [];
|
||||||
|
parts.push(status.charAt(0).toUpperCase() + status.slice(1));
|
||||||
|
if (metadata.description)
|
||||||
|
parts.push('project');
|
||||||
|
if (lead)
|
||||||
|
parts.push(`led by ${lead}`);
|
||||||
|
if (budget)
|
||||||
|
parts.push(`($${parseInt(String(budget)).toLocaleString()} budget)`);
|
||||||
|
return parts.join(' ');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[NounType.Document]: [
|
||||||
|
{
|
||||||
|
fields: ['author', 'creator', 'writer'],
|
||||||
|
displayField: 'description',
|
||||||
|
confidence: 0.7,
|
||||||
|
transform: (value, context) => {
|
||||||
|
const { metadata } = context;
|
||||||
|
const docType = metadata.type || metadata.category || 'document';
|
||||||
|
const date = metadata.date || metadata.created || metadata.published;
|
||||||
|
const parts = [];
|
||||||
|
if (docType)
|
||||||
|
parts.push(docType);
|
||||||
|
if (value)
|
||||||
|
parts.push(`by ${value}`);
|
||||||
|
if (date) {
|
||||||
|
const dateStr = new Date(date).toLocaleDateString();
|
||||||
|
parts.push(`(${dateStr})`);
|
||||||
|
}
|
||||||
|
return parts.join(' ');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[NounType.Task]: [
|
||||||
|
{
|
||||||
|
fields: ['priority', 'urgency', 'importance'],
|
||||||
|
displayField: 'tags',
|
||||||
|
confidence: 0.7
|
||||||
|
}
|
||||||
|
]
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Get field patterns for a specific entity type
|
||||||
|
* @param entityType The type of entity (noun or verb)
|
||||||
|
* @param specificType Optional specific noun/verb type
|
||||||
|
* @returns Array of applicable field patterns
|
||||||
|
*/
|
||||||
|
export function getFieldPatterns(entityType, specificType) {
|
||||||
|
const patterns = [...UNIVERSAL_FIELD_PATTERNS];
|
||||||
|
if (entityType === 'noun' && specificType && TYPE_SPECIFIC_PATTERNS[specificType]) {
|
||||||
|
patterns.unshift(...TYPE_SPECIFIC_PATTERNS[specificType]);
|
||||||
|
}
|
||||||
|
return patterns.sort((a, b) => b.confidence - a.confidence);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Priority fields for different entity types (for AI analysis)
|
||||||
|
* Used by the BrainyTypes and neural processing
|
||||||
|
*/
|
||||||
|
export const TYPE_PRIORITY_FIELDS = {
|
||||||
|
[NounType.Person]: [
|
||||||
|
'name', 'firstName', 'lastName', 'fullName', 'displayName',
|
||||||
|
'email', 'role', 'jobTitle', 'position', 'title',
|
||||||
|
'bio', 'description', 'about', 'profile',
|
||||||
|
'company', 'organization', 'employer'
|
||||||
|
],
|
||||||
|
[NounType.Organization]: [
|
||||||
|
'name', 'companyName', 'organizationName', 'title',
|
||||||
|
'industry', 'sector', 'domain', 'type',
|
||||||
|
'description', 'about', 'summary',
|
||||||
|
'location', 'city', 'country', 'headquarters',
|
||||||
|
'website', 'url'
|
||||||
|
],
|
||||||
|
[NounType.Project]: [
|
||||||
|
'name', 'projectName', 'title', 'projectTitle',
|
||||||
|
'description', 'summary', 'overview', 'goal',
|
||||||
|
'status', 'phase', 'stage', 'state',
|
||||||
|
'lead', 'manager', 'owner', 'team',
|
||||||
|
'budget', 'timeline', 'deadline'
|
||||||
|
],
|
||||||
|
[NounType.Document]: [
|
||||||
|
'title', 'filename', 'name', 'subject',
|
||||||
|
'content', 'text', 'body', 'summary',
|
||||||
|
'author', 'creator', 'writer',
|
||||||
|
'type', 'category', 'format',
|
||||||
|
'date', 'created', 'published'
|
||||||
|
],
|
||||||
|
[NounType.Task]: [
|
||||||
|
'title', 'name', 'taskName', 'action',
|
||||||
|
'description', 'details', 'notes',
|
||||||
|
'status', 'state', 'priority',
|
||||||
|
'assignee', 'owner', 'responsible',
|
||||||
|
'due', 'deadline', 'dueDate'
|
||||||
|
],
|
||||||
|
[NounType.Event]: [
|
||||||
|
'name', 'title', 'eventName',
|
||||||
|
'description', 'details', 'summary',
|
||||||
|
'startDate', 'endDate', 'date', 'time',
|
||||||
|
'location', 'venue', 'address',
|
||||||
|
'organizer', 'host', 'creator'
|
||||||
|
],
|
||||||
|
[NounType.Product]: [
|
||||||
|
'name', 'productName', 'title',
|
||||||
|
'description', 'summary', 'features',
|
||||||
|
'price', 'cost', 'value',
|
||||||
|
'category', 'type', 'brand',
|
||||||
|
'manufacturer', 'vendor'
|
||||||
|
]
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Get priority fields for intelligent analysis
|
||||||
|
* @param entityType The type of entity
|
||||||
|
* @param specificType Optional specific type
|
||||||
|
* @returns Array of priority field names
|
||||||
|
*/
|
||||||
|
export function getPriorityFields(entityType, specificType) {
|
||||||
|
if (entityType === 'noun' && specificType && TYPE_PRIORITY_FIELDS[specificType]) {
|
||||||
|
return TYPE_PRIORITY_FIELDS[specificType];
|
||||||
|
}
|
||||||
|
// Default priority fields for any entity
|
||||||
|
return [
|
||||||
|
'name', 'title', 'label', 'displayName',
|
||||||
|
'description', 'summary', 'about', 'details',
|
||||||
|
'type', 'category', 'kind', 'classification',
|
||||||
|
'tags', 'keywords', 'labels'
|
||||||
|
];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Smart field value extraction with type-aware processing
|
||||||
|
* @param data The data object to extract from
|
||||||
|
* @param pattern The field pattern to apply
|
||||||
|
* @param context The computation context
|
||||||
|
* @returns The extracted and processed field value
|
||||||
|
*/
|
||||||
|
export function extractFieldValue(data, pattern, context) {
|
||||||
|
// Find the first matching field
|
||||||
|
let value = null;
|
||||||
|
let matchedField = null;
|
||||||
|
for (const field of pattern.fields) {
|
||||||
|
if (data[field] !== undefined && data[field] !== null && data[field] !== '') {
|
||||||
|
value = data[field];
|
||||||
|
matchedField = field;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (value === null)
|
||||||
|
return null;
|
||||||
|
// Apply transformation if provided
|
||||||
|
if (pattern.transform) {
|
||||||
|
try {
|
||||||
|
return pattern.transform(value, context);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.warn(`Field transformation error for ${matchedField}:`, error);
|
||||||
|
return String(value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Default processing based on display field type
|
||||||
|
switch (pattern.displayField) {
|
||||||
|
case 'title':
|
||||||
|
case 'description':
|
||||||
|
case 'type':
|
||||||
|
return String(value);
|
||||||
|
case 'tags':
|
||||||
|
if (Array.isArray(value))
|
||||||
|
return value;
|
||||||
|
if (typeof value === 'string') {
|
||||||
|
return value.split(/[,;]\s*|\s+/).filter(Boolean);
|
||||||
|
}
|
||||||
|
return [String(value)];
|
||||||
|
default:
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Calculate confidence score for field detection
|
||||||
|
* @param pattern The field pattern
|
||||||
|
* @param context The computation context
|
||||||
|
* @param value The extracted value
|
||||||
|
* @returns Confidence score (0-1)
|
||||||
|
*/
|
||||||
|
export function calculateFieldConfidence(pattern, context, value) {
|
||||||
|
let confidence = pattern.confidence;
|
||||||
|
// Boost confidence if type matches
|
||||||
|
if (pattern.applicableTypes && context.typeResult) {
|
||||||
|
if (pattern.applicableTypes.includes(context.typeResult.type)) {
|
||||||
|
confidence = Math.min(1.0, confidence + 0.1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Reduce confidence for empty or very short values
|
||||||
|
if (typeof value === 'string') {
|
||||||
|
if (value.length < 2) {
|
||||||
|
confidence *= 0.5;
|
||||||
|
}
|
||||||
|
else if (value.length < 5) {
|
||||||
|
confidence *= 0.8;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Reduce confidence for generic values
|
||||||
|
const genericValues = ['unknown', 'n/a', 'null', 'undefined', 'default'];
|
||||||
|
if (typeof value === 'string' && genericValues.includes(value.toLowerCase())) {
|
||||||
|
confidence *= 0.3;
|
||||||
|
}
|
||||||
|
return Math.max(0, Math.min(1, confidence));
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=fieldPatterns.js.map
|
||||||
File diff suppressed because one or more lines are too long
57
.recovery-workspace/dist-backup-20250910-141917/augmentations/display/iconMappings.d.ts
vendored
Normal file
57
.recovery-workspace/dist-backup-20250910-141917/augmentations/display/iconMappings.d.ts
vendored
Normal file
|
|
@ -0,0 +1,57 @@
|
||||||
|
/**
|
||||||
|
* Universal Display Augmentation - Clean Display
|
||||||
|
*
|
||||||
|
* Simple, clean display without icons - focusing on AI-powered
|
||||||
|
* titles, descriptions, and smart formatting that matches
|
||||||
|
* Soulcraft's minimal aesthetic
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* No icon mappings - clean, minimal approach
|
||||||
|
* The real value is in AI-generated titles and enhanced descriptions,
|
||||||
|
* not visual clutter that doesn't align with professional aesthetics
|
||||||
|
*/
|
||||||
|
export declare const NOUN_TYPE_ICONS: Record<string, string>;
|
||||||
|
/**
|
||||||
|
* No icon mappings for verbs either - focus on clear relationship descriptions
|
||||||
|
* Human-readable relationship text is more valuable than symbolic representations
|
||||||
|
*/
|
||||||
|
export declare const VERB_TYPE_ICONS: Record<string, string>;
|
||||||
|
/**
|
||||||
|
* Get icon for a noun type (returns empty string for clean display)
|
||||||
|
* @param type The noun type
|
||||||
|
* @returns Empty string (no icons)
|
||||||
|
*/
|
||||||
|
export declare function getNounIcon(type: string): string;
|
||||||
|
/**
|
||||||
|
* Get icon for a verb type (returns empty string for clean display)
|
||||||
|
* @param type The verb type
|
||||||
|
* @returns Empty string (no icons)
|
||||||
|
*/
|
||||||
|
export declare function getVerbIcon(type: string): string;
|
||||||
|
/**
|
||||||
|
* Get coverage statistics (for backwards compatibility)
|
||||||
|
* @returns Coverage info showing clean approach
|
||||||
|
*/
|
||||||
|
export declare function getIconCoverage(): {
|
||||||
|
nounTypes: {
|
||||||
|
total: string;
|
||||||
|
covered: string;
|
||||||
|
};
|
||||||
|
verbTypes: {
|
||||||
|
total: string;
|
||||||
|
covered: string;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Check if an icon exists for a type (always false for clean display)
|
||||||
|
* @param type The type to check
|
||||||
|
* @param entityType Whether it's a noun or verb
|
||||||
|
* @returns Always false (no icons)
|
||||||
|
*/
|
||||||
|
export declare function hasIcon(type: string, entityType?: 'noun' | 'verb'): boolean;
|
||||||
|
/**
|
||||||
|
* Get fallback icon (returns empty string for clean display)
|
||||||
|
* @param entityType The entity type
|
||||||
|
* @returns Empty string (no fallback icons)
|
||||||
|
*/
|
||||||
|
export declare function getFallbackIcon(entityType?: 'noun' | 'verb'): string;
|
||||||
|
|
@ -0,0 +1,68 @@
|
||||||
|
/**
|
||||||
|
* Universal Display Augmentation - Clean Display
|
||||||
|
*
|
||||||
|
* Simple, clean display without icons - focusing on AI-powered
|
||||||
|
* titles, descriptions, and smart formatting that matches
|
||||||
|
* Soulcraft's minimal aesthetic
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* No icon mappings - clean, minimal approach
|
||||||
|
* The real value is in AI-generated titles and enhanced descriptions,
|
||||||
|
* not visual clutter that doesn't align with professional aesthetics
|
||||||
|
*/
|
||||||
|
export const NOUN_TYPE_ICONS = {};
|
||||||
|
/**
|
||||||
|
* No icon mappings for verbs either - focus on clear relationship descriptions
|
||||||
|
* Human-readable relationship text is more valuable than symbolic representations
|
||||||
|
*/
|
||||||
|
export const VERB_TYPE_ICONS = {};
|
||||||
|
/**
|
||||||
|
* Get icon for a noun type (returns empty string for clean display)
|
||||||
|
* @param type The noun type
|
||||||
|
* @returns Empty string (no icons)
|
||||||
|
*/
|
||||||
|
export function getNounIcon(type) {
|
||||||
|
return ''; // Clean, no icons
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get icon for a verb type (returns empty string for clean display)
|
||||||
|
* @param type The verb type
|
||||||
|
* @returns Empty string (no icons)
|
||||||
|
*/
|
||||||
|
export function getVerbIcon(type) {
|
||||||
|
return ''; // Clean, no icons
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get coverage statistics (for backwards compatibility)
|
||||||
|
* @returns Coverage info showing clean approach
|
||||||
|
*/
|
||||||
|
export function getIconCoverage() {
|
||||||
|
return {
|
||||||
|
nounTypes: {
|
||||||
|
total: 'Clean display - no icons needed',
|
||||||
|
covered: 'Focus on AI-powered content'
|
||||||
|
},
|
||||||
|
verbTypes: {
|
||||||
|
total: 'Clean display - no icons needed',
|
||||||
|
covered: 'Focus on relationship descriptions'
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if an icon exists for a type (always false for clean display)
|
||||||
|
* @param type The type to check
|
||||||
|
* @param entityType Whether it's a noun or verb
|
||||||
|
* @returns Always false (no icons)
|
||||||
|
*/
|
||||||
|
export function hasIcon(type, entityType = 'noun') {
|
||||||
|
return false; // Clean approach - no icons
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get fallback icon (returns empty string for clean display)
|
||||||
|
* @param entityType The entity type
|
||||||
|
* @returns Empty string (no fallback icons)
|
||||||
|
*/
|
||||||
|
export function getFallbackIcon(entityType = 'noun') {
|
||||||
|
return ''; // Clean, minimal display
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=iconMappings.js.map
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
{"version":3,"file":"iconMappings.js","sourceRoot":"","sources":["../../../src/augmentations/display/iconMappings.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAIH;;;;GAIG;AACH,MAAM,CAAC,MAAM,eAAe,GAA2B,EAAE,CAAA;AAEzD;;;GAGG;AACH,MAAM,CAAC,MAAM,eAAe,GAA2B,EAAE,CAAA;AAEzD;;;;GAIG;AACH,MAAM,UAAU,WAAW,CAAC,IAAY;IACtC,OAAO,EAAE,CAAA,CAAC,kBAAkB;AAC9B,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,WAAW,CAAC,IAAY;IACtC,OAAO,EAAE,CAAA,CAAC,kBAAkB;AAC9B,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,eAAe;IAC7B,OAAO;QACL,SAAS,EAAE;YACT,KAAK,EAAE,iCAAiC;YACxC,OAAO,EAAE,6BAA6B;SACvC;QACD,SAAS,EAAE;YACT,KAAK,EAAE,iCAAiC;YACxC,OAAO,EAAE,oCAAoC;SAC9C;KACF,CAAA;AACH,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,OAAO,CAAC,IAAY,EAAE,aAA8B,MAAM;IACxE,OAAO,KAAK,CAAA,CAAC,4BAA4B;AAC3C,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,eAAe,CAAC,aAA8B,MAAM;IAClE,OAAO,EAAE,CAAA,CAAC,yBAAyB;AACrC,CAAC"}
|
||||||
109
.recovery-workspace/dist-backup-20250910-141917/augmentations/display/intelligentComputation.d.ts
vendored
Normal file
109
.recovery-workspace/dist-backup-20250910-141917/augmentations/display/intelligentComputation.d.ts
vendored
Normal file
|
|
@ -0,0 +1,109 @@
|
||||||
|
/**
|
||||||
|
* Universal Display Augmentation - Intelligent Computation Engine
|
||||||
|
*
|
||||||
|
* Leverages existing Brainy AI infrastructure for intelligent field computation:
|
||||||
|
* - BrainyTypes for semantic type detection
|
||||||
|
* - Neural Import patterns for field analysis
|
||||||
|
* - JSON processing utilities for field extraction
|
||||||
|
* - Existing NounType/VerbType taxonomy (31+40 types)
|
||||||
|
*/
|
||||||
|
import type { ComputedDisplayFields, DisplayConfig } from './types.js';
|
||||||
|
import type { GraphVerb } from '../../coreTypes.js';
|
||||||
|
/**
|
||||||
|
* Intelligent field computation engine
|
||||||
|
* Coordinates AI-powered analysis with fallback heuristics
|
||||||
|
*/
|
||||||
|
export declare class IntelligentComputationEngine {
|
||||||
|
private typeMatcher;
|
||||||
|
protected config: DisplayConfig;
|
||||||
|
private initialized;
|
||||||
|
constructor(config: DisplayConfig);
|
||||||
|
/**
|
||||||
|
* Initialize the computation engine with AI components
|
||||||
|
*/
|
||||||
|
initialize(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Compute display fields for a noun using AI-first approach
|
||||||
|
* @param data The noun data/metadata
|
||||||
|
* @param id Optional noun ID
|
||||||
|
* @returns Computed display fields
|
||||||
|
*/
|
||||||
|
computeNounDisplay(data: any, id?: string): Promise<ComputedDisplayFields>;
|
||||||
|
/**
|
||||||
|
* Compute display fields for a verb using AI-first approach
|
||||||
|
* @param verb The verb/relationship data
|
||||||
|
* @returns Computed display fields
|
||||||
|
*/
|
||||||
|
computeVerbDisplay(verb: GraphVerb): Promise<ComputedDisplayFields>;
|
||||||
|
/**
|
||||||
|
* AI-powered computation using your existing BrainyTypes
|
||||||
|
* @param data Entity data/metadata
|
||||||
|
* @param entityType Type of entity (noun/verb)
|
||||||
|
* @param options Additional options
|
||||||
|
* @returns AI-computed display fields
|
||||||
|
*/
|
||||||
|
private computeWithAI;
|
||||||
|
/**
|
||||||
|
* AI-powered verb computation using relationship analysis
|
||||||
|
* @param verb The verb/relationship
|
||||||
|
* @returns AI-computed display fields
|
||||||
|
*/
|
||||||
|
private computeVerbWithAI;
|
||||||
|
/**
|
||||||
|
* Heuristic computation when AI is unavailable
|
||||||
|
* @param data Entity data
|
||||||
|
* @param entityType Type of entity
|
||||||
|
* @param options Additional options
|
||||||
|
* @returns Heuristically computed display fields
|
||||||
|
*/
|
||||||
|
private computeWithHeuristics;
|
||||||
|
/**
|
||||||
|
* Compute intelligent title using AI insights and your field extraction
|
||||||
|
* @param context Computation context with AI results
|
||||||
|
* @returns Computed title
|
||||||
|
*/
|
||||||
|
private computeIntelligentTitle;
|
||||||
|
/**
|
||||||
|
* Compute intelligent description using AI insights and context
|
||||||
|
* @param context Computation context
|
||||||
|
* @returns Enhanced description
|
||||||
|
*/
|
||||||
|
private computeIntelligentDescription;
|
||||||
|
/**
|
||||||
|
* Compute intelligent tags using type analysis
|
||||||
|
* @param context Computation context
|
||||||
|
* @returns Generated tags array
|
||||||
|
*/
|
||||||
|
private computeIntelligentTags;
|
||||||
|
/**
|
||||||
|
* Compute verb title (relationship summary)
|
||||||
|
* @param context Verb computation context
|
||||||
|
* @returns Verb title
|
||||||
|
*/
|
||||||
|
private computeVerbTitle;
|
||||||
|
/**
|
||||||
|
* Create minimal display for error cases
|
||||||
|
* @param data Entity data
|
||||||
|
* @param entityType Entity type
|
||||||
|
* @returns Minimal display fields
|
||||||
|
*/
|
||||||
|
private createMinimalDisplay;
|
||||||
|
private computePersonTitle;
|
||||||
|
private computeOrganizationTitle;
|
||||||
|
private computeProjectTitle;
|
||||||
|
private computeDocumentTitle;
|
||||||
|
private extractBestTitle;
|
||||||
|
private createContextAwareDescription;
|
||||||
|
private extractExplicitTags;
|
||||||
|
private generateSemanticTags;
|
||||||
|
private getReadableVerbPhrase;
|
||||||
|
private computeVerbDescription;
|
||||||
|
private computeVerbTags;
|
||||||
|
private computeHumanReadableRelationship;
|
||||||
|
private detectTypeHeuristically;
|
||||||
|
private extractFieldWithPatterns;
|
||||||
|
/**
|
||||||
|
* Shutdown the computation engine
|
||||||
|
*/
|
||||||
|
shutdown(): Promise<void>;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,462 @@
|
||||||
|
/**
|
||||||
|
* Universal Display Augmentation - Intelligent Computation Engine
|
||||||
|
*
|
||||||
|
* Leverages existing Brainy AI infrastructure for intelligent field computation:
|
||||||
|
* - BrainyTypes for semantic type detection
|
||||||
|
* - Neural Import patterns for field analysis
|
||||||
|
* - JSON processing utilities for field extraction
|
||||||
|
* - Existing NounType/VerbType taxonomy (31+40 types)
|
||||||
|
*/
|
||||||
|
import { getBrainyTypes } from '../typeMatching/brainyTypes.js';
|
||||||
|
import { getFieldPatterns, getPriorityFields } from './fieldPatterns.js';
|
||||||
|
import { prepareJsonForVectorization } from '../../utils/jsonProcessing.js';
|
||||||
|
import { NounType, VerbType } from '../../types/graphTypes.js';
|
||||||
|
/**
|
||||||
|
* Intelligent field computation engine
|
||||||
|
* Coordinates AI-powered analysis with fallback heuristics
|
||||||
|
*/
|
||||||
|
export class IntelligentComputationEngine {
|
||||||
|
constructor(config) {
|
||||||
|
this.typeMatcher = null;
|
||||||
|
this.initialized = false;
|
||||||
|
this.config = config;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Initialize the computation engine with AI components
|
||||||
|
*/
|
||||||
|
async initialize() {
|
||||||
|
if (this.initialized)
|
||||||
|
return;
|
||||||
|
try {
|
||||||
|
// 🧠 LEVERAGE YOUR EXISTING AI INFRASTRUCTURE
|
||||||
|
this.typeMatcher = await getBrainyTypes();
|
||||||
|
if (this.typeMatcher) {
|
||||||
|
console.log('🎨 Display computation engine initialized with AI intelligence');
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
console.warn('🎨 Display computation engine running in basic mode (AI unavailable)');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.warn('🎨 AI initialization failed, using heuristic fallback:', error);
|
||||||
|
}
|
||||||
|
this.initialized = true;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Compute display fields for a noun using AI-first approach
|
||||||
|
* @param data The noun data/metadata
|
||||||
|
* @param id Optional noun ID
|
||||||
|
* @returns Computed display fields
|
||||||
|
*/
|
||||||
|
async computeNounDisplay(data, id) {
|
||||||
|
const startTime = Date.now();
|
||||||
|
try {
|
||||||
|
// 🟢 PRIMARY PATH: Use your existing AI intelligence
|
||||||
|
if (this.typeMatcher) {
|
||||||
|
return await this.computeWithAI(data, 'noun', { id });
|
||||||
|
}
|
||||||
|
// 🟡 FALLBACK PATH: Use heuristic patterns
|
||||||
|
return await this.computeWithHeuristics(data, 'noun', { id });
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.warn('Display computation failed, using minimal fallback:', error);
|
||||||
|
return this.createMinimalDisplay(data, 'noun');
|
||||||
|
}
|
||||||
|
finally {
|
||||||
|
const computationTime = Date.now() - startTime;
|
||||||
|
if (this.config.debugMode) {
|
||||||
|
console.log(`Display computation took ${computationTime}ms`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Compute display fields for a verb using AI-first approach
|
||||||
|
* @param verb The verb/relationship data
|
||||||
|
* @returns Computed display fields
|
||||||
|
*/
|
||||||
|
async computeVerbDisplay(verb) {
|
||||||
|
const startTime = Date.now();
|
||||||
|
try {
|
||||||
|
// 🟢 PRIMARY PATH: Use your existing AI for verb analysis
|
||||||
|
if (this.typeMatcher) {
|
||||||
|
return await this.computeVerbWithAI(verb);
|
||||||
|
}
|
||||||
|
// 🟡 FALLBACK PATH: Use heuristic patterns for verbs
|
||||||
|
return await this.computeWithHeuristics(verb, 'verb');
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.warn('Verb display computation failed, using minimal fallback:', error);
|
||||||
|
return this.createMinimalDisplay(verb, 'verb');
|
||||||
|
}
|
||||||
|
finally {
|
||||||
|
const computationTime = Date.now() - startTime;
|
||||||
|
if (this.config.debugMode) {
|
||||||
|
console.log(`Verb display computation took ${computationTime}ms`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* AI-powered computation using your existing BrainyTypes
|
||||||
|
* @param data Entity data/metadata
|
||||||
|
* @param entityType Type of entity (noun/verb)
|
||||||
|
* @param options Additional options
|
||||||
|
* @returns AI-computed display fields
|
||||||
|
*/
|
||||||
|
async computeWithAI(data, entityType, options = {}) {
|
||||||
|
// 🧠 USE YOUR EXISTING TYPE DETECTION AI
|
||||||
|
const typeResult = await this.typeMatcher.matchNounType(data);
|
||||||
|
// Create computation context
|
||||||
|
const context = {
|
||||||
|
data,
|
||||||
|
metadata: data,
|
||||||
|
typeResult,
|
||||||
|
config: this.config,
|
||||||
|
entityType
|
||||||
|
};
|
||||||
|
// 🟢 INTELLIGENT FIELD EXTRACTION using your patterns + AI insights
|
||||||
|
const displayFields = {
|
||||||
|
title: await this.computeIntelligentTitle(context),
|
||||||
|
description: await this.computeIntelligentDescription(context),
|
||||||
|
type: typeResult.type,
|
||||||
|
tags: await this.computeIntelligentTags(context),
|
||||||
|
confidence: typeResult.confidence,
|
||||||
|
reasoning: this.config.debugMode ? typeResult.reasoning : undefined,
|
||||||
|
alternatives: this.config.debugMode ? typeResult.alternatives : undefined,
|
||||||
|
computedAt: Date.now(),
|
||||||
|
version: '1.0.0'
|
||||||
|
};
|
||||||
|
return displayFields;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* AI-powered verb computation using relationship analysis
|
||||||
|
* @param verb The verb/relationship
|
||||||
|
* @returns AI-computed display fields
|
||||||
|
*/
|
||||||
|
async computeVerbWithAI(verb) {
|
||||||
|
// 🧠 USE YOUR EXISTING VERB TYPE DETECTION
|
||||||
|
const typeResult = await this.typeMatcher.matchVerbType(verb, 0.7);
|
||||||
|
// Create verb computation context
|
||||||
|
const context = {
|
||||||
|
data: verb,
|
||||||
|
metadata: verb.metadata || {},
|
||||||
|
typeResult,
|
||||||
|
config: this.config,
|
||||||
|
entityType: 'verb',
|
||||||
|
verbContext: {
|
||||||
|
sourceId: verb.sourceId,
|
||||||
|
targetId: verb.targetId,
|
||||||
|
verbType: verb.type
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// 🟢 INTELLIGENT VERB DISPLAY COMPUTATION
|
||||||
|
const displayFields = {
|
||||||
|
title: await this.computeVerbTitle(context),
|
||||||
|
description: await this.computeVerbDescription(context),
|
||||||
|
type: typeResult.type,
|
||||||
|
tags: await this.computeVerbTags(context),
|
||||||
|
relationship: await this.computeHumanReadableRelationship(context),
|
||||||
|
confidence: typeResult.confidence,
|
||||||
|
reasoning: this.config.debugMode ? typeResult.reasoning : undefined,
|
||||||
|
alternatives: this.config.debugMode ? typeResult.alternatives : undefined,
|
||||||
|
computedAt: Date.now(),
|
||||||
|
version: '1.0.0'
|
||||||
|
};
|
||||||
|
return displayFields;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Heuristic computation when AI is unavailable
|
||||||
|
* @param data Entity data
|
||||||
|
* @param entityType Type of entity
|
||||||
|
* @param options Additional options
|
||||||
|
* @returns Heuristically computed display fields
|
||||||
|
*/
|
||||||
|
async computeWithHeuristics(data, entityType, options = {}) {
|
||||||
|
// Use basic type detection
|
||||||
|
const detectedType = this.detectTypeHeuristically(data, entityType);
|
||||||
|
const typeResult = {
|
||||||
|
type: detectedType,
|
||||||
|
confidence: 0.6, // Lower confidence for heuristics
|
||||||
|
reasoning: 'Heuristic detection (AI unavailable)',
|
||||||
|
alternatives: []
|
||||||
|
};
|
||||||
|
const context = {
|
||||||
|
data,
|
||||||
|
metadata: data,
|
||||||
|
typeResult: typeResult,
|
||||||
|
config: this.config,
|
||||||
|
entityType
|
||||||
|
};
|
||||||
|
// Use pattern-based field extraction
|
||||||
|
const patterns = getFieldPatterns(entityType, detectedType);
|
||||||
|
return {
|
||||||
|
title: this.extractFieldWithPatterns(data, patterns, 'title') || 'Untitled',
|
||||||
|
description: this.extractFieldWithPatterns(data, patterns, 'description') || 'No description',
|
||||||
|
type: detectedType,
|
||||||
|
tags: this.extractFieldWithPatterns(data, patterns, 'tags') || [],
|
||||||
|
confidence: typeResult.confidence,
|
||||||
|
reasoning: this.config.debugMode ? typeResult.reasoning : undefined,
|
||||||
|
computedAt: Date.now(),
|
||||||
|
version: '1.0.0'
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Compute intelligent title using AI insights and your field extraction
|
||||||
|
* @param context Computation context with AI results
|
||||||
|
* @returns Computed title
|
||||||
|
*/
|
||||||
|
async computeIntelligentTitle(context) {
|
||||||
|
const { data, typeResult } = context;
|
||||||
|
// 🟢 USE TYPE-SPECIFIC LOGIC based on your NounType taxonomy
|
||||||
|
switch (typeResult?.type) {
|
||||||
|
case NounType.Person:
|
||||||
|
return this.computePersonTitle(data);
|
||||||
|
case NounType.Organization:
|
||||||
|
return this.computeOrganizationTitle(data);
|
||||||
|
case NounType.Project:
|
||||||
|
return this.computeProjectTitle(data);
|
||||||
|
case NounType.Document:
|
||||||
|
return this.computeDocumentTitle(data);
|
||||||
|
default:
|
||||||
|
// 🟢 LEVERAGE YOUR JSON PROCESSING for unknown types
|
||||||
|
return this.extractBestTitle(data, typeResult?.type);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Compute intelligent description using AI insights and context
|
||||||
|
* @param context Computation context
|
||||||
|
* @returns Enhanced description
|
||||||
|
*/
|
||||||
|
async computeIntelligentDescription(context) {
|
||||||
|
const { data, typeResult } = context;
|
||||||
|
// 🟢 USE YOUR EXISTING JSON PROCESSING for vectorization-quality text
|
||||||
|
const priorityFields = getPriorityFields('noun', typeResult?.type);
|
||||||
|
const enhancedText = prepareJsonForVectorization(data, {
|
||||||
|
priorityFields,
|
||||||
|
includeFieldNames: false,
|
||||||
|
maxDepth: 2
|
||||||
|
});
|
||||||
|
// Create context-aware description based on type
|
||||||
|
return this.createContextAwareDescription(data, typeResult, enhancedText);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Compute intelligent tags using type analysis
|
||||||
|
* @param context Computation context
|
||||||
|
* @returns Generated tags array
|
||||||
|
*/
|
||||||
|
async computeIntelligentTags(context) {
|
||||||
|
const { data, typeResult } = context;
|
||||||
|
const tags = [];
|
||||||
|
// Add type-based tag
|
||||||
|
if (typeResult?.type) {
|
||||||
|
tags.push(typeResult.type.toLowerCase());
|
||||||
|
}
|
||||||
|
// Extract explicit tags from data
|
||||||
|
const explicitTags = this.extractExplicitTags(data);
|
||||||
|
tags.push(...explicitTags);
|
||||||
|
// Add semantic tags based on AI analysis
|
||||||
|
if (typeResult && this.typeMatcher) {
|
||||||
|
const semanticTags = this.generateSemanticTags(data, typeResult);
|
||||||
|
tags.push(...semanticTags);
|
||||||
|
}
|
||||||
|
// Remove duplicates and return
|
||||||
|
return [...new Set(tags.filter(Boolean))];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Compute verb title (relationship summary)
|
||||||
|
* @param context Verb computation context
|
||||||
|
* @returns Verb title
|
||||||
|
*/
|
||||||
|
async computeVerbTitle(context) {
|
||||||
|
const { verbContext, typeResult } = context;
|
||||||
|
if (!verbContext)
|
||||||
|
return 'Relationship';
|
||||||
|
const { sourceId, targetId } = verbContext;
|
||||||
|
const relationshipType = typeResult?.type || 'RelatedTo';
|
||||||
|
// Try to get readable names for source and target
|
||||||
|
// This could be enhanced to actually resolve the entities
|
||||||
|
return `${sourceId} ${this.getReadableVerbPhrase(relationshipType)} ${targetId}`;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Create minimal display for error cases
|
||||||
|
* @param data Entity data
|
||||||
|
* @param entityType Entity type
|
||||||
|
* @returns Minimal display fields
|
||||||
|
*/
|
||||||
|
createMinimalDisplay(data, entityType) {
|
||||||
|
return {
|
||||||
|
title: data.name || data.title || data.id || 'Untitled',
|
||||||
|
description: data.description || data.summary || 'No description available',
|
||||||
|
type: entityType === 'noun' ? 'Item' : 'RelatedTo',
|
||||||
|
tags: [],
|
||||||
|
confidence: 0.1, // Very low confidence for fallback
|
||||||
|
computedAt: Date.now(),
|
||||||
|
version: '1.0.0'
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// Helper methods for specific noun types
|
||||||
|
computePersonTitle(data) {
|
||||||
|
if (data.firstName && data.lastName) {
|
||||||
|
return `${data.firstName} ${data.lastName}`.trim();
|
||||||
|
}
|
||||||
|
return data.name || data.fullName || data.displayName || data.firstName || data.lastName || 'Person';
|
||||||
|
}
|
||||||
|
computeOrganizationTitle(data) {
|
||||||
|
return data.name || data.companyName || data.organizationName || data.title || 'Organization';
|
||||||
|
}
|
||||||
|
computeProjectTitle(data) {
|
||||||
|
return data.name || data.projectName || data.title || data.projectTitle || 'Project';
|
||||||
|
}
|
||||||
|
computeDocumentTitle(data) {
|
||||||
|
return data.title || data.filename || data.name || data.subject || 'Document';
|
||||||
|
}
|
||||||
|
extractBestTitle(data, type) {
|
||||||
|
const titleFields = ['name', 'title', 'displayName', 'label', 'subject', 'heading'];
|
||||||
|
for (const field of titleFields) {
|
||||||
|
if (data[field])
|
||||||
|
return String(data[field]);
|
||||||
|
}
|
||||||
|
return data.id || Object.keys(data)[0] || 'Untitled';
|
||||||
|
}
|
||||||
|
createContextAwareDescription(data, typeResult, enhancedText) {
|
||||||
|
// Start with basic description fields
|
||||||
|
const basicDesc = data.description || data.summary || data.about || data.details;
|
||||||
|
if (basicDesc)
|
||||||
|
return String(basicDesc);
|
||||||
|
// Use enhanced text from JSON processing
|
||||||
|
if (enhancedText && enhancedText.length > 10) {
|
||||||
|
return enhancedText.substring(0, 200) + (enhancedText.length > 200 ? '...' : '');
|
||||||
|
}
|
||||||
|
// Generate from available fields
|
||||||
|
const parts = [];
|
||||||
|
if (data.role)
|
||||||
|
parts.push(data.role);
|
||||||
|
if (data.company)
|
||||||
|
parts.push(`at ${data.company}`);
|
||||||
|
if (data.location)
|
||||||
|
parts.push(`in ${data.location}`);
|
||||||
|
return parts.length > 0 ? parts.join(' ') : 'No description available';
|
||||||
|
}
|
||||||
|
extractExplicitTags(data) {
|
||||||
|
const tagFields = ['tags', 'keywords', 'labels', 'categories', 'topics'];
|
||||||
|
for (const field of tagFields) {
|
||||||
|
if (data[field]) {
|
||||||
|
if (Array.isArray(data[field])) {
|
||||||
|
return data[field].map(String).filter(Boolean);
|
||||||
|
}
|
||||||
|
if (typeof data[field] === 'string') {
|
||||||
|
return data[field].split(/[,;]\s*|\s+/).filter(Boolean);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
generateSemanticTags(data, typeResult) {
|
||||||
|
const tags = [];
|
||||||
|
// Add confidence-based tags
|
||||||
|
if (typeResult.confidence > 0.9)
|
||||||
|
tags.push('verified');
|
||||||
|
else if (typeResult.confidence < 0.7)
|
||||||
|
tags.push('uncertain');
|
||||||
|
// Add type-specific semantic tags
|
||||||
|
if (data.status)
|
||||||
|
tags.push(String(data.status).toLowerCase());
|
||||||
|
if (data.priority)
|
||||||
|
tags.push(String(data.priority).toLowerCase());
|
||||||
|
if (data.category)
|
||||||
|
tags.push(String(data.category).toLowerCase());
|
||||||
|
return tags;
|
||||||
|
}
|
||||||
|
getReadableVerbPhrase(verbType) {
|
||||||
|
const verbPhrases = {
|
||||||
|
[VerbType.WorksWith]: 'works with',
|
||||||
|
[VerbType.MemberOf]: 'is member of',
|
||||||
|
[VerbType.ReportsTo]: 'reports to',
|
||||||
|
[VerbType.CreatedBy]: 'created by',
|
||||||
|
[VerbType.Owns]: 'owns',
|
||||||
|
[VerbType.LocatedAt]: 'located at',
|
||||||
|
[VerbType.Likes]: 'likes',
|
||||||
|
[VerbType.Follows]: 'follows',
|
||||||
|
[VerbType.Supervises]: 'supervises'
|
||||||
|
};
|
||||||
|
return verbPhrases[verbType] || 'related to';
|
||||||
|
}
|
||||||
|
async computeVerbDescription(context) {
|
||||||
|
const { data, verbContext, typeResult } = context;
|
||||||
|
if (data.description)
|
||||||
|
return String(data.description);
|
||||||
|
// Generate contextual description for relationship
|
||||||
|
if (verbContext && typeResult) {
|
||||||
|
const parts = [];
|
||||||
|
const relationshipPhrase = this.getReadableVerbPhrase(typeResult.type);
|
||||||
|
if (data.role)
|
||||||
|
parts.push(`Role: ${data.role}`);
|
||||||
|
if (data.startDate)
|
||||||
|
parts.push(`Since: ${new Date(data.startDate).toLocaleDateString()}`);
|
||||||
|
if (data.department)
|
||||||
|
parts.push(`Department: ${data.department}`);
|
||||||
|
return parts.length > 0
|
||||||
|
? `${relationshipPhrase} - ${parts.join(', ')}`
|
||||||
|
: `${relationshipPhrase} relationship`;
|
||||||
|
}
|
||||||
|
return 'Relationship';
|
||||||
|
}
|
||||||
|
async computeVerbTags(context) {
|
||||||
|
const { data, typeResult } = context;
|
||||||
|
const tags = ['relationship'];
|
||||||
|
if (typeResult?.type) {
|
||||||
|
tags.push(typeResult.type.toLowerCase());
|
||||||
|
}
|
||||||
|
// Add relationship-specific tags
|
||||||
|
if (data.status)
|
||||||
|
tags.push(String(data.status).toLowerCase());
|
||||||
|
if (data.type)
|
||||||
|
tags.push(String(data.type).toLowerCase());
|
||||||
|
return [...new Set(tags)];
|
||||||
|
}
|
||||||
|
async computeHumanReadableRelationship(context) {
|
||||||
|
const { verbContext, typeResult } = context;
|
||||||
|
if (!verbContext || !typeResult)
|
||||||
|
return 'Related';
|
||||||
|
const { sourceId, targetId } = verbContext;
|
||||||
|
const phrase = this.getReadableVerbPhrase(typeResult.type);
|
||||||
|
return `${sourceId} ${phrase} ${targetId}`;
|
||||||
|
}
|
||||||
|
detectTypeHeuristically(data, entityType) {
|
||||||
|
if (entityType === 'verb')
|
||||||
|
return VerbType.RelatedTo;
|
||||||
|
// Basic heuristics for noun types
|
||||||
|
if (data.firstName || data.lastName || data.email)
|
||||||
|
return NounType.Person;
|
||||||
|
if (data.companyName || data.organization)
|
||||||
|
return NounType.Organization;
|
||||||
|
if (data.filename || data.fileType)
|
||||||
|
return NounType.Document;
|
||||||
|
if (data.projectName || data.initiative)
|
||||||
|
return NounType.Project;
|
||||||
|
if (data.taskName || data.todo)
|
||||||
|
return NounType.Task;
|
||||||
|
if (data.startDate || data.endDate)
|
||||||
|
return NounType.Event;
|
||||||
|
return 'Item'; // Generic fallback
|
||||||
|
}
|
||||||
|
extractFieldWithPatterns(data, patterns, fieldType) {
|
||||||
|
const relevantPatterns = patterns.filter(p => p.displayField === fieldType);
|
||||||
|
for (const pattern of relevantPatterns) {
|
||||||
|
for (const field of pattern.fields) {
|
||||||
|
if (data[field]) {
|
||||||
|
return pattern.transform ? pattern.transform(data[field], { data, config: this.config }) : data[field];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Shutdown the computation engine
|
||||||
|
*/
|
||||||
|
async shutdown() {
|
||||||
|
// Cleanup if needed
|
||||||
|
this.typeMatcher = null;
|
||||||
|
this.initialized = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=intelligentComputation.js.map
|
||||||
File diff suppressed because one or more lines are too long
203
.recovery-workspace/dist-backup-20250910-141917/augmentations/display/types.d.ts
vendored
Normal file
203
.recovery-workspace/dist-backup-20250910-141917/augmentations/display/types.d.ts
vendored
Normal file
|
|
@ -0,0 +1,203 @@
|
||||||
|
/**
|
||||||
|
* Universal Display Augmentation - Type Definitions
|
||||||
|
*
|
||||||
|
* Clean TypeScript interfaces for the display augmentation system
|
||||||
|
*/
|
||||||
|
import type { VectorDocument, GraphVerb } from '../../coreTypes.js';
|
||||||
|
/**
|
||||||
|
* Configuration interface for the Universal Display Augmentation
|
||||||
|
*/
|
||||||
|
export interface DisplayConfig {
|
||||||
|
/** Enable/disable the augmentation */
|
||||||
|
enabled: boolean;
|
||||||
|
/** LRU cache size for computed display fields */
|
||||||
|
cacheSize: number;
|
||||||
|
/** Use lazy computation (recommended for performance) */
|
||||||
|
lazyComputation: boolean;
|
||||||
|
/** Batch processing size for multiple requests */
|
||||||
|
batchSize: number;
|
||||||
|
/** Minimum confidence threshold for AI type detection */
|
||||||
|
confidenceThreshold: number;
|
||||||
|
/** Custom field mappings (userField -> displayField) */
|
||||||
|
customFieldMappings: Record<string, string>;
|
||||||
|
/** Type-specific priority fields for intelligent detection */
|
||||||
|
priorityFields: Record<string, string[]>;
|
||||||
|
/** Enable debug mode with reasoning output */
|
||||||
|
debugMode: boolean;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Computed display fields for any noun or verb
|
||||||
|
*/
|
||||||
|
export interface ComputedDisplayFields {
|
||||||
|
/** Primary display name (AI-detected best field combination) */
|
||||||
|
title: string;
|
||||||
|
/** Enhanced description with context awareness */
|
||||||
|
description: string;
|
||||||
|
/** Human-readable type name */
|
||||||
|
type: string;
|
||||||
|
/** Generated display tags for categorization */
|
||||||
|
tags: string[];
|
||||||
|
/** For verbs: human-readable relationship description */
|
||||||
|
relationship?: string;
|
||||||
|
/** AI confidence score (0-1) */
|
||||||
|
confidence: number;
|
||||||
|
/** Explanation of type detection reasoning (debug mode) */
|
||||||
|
reasoning?: string;
|
||||||
|
/** Alternative type suggestions with confidence scores */
|
||||||
|
alternatives?: Array<{
|
||||||
|
type: string;
|
||||||
|
confidence: number;
|
||||||
|
}>;
|
||||||
|
/** Timestamp when fields were computed */
|
||||||
|
computedAt: number;
|
||||||
|
/** Version of augmentation that computed these fields */
|
||||||
|
version: string;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Cache entry for computed display fields
|
||||||
|
*/
|
||||||
|
export interface DisplayCacheEntry {
|
||||||
|
fields: ComputedDisplayFields;
|
||||||
|
lastAccessed: number;
|
||||||
|
accessCount: number;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Field computation context passed to computation functions
|
||||||
|
*/
|
||||||
|
export interface FieldComputationContext {
|
||||||
|
/** The original data object */
|
||||||
|
data: any;
|
||||||
|
/** Metadata associated with the object */
|
||||||
|
metadata: any;
|
||||||
|
/** Type detection result from AI */
|
||||||
|
typeResult?: TypeMatchResult;
|
||||||
|
/** Display configuration */
|
||||||
|
config: DisplayConfig;
|
||||||
|
/** Whether this is a noun or verb */
|
||||||
|
entityType: 'noun' | 'verb';
|
||||||
|
/** For verbs: source and target information */
|
||||||
|
verbContext?: {
|
||||||
|
sourceId: string;
|
||||||
|
targetId: string;
|
||||||
|
verbType?: string;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Type matching result from BrainyTypes
|
||||||
|
*/
|
||||||
|
export interface TypeMatchResult {
|
||||||
|
type: string;
|
||||||
|
confidence: number;
|
||||||
|
reasoning: string;
|
||||||
|
alternatives: Array<{
|
||||||
|
type: string;
|
||||||
|
confidence: number;
|
||||||
|
}>;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Enhanced VectorDocument with display capabilities
|
||||||
|
*/
|
||||||
|
export interface EnhancedVectorDocument<T = any> extends VectorDocument<T> {
|
||||||
|
/**
|
||||||
|
* Get computed display field(s)
|
||||||
|
* @param field Optional specific field name
|
||||||
|
* @returns Single field value or all display fields
|
||||||
|
*/
|
||||||
|
getDisplay(): Promise<ComputedDisplayFields>;
|
||||||
|
getDisplay(field: keyof ComputedDisplayFields): Promise<any>;
|
||||||
|
/**
|
||||||
|
* Get available fields for a specific augmentation namespace
|
||||||
|
* @param namespace The augmentation namespace (e.g., 'display')
|
||||||
|
* @returns Array of available field names
|
||||||
|
*/
|
||||||
|
getAvailableFields(namespace: string): string[];
|
||||||
|
/**
|
||||||
|
* Get available augmentation namespaces
|
||||||
|
* @returns Array of available augmentation names
|
||||||
|
*/
|
||||||
|
getAvailableAugmentations(): string[];
|
||||||
|
/**
|
||||||
|
* Debug exploration of all computed fields
|
||||||
|
*/
|
||||||
|
explore(): Promise<void>;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Enhanced GraphVerb with display capabilities
|
||||||
|
*/
|
||||||
|
export interface EnhancedGraphVerb extends GraphVerb {
|
||||||
|
/**
|
||||||
|
* Get computed display field(s) for relationships
|
||||||
|
* @param field Optional specific field name
|
||||||
|
* @returns Single field value or all display fields
|
||||||
|
*/
|
||||||
|
getDisplay(): Promise<ComputedDisplayFields>;
|
||||||
|
getDisplay(field: keyof ComputedDisplayFields): Promise<any>;
|
||||||
|
/**
|
||||||
|
* Get available fields for a specific augmentation namespace
|
||||||
|
* @param namespace The augmentation namespace (e.g., 'display')
|
||||||
|
* @returns Array of available field names
|
||||||
|
*/
|
||||||
|
getAvailableFields(namespace: string): string[];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Batch computation request for performance optimization
|
||||||
|
*/
|
||||||
|
export interface BatchComputationRequest {
|
||||||
|
id: string;
|
||||||
|
data: any;
|
||||||
|
metadata: any;
|
||||||
|
entityType: 'noun' | 'verb';
|
||||||
|
verbContext?: {
|
||||||
|
sourceId: string;
|
||||||
|
targetId: string;
|
||||||
|
verbType?: string;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Batch computation result
|
||||||
|
*/
|
||||||
|
export interface BatchComputationResult {
|
||||||
|
id: string;
|
||||||
|
fields: ComputedDisplayFields;
|
||||||
|
error?: string;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Field pattern for intelligent field detection
|
||||||
|
*/
|
||||||
|
export interface FieldPattern {
|
||||||
|
/** Field names that match this pattern */
|
||||||
|
fields: string[];
|
||||||
|
/** Target display field name */
|
||||||
|
displayField: keyof ComputedDisplayFields;
|
||||||
|
/** Confidence score for this pattern match */
|
||||||
|
confidence: number;
|
||||||
|
/** Optional: specific noun/verb types this applies to */
|
||||||
|
applicableTypes?: string[];
|
||||||
|
/** Optional: transformation function for the field value */
|
||||||
|
transform?: (value: any, context: FieldComputationContext) => string;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Statistics for the display augmentation
|
||||||
|
*/
|
||||||
|
export interface DisplayAugmentationStats {
|
||||||
|
/** Total number of computations performed */
|
||||||
|
totalComputations: number;
|
||||||
|
/** Cache hit ratio */
|
||||||
|
cacheHitRatio: number;
|
||||||
|
/** Average computation time in milliseconds */
|
||||||
|
averageComputationTime: number;
|
||||||
|
/** Type detection accuracy (when ground truth available) */
|
||||||
|
typeDetectionAccuracy?: number;
|
||||||
|
/** Most commonly detected types */
|
||||||
|
commonTypes: Array<{
|
||||||
|
type: string;
|
||||||
|
count: number;
|
||||||
|
percentage: number;
|
||||||
|
}>;
|
||||||
|
/** Performance metrics */
|
||||||
|
performance: {
|
||||||
|
fastestComputation: number;
|
||||||
|
slowestComputation: number;
|
||||||
|
totalComputationTime: number;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,7 @@
|
||||||
|
/**
|
||||||
|
* Universal Display Augmentation - Type Definitions
|
||||||
|
*
|
||||||
|
* Clean TypeScript interfaces for the display augmentation system
|
||||||
|
*/
|
||||||
|
export {};
|
||||||
|
//# sourceMappingURL=types.js.map
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
{"version":3,"file":"types.js","sourceRoot":"","sources":["../../../src/augmentations/display/types.ts"],"names":[],"mappings":"AAAA;;;;GAIG"}
|
||||||
130
.recovery-workspace/dist-backup-20250910-141917/augmentations/entityRegistryAugmentation.d.ts
vendored
Normal file
130
.recovery-workspace/dist-backup-20250910-141917/augmentations/entityRegistryAugmentation.d.ts
vendored
Normal file
|
|
@ -0,0 +1,130 @@
|
||||||
|
/**
|
||||||
|
* Entity Registry Augmentation
|
||||||
|
* Fast external-ID to internal-UUID mapping for streaming data processing
|
||||||
|
* Works in write-only mode for high-performance deduplication
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation, AugmentationContext } from './brainyAugmentation.js';
|
||||||
|
export interface EntityRegistryConfig {
|
||||||
|
/**
|
||||||
|
* Maximum number of entries to keep in memory cache
|
||||||
|
* Default: 100,000 entries
|
||||||
|
*/
|
||||||
|
maxCacheSize?: number;
|
||||||
|
/**
|
||||||
|
* Time to live for cache entries in milliseconds
|
||||||
|
* Default: 300,000 (5 minutes)
|
||||||
|
*/
|
||||||
|
cacheTTL?: number;
|
||||||
|
/**
|
||||||
|
* Fields to index for fast lookup
|
||||||
|
* Default: ['did', 'handle', 'uri', 'id', 'external_id']
|
||||||
|
*/
|
||||||
|
indexedFields?: string[];
|
||||||
|
/**
|
||||||
|
* Persistence strategy
|
||||||
|
* memory: Keep only in memory (fast, but lost on restart)
|
||||||
|
* storage: Persist to storage (survives restarts)
|
||||||
|
* hybrid: Memory + periodic storage sync
|
||||||
|
*/
|
||||||
|
persistence?: 'memory' | 'storage' | 'hybrid';
|
||||||
|
/**
|
||||||
|
* How often to sync memory cache to storage (hybrid mode)
|
||||||
|
* Default: 30000 (30 seconds)
|
||||||
|
*/
|
||||||
|
syncInterval?: number;
|
||||||
|
}
|
||||||
|
export interface EntityMapping {
|
||||||
|
externalId: string;
|
||||||
|
field: string;
|
||||||
|
brainyId: string;
|
||||||
|
nounType: string;
|
||||||
|
lastAccessed: number;
|
||||||
|
metadata?: any;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* High-performance entity registry for external ID to Brainy UUID mapping
|
||||||
|
* Optimized for streaming data scenarios like Bluesky firehose processing
|
||||||
|
*/
|
||||||
|
export declare class EntityRegistryAugmentation extends BaseAugmentation {
|
||||||
|
readonly metadata: "readonly";
|
||||||
|
readonly name = "entity-registry";
|
||||||
|
readonly description = "Fast external-ID to internal-UUID mapping for streaming data";
|
||||||
|
readonly timing: 'before' | 'after' | 'around' | 'replace';
|
||||||
|
readonly operations: ("add" | "addNoun" | "addVerb")[];
|
||||||
|
readonly priority = 90;
|
||||||
|
protected config: Required<EntityRegistryConfig>;
|
||||||
|
private memoryIndex;
|
||||||
|
private fieldIndices;
|
||||||
|
private syncTimer?;
|
||||||
|
private brain?;
|
||||||
|
private storage?;
|
||||||
|
private cacheHits;
|
||||||
|
private cacheMisses;
|
||||||
|
constructor(config?: EntityRegistryConfig);
|
||||||
|
initialize(context: AugmentationContext): Promise<void>;
|
||||||
|
shutdown(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Execute the augmentation
|
||||||
|
*/
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
/**
|
||||||
|
* Register a new entity mapping
|
||||||
|
*/
|
||||||
|
registerEntity(brainyId: string, metadata: any, nounType: string): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Fast lookup: external ID → Brainy UUID
|
||||||
|
* Works in write-only mode without search indexes
|
||||||
|
*/
|
||||||
|
lookupEntity(field: string, value: string): Promise<string | null>;
|
||||||
|
/**
|
||||||
|
* Batch lookup for multiple external IDs
|
||||||
|
*/
|
||||||
|
lookupBatch(lookups: Array<{
|
||||||
|
field: string;
|
||||||
|
value: string;
|
||||||
|
}>): Promise<Map<string, string | null>>;
|
||||||
|
/**
|
||||||
|
* Check if entity exists (faster than lookupEntity for existence checks)
|
||||||
|
*/
|
||||||
|
hasEntity(field: string, value: string): Promise<boolean>;
|
||||||
|
/**
|
||||||
|
* Get all entities by field (e.g., all DIDs)
|
||||||
|
*/
|
||||||
|
getEntitiesByField(field: string): Promise<string[]>;
|
||||||
|
/**
|
||||||
|
* Get registry statistics
|
||||||
|
*/
|
||||||
|
getStats(): {
|
||||||
|
totalMappings: number;
|
||||||
|
fieldCounts: Record<string, number>;
|
||||||
|
cacheHitRate: number;
|
||||||
|
memoryUsage: number;
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Clear all cached mappings
|
||||||
|
*/
|
||||||
|
clearCache(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Check if an ID looks like it could be an external ID for a specific field
|
||||||
|
*/
|
||||||
|
private looksLikeExternalId;
|
||||||
|
private extractFieldValue;
|
||||||
|
private evictOldEntries;
|
||||||
|
private loadFromStorage;
|
||||||
|
private syncToStorage;
|
||||||
|
private loadFromStorageByField;
|
||||||
|
private loadBatchFromStorage;
|
||||||
|
private estimateMemoryUsage;
|
||||||
|
}
|
||||||
|
export declare class AutoRegisterEntitiesAugmentation extends BaseAugmentation {
|
||||||
|
readonly metadata: "readonly";
|
||||||
|
readonly name = "auto-register-entities";
|
||||||
|
readonly description = "Automatically register entities in the registry when added";
|
||||||
|
readonly timing: 'before' | 'after' | 'around' | 'replace';
|
||||||
|
readonly operations: ("add" | "addNoun" | "addVerb")[];
|
||||||
|
readonly priority = 85;
|
||||||
|
private registry?;
|
||||||
|
private brain?;
|
||||||
|
initialize(context: AugmentationContext): Promise<void>;
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,392 @@
|
||||||
|
/**
|
||||||
|
* Entity Registry Augmentation
|
||||||
|
* Fast external-ID to internal-UUID mapping for streaming data processing
|
||||||
|
* Works in write-only mode for high-performance deduplication
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
/**
|
||||||
|
* High-performance entity registry for external ID to Brainy UUID mapping
|
||||||
|
* Optimized for streaming data scenarios like Bluesky firehose processing
|
||||||
|
*/
|
||||||
|
export class EntityRegistryAugmentation extends BaseAugmentation {
|
||||||
|
constructor(config = {}) {
|
||||||
|
super();
|
||||||
|
this.metadata = 'readonly'; // Reads metadata to register entities
|
||||||
|
this.name = 'entity-registry';
|
||||||
|
this.description = 'Fast external-ID to internal-UUID mapping for streaming data';
|
||||||
|
this.timing = 'before';
|
||||||
|
this.operations = ['add', 'addNoun', 'addVerb'];
|
||||||
|
this.priority = 90; // High priority for entity registration
|
||||||
|
this.memoryIndex = new Map();
|
||||||
|
this.fieldIndices = new Map(); // field -> value -> brainyId
|
||||||
|
this.cacheHits = 0;
|
||||||
|
this.cacheMisses = 0;
|
||||||
|
this.config = {
|
||||||
|
maxCacheSize: config.maxCacheSize ?? 100000,
|
||||||
|
cacheTTL: config.cacheTTL ?? 300000, // 5 minutes
|
||||||
|
indexedFields: config.indexedFields ?? ['did', 'handle', 'uri', 'id', 'external_id'],
|
||||||
|
persistence: config.persistence ?? 'hybrid',
|
||||||
|
syncInterval: config.syncInterval ?? 30000 // 30 seconds
|
||||||
|
};
|
||||||
|
// Initialize field indices
|
||||||
|
for (const field of this.config.indexedFields) {
|
||||||
|
this.fieldIndices.set(field, new Map());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async initialize(context) {
|
||||||
|
this.brain = context.brain;
|
||||||
|
this.storage = context.storage;
|
||||||
|
// Load existing mappings from storage
|
||||||
|
if (this.config.persistence === 'storage' || this.config.persistence === 'hybrid') {
|
||||||
|
await this.loadFromStorage();
|
||||||
|
}
|
||||||
|
// Start sync timer for hybrid mode
|
||||||
|
if (this.config.persistence === 'hybrid') {
|
||||||
|
this.syncTimer = setInterval(() => {
|
||||||
|
this.syncToStorage().catch(console.error);
|
||||||
|
}, this.config.syncInterval);
|
||||||
|
}
|
||||||
|
console.log(`🔍 EntityRegistry initialized: ${this.memoryIndex.size} cached mappings`);
|
||||||
|
}
|
||||||
|
async shutdown() {
|
||||||
|
// Final sync before shutdown
|
||||||
|
if (this.config.persistence === 'storage' || this.config.persistence === 'hybrid') {
|
||||||
|
await this.syncToStorage();
|
||||||
|
}
|
||||||
|
if (this.syncTimer) {
|
||||||
|
clearInterval(this.syncTimer);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Execute the augmentation
|
||||||
|
*/
|
||||||
|
async execute(operation, params, next) {
|
||||||
|
console.log(`🔍 [EntityRegistry] execute called: operation=${operation}`);
|
||||||
|
// For add operations, check for duplicates first
|
||||||
|
if (operation === 'add' || operation === 'addNoun') {
|
||||||
|
const metadata = params.metadata || {};
|
||||||
|
// Check if entity already exists
|
||||||
|
for (const field of this.config.indexedFields) {
|
||||||
|
const value = this.extractFieldValue(metadata, field);
|
||||||
|
if (value) {
|
||||||
|
const existingId = await this.lookupEntity(field, value);
|
||||||
|
if (existingId) {
|
||||||
|
// Entity already exists, return the existing one
|
||||||
|
console.log(`🔍 Duplicate detected: ${field}:${value} → ${existingId}`);
|
||||||
|
return { id: existingId, duplicate: true };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// For addVerb operations, resolve external IDs to internal UUIDs
|
||||||
|
if (operation === 'addVerb') {
|
||||||
|
const sourceId = params.sourceId;
|
||||||
|
const targetId = params.targetId;
|
||||||
|
// Try to resolve source and target IDs if they look like external IDs
|
||||||
|
for (const field of this.config.indexedFields) {
|
||||||
|
// Check if sourceId matches an external ID pattern
|
||||||
|
if (typeof sourceId === 'string' && this.looksLikeExternalId(sourceId, field)) {
|
||||||
|
const resolvedSourceId = await this.lookupEntity(field, sourceId);
|
||||||
|
if (resolvedSourceId) {
|
||||||
|
console.log(`🔍 [EntityRegistry] Resolved source: ${sourceId} → ${resolvedSourceId}`);
|
||||||
|
params.sourceId = resolvedSourceId;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Check if targetId matches an external ID pattern
|
||||||
|
if (typeof targetId === 'string' && this.looksLikeExternalId(targetId, field)) {
|
||||||
|
const resolvedTargetId = await this.lookupEntity(field, targetId);
|
||||||
|
if (resolvedTargetId) {
|
||||||
|
console.log(`🔍 [EntityRegistry] Resolved target: ${targetId} → ${resolvedTargetId}`);
|
||||||
|
params.targetId = resolvedTargetId;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Proceed with the operation
|
||||||
|
const result = await next();
|
||||||
|
// Register the entity after successful add
|
||||||
|
if ((operation === 'add' || operation === 'addNoun' || operation === 'addVerb') && result) {
|
||||||
|
// Handle both formats: string UUID or object with id property
|
||||||
|
const brainyId = typeof result === 'string' ? result : result.id;
|
||||||
|
if (brainyId) {
|
||||||
|
const metadata = params.metadata || {};
|
||||||
|
const nounType = params.nounType || 'default';
|
||||||
|
console.log(`🔍 [EntityRegistry] Registering entity: ${brainyId}`);
|
||||||
|
await this.registerEntity(brainyId, metadata, nounType);
|
||||||
|
console.log(`✅ [EntityRegistry] Entity registered successfully`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Register a new entity mapping
|
||||||
|
*/
|
||||||
|
async registerEntity(brainyId, metadata, nounType) {
|
||||||
|
const now = Date.now();
|
||||||
|
// Extract indexed fields from metadata
|
||||||
|
for (const field of this.config.indexedFields) {
|
||||||
|
const value = this.extractFieldValue(metadata, field);
|
||||||
|
if (value) {
|
||||||
|
const key = `${field}:${value}`;
|
||||||
|
// Add to memory index
|
||||||
|
const mapping = {
|
||||||
|
externalId: value,
|
||||||
|
field,
|
||||||
|
brainyId,
|
||||||
|
nounType,
|
||||||
|
lastAccessed: now,
|
||||||
|
metadata
|
||||||
|
};
|
||||||
|
this.memoryIndex.set(key, mapping);
|
||||||
|
// Add to field-specific index
|
||||||
|
const fieldIndex = this.fieldIndices.get(field);
|
||||||
|
if (fieldIndex) {
|
||||||
|
fieldIndex.set(value, brainyId);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Enforce cache size limit (LRU eviction)
|
||||||
|
await this.evictOldEntries();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Fast lookup: external ID → Brainy UUID
|
||||||
|
* Works in write-only mode without search indexes
|
||||||
|
*/
|
||||||
|
async lookupEntity(field, value) {
|
||||||
|
const key = `${field}:${value}`;
|
||||||
|
const cached = this.memoryIndex.get(key);
|
||||||
|
if (cached) {
|
||||||
|
// Update last accessed time
|
||||||
|
cached.lastAccessed = Date.now();
|
||||||
|
this.cacheHits++;
|
||||||
|
return cached.brainyId;
|
||||||
|
}
|
||||||
|
this.cacheMisses++;
|
||||||
|
// If not in cache and using storage persistence, try loading from storage
|
||||||
|
if (this.config.persistence === 'storage' || this.config.persistence === 'hybrid') {
|
||||||
|
const stored = await this.loadFromStorageByField(field, value);
|
||||||
|
if (stored) {
|
||||||
|
// Add to memory cache
|
||||||
|
this.memoryIndex.set(key, stored);
|
||||||
|
const fieldIndex = this.fieldIndices.get(field);
|
||||||
|
if (fieldIndex) {
|
||||||
|
fieldIndex.set(value, stored.brainyId);
|
||||||
|
}
|
||||||
|
return stored.brainyId;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Batch lookup for multiple external IDs
|
||||||
|
*/
|
||||||
|
async lookupBatch(lookups) {
|
||||||
|
const results = new Map();
|
||||||
|
const missingKeys = [];
|
||||||
|
// Check memory cache first
|
||||||
|
for (const lookup of lookups) {
|
||||||
|
const key = `${lookup.field}:${lookup.value}`;
|
||||||
|
const cached = this.memoryIndex.get(key);
|
||||||
|
if (cached) {
|
||||||
|
cached.lastAccessed = Date.now();
|
||||||
|
results.set(key, cached.brainyId);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
missingKeys.push({ ...lookup, key });
|
||||||
|
results.set(key, null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Batch load missing keys from storage
|
||||||
|
if (missingKeys.length > 0 && (this.config.persistence === 'storage' || this.config.persistence === 'hybrid')) {
|
||||||
|
const stored = await this.loadBatchFromStorage(missingKeys);
|
||||||
|
for (const [key, mapping] of stored) {
|
||||||
|
if (mapping) {
|
||||||
|
// Add to memory cache
|
||||||
|
this.memoryIndex.set(key, mapping);
|
||||||
|
const fieldIndex = this.fieldIndices.get(mapping.field);
|
||||||
|
if (fieldIndex) {
|
||||||
|
fieldIndex.set(mapping.externalId, mapping.brainyId);
|
||||||
|
}
|
||||||
|
results.set(key, mapping.brainyId);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return results;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Check if entity exists (faster than lookupEntity for existence checks)
|
||||||
|
*/
|
||||||
|
async hasEntity(field, value) {
|
||||||
|
const fieldIndex = this.fieldIndices.get(field);
|
||||||
|
if (fieldIndex && fieldIndex.has(value)) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
return (await this.lookupEntity(field, value)) !== null;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get all entities by field (e.g., all DIDs)
|
||||||
|
*/
|
||||||
|
async getEntitiesByField(field) {
|
||||||
|
const fieldIndex = this.fieldIndices.get(field);
|
||||||
|
return fieldIndex ? Array.from(fieldIndex.keys()) : [];
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get registry statistics
|
||||||
|
*/
|
||||||
|
getStats() {
|
||||||
|
const fieldCounts = {};
|
||||||
|
for (const [field, index] of this.fieldIndices) {
|
||||||
|
fieldCounts[field] = index.size;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
totalMappings: this.memoryIndex.size,
|
||||||
|
fieldCounts,
|
||||||
|
cacheHitRate: this.cacheHits > 0 ? this.cacheHits / (this.cacheHits + this.cacheMisses) : 0,
|
||||||
|
memoryUsage: this.estimateMemoryUsage()
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear all cached mappings
|
||||||
|
*/
|
||||||
|
async clearCache() {
|
||||||
|
this.memoryIndex.clear();
|
||||||
|
for (const fieldIndex of this.fieldIndices.values()) {
|
||||||
|
fieldIndex.clear();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Private helper methods
|
||||||
|
/**
|
||||||
|
* Check if an ID looks like it could be an external ID for a specific field
|
||||||
|
*/
|
||||||
|
looksLikeExternalId(id, field) {
|
||||||
|
// Basic heuristics to detect external ID patterns
|
||||||
|
switch (field) {
|
||||||
|
case 'did':
|
||||||
|
return id.startsWith('did:');
|
||||||
|
case 'handle':
|
||||||
|
return id.includes('.') && (id.includes('bsky') || id.includes('social'));
|
||||||
|
case 'external_id':
|
||||||
|
return !id.match(/^[a-f0-9-]{36}$/i); // Not a UUID
|
||||||
|
case 'uri':
|
||||||
|
return id.startsWith('http') || id.startsWith('at://');
|
||||||
|
case 'id':
|
||||||
|
return !id.match(/^[a-f0-9-]{36}$/i); // Not a UUID
|
||||||
|
default:
|
||||||
|
// For custom fields, assume non-UUID strings might be external IDs
|
||||||
|
return typeof id === 'string' && id.length > 3 && !id.match(/^[a-f0-9-]{36}$/i);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
extractFieldValue(metadata, field) {
|
||||||
|
if (!metadata)
|
||||||
|
return null;
|
||||||
|
// Support nested field access (e.g., "author.did")
|
||||||
|
const parts = field.split('.');
|
||||||
|
let value = metadata;
|
||||||
|
for (const part of parts) {
|
||||||
|
value = value?.[part];
|
||||||
|
if (value === undefined || value === null) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return typeof value === 'string' ? value : String(value);
|
||||||
|
}
|
||||||
|
async evictOldEntries() {
|
||||||
|
if (this.memoryIndex.size <= this.config.maxCacheSize) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Sort by last accessed time and remove oldest entries
|
||||||
|
const entries = Array.from(this.memoryIndex.entries());
|
||||||
|
entries.sort(([, a], [, b]) => a.lastAccessed - b.lastAccessed);
|
||||||
|
const toRemove = entries.slice(0, entries.length - this.config.maxCacheSize);
|
||||||
|
for (const [key, mapping] of toRemove) {
|
||||||
|
this.memoryIndex.delete(key);
|
||||||
|
const fieldIndex = this.fieldIndices.get(mapping.field);
|
||||||
|
if (fieldIndex) {
|
||||||
|
fieldIndex.delete(mapping.externalId);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async loadFromStorage() {
|
||||||
|
if (!this.brain)
|
||||||
|
return;
|
||||||
|
try {
|
||||||
|
// Load registry data from a special storage location
|
||||||
|
const registryData = await this.brain.storage?.getMetadata('__entity_registry__');
|
||||||
|
if (registryData && registryData.mappings) {
|
||||||
|
for (const mapping of registryData.mappings) {
|
||||||
|
const key = `${mapping.field}:${mapping.externalId}`;
|
||||||
|
this.memoryIndex.set(key, mapping);
|
||||||
|
const fieldIndex = this.fieldIndices.get(mapping.field);
|
||||||
|
if (fieldIndex) {
|
||||||
|
fieldIndex.set(mapping.externalId, mapping.brainyId);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.warn('Failed to load entity registry from storage:', error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async syncToStorage() {
|
||||||
|
if (!this.brain)
|
||||||
|
return;
|
||||||
|
try {
|
||||||
|
const mappings = Array.from(this.memoryIndex.values());
|
||||||
|
await this.brain.storage?.saveMetadata('__entity_registry__', {
|
||||||
|
version: 1,
|
||||||
|
lastSync: Date.now(),
|
||||||
|
mappings
|
||||||
|
});
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.warn('Failed to sync entity registry to storage:', error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async loadFromStorageByField(field, value) {
|
||||||
|
// For now, this would require a full load. In production, you'd want
|
||||||
|
// a more sophisticated storage index system
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
async loadBatchFromStorage(keys) {
|
||||||
|
// For now, return empty. In production, implement batch storage lookup
|
||||||
|
return new Map();
|
||||||
|
}
|
||||||
|
estimateMemoryUsage() {
|
||||||
|
// Rough estimate: 200 bytes per mapping on average
|
||||||
|
return this.memoryIndex.size * 200;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Hook into Brainy's add operations to automatically register entities
|
||||||
|
export class AutoRegisterEntitiesAugmentation extends BaseAugmentation {
|
||||||
|
constructor() {
|
||||||
|
super(...arguments);
|
||||||
|
this.metadata = 'readonly'; // Reads metadata for auto-registration
|
||||||
|
this.name = 'auto-register-entities';
|
||||||
|
this.description = 'Automatically register entities in the registry when added';
|
||||||
|
this.timing = 'after';
|
||||||
|
this.operations = ['add', 'addNoun', 'addVerb'];
|
||||||
|
this.priority = 85; // After entity registry
|
||||||
|
}
|
||||||
|
async initialize(context) {
|
||||||
|
this.brain = context.brain;
|
||||||
|
// Find the entity registry augmentation from the registry
|
||||||
|
this.registry = this.brain?.augmentations?.augmentations?.find((aug) => aug instanceof EntityRegistryAugmentation);
|
||||||
|
}
|
||||||
|
async execute(operation, params, next) {
|
||||||
|
const result = await next();
|
||||||
|
// After successful add, register the entity
|
||||||
|
if ((operation === 'add' || operation === 'addNoun' || operation === 'addVerb') && result) {
|
||||||
|
if (this.registry) {
|
||||||
|
// Handle both formats: string UUID or object with id property
|
||||||
|
const brainyId = typeof result === 'string' ? result : result.id;
|
||||||
|
if (brainyId) {
|
||||||
|
const metadata = params.metadata || {};
|
||||||
|
const nounType = params.nounType || 'default';
|
||||||
|
await this.registry.registerEntity(brainyId, metadata, nounType);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=entityRegistryAugmentation.js.map
|
||||||
File diff suppressed because one or more lines are too long
120
.recovery-workspace/dist-backup-20250910-141917/augmentations/indexAugmentation.d.ts
vendored
Normal file
120
.recovery-workspace/dist-backup-20250910-141917/augmentations/indexAugmentation.d.ts
vendored
Normal file
|
|
@ -0,0 +1,120 @@
|
||||||
|
/**
|
||||||
|
* Index Augmentation - Optional Metadata Indexing
|
||||||
|
*
|
||||||
|
* Replaces the hardcoded MetadataIndex in Brainy with an optional augmentation.
|
||||||
|
* Provides O(1) metadata filtering and field lookups.
|
||||||
|
*
|
||||||
|
* Zero-config: Automatically enabled for better search performance
|
||||||
|
* Can be disabled or customized via augmentation registry
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
import { MetadataIndexManager } from '../utils/metadataIndex.js';
|
||||||
|
export interface IndexConfig {
|
||||||
|
enabled?: boolean;
|
||||||
|
maxFieldValues?: number;
|
||||||
|
maxIndexSize?: number;
|
||||||
|
autoRebuild?: boolean;
|
||||||
|
rebuildThreshold?: number;
|
||||||
|
flushInterval?: number;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* IndexAugmentation - Makes metadata indexing optional and pluggable
|
||||||
|
*
|
||||||
|
* Features:
|
||||||
|
* - O(1) metadata field lookups
|
||||||
|
* - Fast pre-filtering for searches
|
||||||
|
* - Automatic index maintenance
|
||||||
|
* - Zero-config with smart defaults
|
||||||
|
*/
|
||||||
|
export declare class IndexAugmentation extends BaseAugmentation {
|
||||||
|
readonly metadata: "readonly";
|
||||||
|
readonly name = "index";
|
||||||
|
readonly timing: "after";
|
||||||
|
operations: ("add" | "update" | "updateMetadata" | "delete" | "clear" | "all")[];
|
||||||
|
readonly priority = 60;
|
||||||
|
readonly category: "core";
|
||||||
|
readonly description = "Fast metadata field indexing for O(1) filtering and lookups";
|
||||||
|
private metadataIndex;
|
||||||
|
protected config: IndexConfig;
|
||||||
|
private flushTimer;
|
||||||
|
constructor(config?: IndexConfig);
|
||||||
|
protected onInitialize(): Promise<void>;
|
||||||
|
protected onShutdown(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Execute augmentation - maintain index on data operations
|
||||||
|
*/
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
/**
|
||||||
|
* Handle add operation - index new metadata
|
||||||
|
*/
|
||||||
|
private handleAdd;
|
||||||
|
/**
|
||||||
|
* Handle update operation - reindex metadata
|
||||||
|
*/
|
||||||
|
private handleUpdate;
|
||||||
|
/**
|
||||||
|
* Handle delete operation - remove from index
|
||||||
|
*/
|
||||||
|
private handleDelete;
|
||||||
|
/**
|
||||||
|
* Handle clear operation - clear index
|
||||||
|
*/
|
||||||
|
private handleClear;
|
||||||
|
/**
|
||||||
|
* Start periodic flush timer
|
||||||
|
*/
|
||||||
|
private startFlushTimer;
|
||||||
|
/**
|
||||||
|
* Get IDs that match metadata filter (for pre-filtering)
|
||||||
|
*/
|
||||||
|
getIdsForFilter(filter: Record<string, any>): Promise<string[]>;
|
||||||
|
/**
|
||||||
|
* Get available values for a field
|
||||||
|
*/
|
||||||
|
getFilterValues(field: string): Promise<any[]>;
|
||||||
|
/**
|
||||||
|
* Get all indexed fields
|
||||||
|
*/
|
||||||
|
getFilterFields(): Promise<string[]>;
|
||||||
|
/**
|
||||||
|
* Get index statistics
|
||||||
|
*/
|
||||||
|
getStats(): Promise<{
|
||||||
|
enabled: boolean;
|
||||||
|
totalEntries: number;
|
||||||
|
fieldsIndexed: never[];
|
||||||
|
memoryUsage: number;
|
||||||
|
} | {
|
||||||
|
totalEntries: number;
|
||||||
|
totalIds: number;
|
||||||
|
fieldsIndexed: string[];
|
||||||
|
lastRebuild: number;
|
||||||
|
indexSize: number;
|
||||||
|
enabled: boolean;
|
||||||
|
memoryUsage?: undefined;
|
||||||
|
}>;
|
||||||
|
/**
|
||||||
|
* Rebuild the index from storage
|
||||||
|
*/
|
||||||
|
rebuild(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Flush index to storage
|
||||||
|
*/
|
||||||
|
flush(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Add entry to index (public method for direct access)
|
||||||
|
*/
|
||||||
|
addToIndex(id: string, metadata: Record<string, any>): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Remove entry from index (public method for direct access)
|
||||||
|
*/
|
||||||
|
removeFromIndex(id: string, metadata: Record<string, any>): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Get the underlying MetadataIndexManager instance
|
||||||
|
*/
|
||||||
|
getMetadataIndex(): MetadataIndexManager | null;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Factory function for zero-config index augmentation
|
||||||
|
*/
|
||||||
|
export declare function createIndexAugmentation(config?: IndexConfig): IndexAugmentation;
|
||||||
|
|
@ -0,0 +1,288 @@
|
||||||
|
/**
|
||||||
|
* Index Augmentation - Optional Metadata Indexing
|
||||||
|
*
|
||||||
|
* Replaces the hardcoded MetadataIndex in Brainy with an optional augmentation.
|
||||||
|
* Provides O(1) metadata filtering and field lookups.
|
||||||
|
*
|
||||||
|
* Zero-config: Automatically enabled for better search performance
|
||||||
|
* Can be disabled or customized via augmentation registry
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
import { MetadataIndexManager } from '../utils/metadataIndex.js';
|
||||||
|
/**
|
||||||
|
* IndexAugmentation - Makes metadata indexing optional and pluggable
|
||||||
|
*
|
||||||
|
* Features:
|
||||||
|
* - O(1) metadata field lookups
|
||||||
|
* - Fast pre-filtering for searches
|
||||||
|
* - Automatic index maintenance
|
||||||
|
* - Zero-config with smart defaults
|
||||||
|
*/
|
||||||
|
export class IndexAugmentation extends BaseAugmentation {
|
||||||
|
constructor(config = {}) {
|
||||||
|
super();
|
||||||
|
this.metadata = 'readonly'; // Reads metadata to build indexes
|
||||||
|
this.name = 'index';
|
||||||
|
this.timing = 'after';
|
||||||
|
this.operations = ['add', 'update', 'updateMetadata', 'delete', 'clear', 'all'];
|
||||||
|
this.priority = 60; // Run after data operations
|
||||||
|
// Augmentation metadata
|
||||||
|
this.category = 'core';
|
||||||
|
this.description = 'Fast metadata field indexing for O(1) filtering and lookups';
|
||||||
|
this.metadataIndex = null;
|
||||||
|
this.flushTimer = null;
|
||||||
|
this.config = {
|
||||||
|
enabled: true,
|
||||||
|
maxFieldValues: 1000,
|
||||||
|
autoRebuild: true,
|
||||||
|
rebuildThreshold: 0.3, // Rebuild if 30% inconsistent
|
||||||
|
flushInterval: 30000, // Flush every 30 seconds
|
||||||
|
...config
|
||||||
|
};
|
||||||
|
}
|
||||||
|
async onInitialize() {
|
||||||
|
if (!this.config.enabled) {
|
||||||
|
this.log('Index augmentation disabled by configuration');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Get storage from context
|
||||||
|
const storage = this.context?.storage;
|
||||||
|
if (!storage) {
|
||||||
|
this.log('No storage available, index augmentation inactive', 'warn');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Initialize metadata index
|
||||||
|
this.metadataIndex = new MetadataIndexManager(storage, {
|
||||||
|
maxIndexSize: this.config.maxIndexSize || 10000
|
||||||
|
});
|
||||||
|
// Check if we need to rebuild
|
||||||
|
if (this.config.autoRebuild) {
|
||||||
|
const stats = await this.metadataIndex.getStats();
|
||||||
|
if (stats.totalEntries === 0) {
|
||||||
|
// Check if storage has data but index is empty
|
||||||
|
try {
|
||||||
|
const storageStats = await storage.getStatistics?.();
|
||||||
|
if (storageStats && storageStats.totalNouns > 0) {
|
||||||
|
this.log('Rebuilding metadata index for existing data...');
|
||||||
|
await this.metadataIndex.rebuild();
|
||||||
|
const newStats = await this.metadataIndex.getStats();
|
||||||
|
this.log(`Index rebuilt: ${newStats.totalEntries} entries, ${newStats.fieldsIndexed.length} fields`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (e) {
|
||||||
|
this.log('Could not check storage statistics', 'info');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Start flush timer
|
||||||
|
if (this.config.flushInterval && this.config.flushInterval > 0) {
|
||||||
|
this.startFlushTimer();
|
||||||
|
}
|
||||||
|
this.log('Index augmentation initialized');
|
||||||
|
}
|
||||||
|
async onShutdown() {
|
||||||
|
// Stop flush timer
|
||||||
|
if (this.flushTimer) {
|
||||||
|
clearInterval(this.flushTimer);
|
||||||
|
this.flushTimer = null;
|
||||||
|
}
|
||||||
|
// Flush index one last time
|
||||||
|
if (this.metadataIndex) {
|
||||||
|
try {
|
||||||
|
await this.metadataIndex.flush();
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log('Error flushing index during shutdown', 'warn');
|
||||||
|
}
|
||||||
|
this.metadataIndex = null;
|
||||||
|
}
|
||||||
|
this.log('Index augmentation shut down');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Execute augmentation - maintain index on data operations
|
||||||
|
*/
|
||||||
|
async execute(operation, params, next) {
|
||||||
|
// Execute the operation first
|
||||||
|
const result = await next();
|
||||||
|
// If index is disabled, just return
|
||||||
|
if (!this.metadataIndex || !this.config.enabled) {
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
// Handle index updates after operation completes
|
||||||
|
switch (operation) {
|
||||||
|
case 'add':
|
||||||
|
await this.handleAdd(params);
|
||||||
|
break;
|
||||||
|
case 'updateMetadata':
|
||||||
|
await this.handleUpdate(params);
|
||||||
|
break;
|
||||||
|
case 'delete':
|
||||||
|
await this.handleDelete(params);
|
||||||
|
break;
|
||||||
|
case 'clear':
|
||||||
|
await this.handleClear();
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Handle add operation - index new metadata
|
||||||
|
*/
|
||||||
|
async handleAdd(params) {
|
||||||
|
if (!this.metadataIndex)
|
||||||
|
return;
|
||||||
|
const { id, metadata } = params;
|
||||||
|
if (id && metadata) {
|
||||||
|
await this.metadataIndex.addToIndex(id, metadata);
|
||||||
|
this.log(`Indexed metadata for ${id}`, 'info');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Handle update operation - reindex metadata
|
||||||
|
*/
|
||||||
|
async handleUpdate(params) {
|
||||||
|
if (!this.metadataIndex)
|
||||||
|
return;
|
||||||
|
const { id, oldMetadata, newMetadata } = params;
|
||||||
|
// Remove old metadata
|
||||||
|
if (id && oldMetadata) {
|
||||||
|
await this.metadataIndex.removeFromIndex(id, oldMetadata);
|
||||||
|
}
|
||||||
|
// Add new metadata
|
||||||
|
if (id && newMetadata) {
|
||||||
|
await this.metadataIndex.addToIndex(id, newMetadata);
|
||||||
|
this.log(`Reindexed metadata for ${id}`, 'info');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Handle delete operation - remove from index
|
||||||
|
*/
|
||||||
|
async handleDelete(params) {
|
||||||
|
if (!this.metadataIndex)
|
||||||
|
return;
|
||||||
|
const { id, metadata } = params;
|
||||||
|
if (id && metadata) {
|
||||||
|
await this.metadataIndex.removeFromIndex(id, metadata);
|
||||||
|
this.log(`Removed ${id} from index`, 'info');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Handle clear operation - clear index
|
||||||
|
*/
|
||||||
|
async handleClear() {
|
||||||
|
if (!this.metadataIndex)
|
||||||
|
return;
|
||||||
|
// Clear the index when all data is cleared (rebuild effectively clears it)
|
||||||
|
await this.metadataIndex.rebuild();
|
||||||
|
this.log('Index cleared due to clear operation');
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Start periodic flush timer
|
||||||
|
*/
|
||||||
|
startFlushTimer() {
|
||||||
|
if (this.flushTimer)
|
||||||
|
return;
|
||||||
|
this.flushTimer = setInterval(async () => {
|
||||||
|
if (this.metadataIndex) {
|
||||||
|
try {
|
||||||
|
await this.metadataIndex.flush();
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log('Error during periodic index flush', 'warn');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, this.config.flushInterval);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get IDs that match metadata filter (for pre-filtering)
|
||||||
|
*/
|
||||||
|
async getIdsForFilter(filter) {
|
||||||
|
if (!this.metadataIndex)
|
||||||
|
return [];
|
||||||
|
return this.metadataIndex.getIdsForFilter(filter);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get available values for a field
|
||||||
|
*/
|
||||||
|
async getFilterValues(field) {
|
||||||
|
if (!this.metadataIndex)
|
||||||
|
return [];
|
||||||
|
return this.metadataIndex.getFilterValues(field);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get all indexed fields
|
||||||
|
*/
|
||||||
|
async getFilterFields() {
|
||||||
|
if (!this.metadataIndex)
|
||||||
|
return [];
|
||||||
|
return this.metadataIndex.getFilterFields();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get index statistics
|
||||||
|
*/
|
||||||
|
async getStats() {
|
||||||
|
if (!this.metadataIndex) {
|
||||||
|
return {
|
||||||
|
enabled: false,
|
||||||
|
totalEntries: 0,
|
||||||
|
fieldsIndexed: [],
|
||||||
|
memoryUsage: 0
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const stats = await this.metadataIndex.getStats();
|
||||||
|
return {
|
||||||
|
enabled: true,
|
||||||
|
...stats
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Rebuild the index from storage
|
||||||
|
*/
|
||||||
|
async rebuild() {
|
||||||
|
if (!this.metadataIndex) {
|
||||||
|
throw new Error('Index augmentation is not initialized');
|
||||||
|
}
|
||||||
|
this.log('Rebuilding metadata index...');
|
||||||
|
await this.metadataIndex.rebuild();
|
||||||
|
const stats = await this.metadataIndex.getStats();
|
||||||
|
this.log(`Index rebuilt: ${stats.totalEntries} entries, ${stats.fieldsIndexed.length} fields`);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Flush index to storage
|
||||||
|
*/
|
||||||
|
async flush() {
|
||||||
|
if (this.metadataIndex) {
|
||||||
|
await this.metadataIndex.flush();
|
||||||
|
this.log('Index flushed to storage', 'info');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Add entry to index (public method for direct access)
|
||||||
|
*/
|
||||||
|
async addToIndex(id, metadata) {
|
||||||
|
if (!this.metadataIndex)
|
||||||
|
return;
|
||||||
|
await this.metadataIndex.addToIndex(id, metadata);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Remove entry from index (public method for direct access)
|
||||||
|
*/
|
||||||
|
async removeFromIndex(id, metadata) {
|
||||||
|
if (!this.metadataIndex)
|
||||||
|
return;
|
||||||
|
await this.metadataIndex.removeFromIndex(id, metadata);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get the underlying MetadataIndexManager instance
|
||||||
|
*/
|
||||||
|
getMetadataIndex() {
|
||||||
|
return this.metadataIndex;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Factory function for zero-config index augmentation
|
||||||
|
*/
|
||||||
|
export function createIndexAugmentation(config) {
|
||||||
|
return new IndexAugmentation(config);
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=indexAugmentation.js.map
|
||||||
File diff suppressed because one or more lines are too long
158
.recovery-workspace/dist-backup-20250910-141917/augmentations/intelligentVerbScoring.d.ts
vendored
Normal file
158
.recovery-workspace/dist-backup-20250910-141917/augmentations/intelligentVerbScoring.d.ts
vendored
Normal file
|
|
@ -0,0 +1,158 @@
|
||||||
|
import { ICognitionAugmentation, AugmentationResponse } from '../types/augmentations.js';
|
||||||
|
/**
|
||||||
|
* Configuration options for the Intelligent Verb Scoring augmentation
|
||||||
|
*/
|
||||||
|
export interface IVerbScoringConfig {
|
||||||
|
/** Enable semantic proximity scoring based on entity embeddings */
|
||||||
|
enableSemanticScoring: boolean;
|
||||||
|
/** Enable frequency-based weight amplification */
|
||||||
|
enableFrequencyAmplification: boolean;
|
||||||
|
/** Enable temporal decay for weights */
|
||||||
|
enableTemporalDecay: boolean;
|
||||||
|
/** Decay rate per day for temporal scoring (0-1) */
|
||||||
|
temporalDecayRate: number;
|
||||||
|
/** Minimum weight threshold */
|
||||||
|
minWeight: number;
|
||||||
|
/** Maximum weight threshold */
|
||||||
|
maxWeight: number;
|
||||||
|
/** Base confidence score for new relationships */
|
||||||
|
baseConfidence: number;
|
||||||
|
/** Learning rate for adaptive scoring (0-1) */
|
||||||
|
learningRate: number;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Default configuration for the Intelligent Verb Scoring augmentation
|
||||||
|
*/
|
||||||
|
export declare const DEFAULT_VERB_SCORING_CONFIG: IVerbScoringConfig;
|
||||||
|
/**
|
||||||
|
* Relationship statistics for learning and adaptation
|
||||||
|
*/
|
||||||
|
interface RelationshipStats {
|
||||||
|
count: number;
|
||||||
|
totalWeight: number;
|
||||||
|
averageWeight: number;
|
||||||
|
lastSeen: Date;
|
||||||
|
firstSeen: Date;
|
||||||
|
semanticSimilarity?: number;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Intelligent Verb Scoring Cognition Augmentation
|
||||||
|
*
|
||||||
|
* Automatically generates intelligent weight and confidence scores for verb relationships
|
||||||
|
* using semantic analysis, frequency patterns, and temporal factors.
|
||||||
|
*/
|
||||||
|
export declare class IntelligentVerbScoring implements ICognitionAugmentation {
|
||||||
|
readonly name = "intelligent-verb-scoring";
|
||||||
|
readonly description = "Automatically generates intelligent weight and confidence scores for verb relationships";
|
||||||
|
enabled: boolean;
|
||||||
|
private config;
|
||||||
|
private relationshipStats;
|
||||||
|
private brainyInstance;
|
||||||
|
private isInitialized;
|
||||||
|
constructor(config?: Partial<IVerbScoringConfig>);
|
||||||
|
initialize(): Promise<void>;
|
||||||
|
shutDown(): Promise<void>;
|
||||||
|
getStatus(): Promise<'active' | 'inactive' | 'error'>;
|
||||||
|
/**
|
||||||
|
* Set reference to the BrainyData instance for accessing graph data
|
||||||
|
*/
|
||||||
|
setBrainyInstance(instance: any): void;
|
||||||
|
/**
|
||||||
|
* Main reasoning method for generating intelligent verb scores
|
||||||
|
*/
|
||||||
|
reason(query: string, context?: Record<string, unknown>): AugmentationResponse<{
|
||||||
|
inference: string;
|
||||||
|
confidence: number;
|
||||||
|
}>;
|
||||||
|
infer(dataSubset: Record<string, unknown>): AugmentationResponse<Record<string, unknown>>;
|
||||||
|
executeLogic(ruleId: string, input: Record<string, unknown>): AugmentationResponse<boolean>;
|
||||||
|
/**
|
||||||
|
* Generate intelligent weight and confidence scores for a verb relationship
|
||||||
|
*
|
||||||
|
* @param sourceId - ID of the source entity
|
||||||
|
* @param targetId - ID of the target entity
|
||||||
|
* @param verbType - Type of the relationship
|
||||||
|
* @param existingWeight - Existing weight if any
|
||||||
|
* @param metadata - Additional metadata about the relationship
|
||||||
|
* @returns Computed weight and confidence scores
|
||||||
|
*/
|
||||||
|
computeVerbScores(sourceId: string, targetId: string, verbType: string, existingWeight?: number, metadata?: any): Promise<{
|
||||||
|
weight: number;
|
||||||
|
confidence: number;
|
||||||
|
reasoning: string[];
|
||||||
|
}>;
|
||||||
|
/**
|
||||||
|
* Calculate semantic similarity between two entities using their embeddings
|
||||||
|
*/
|
||||||
|
private calculateSemanticScore;
|
||||||
|
/**
|
||||||
|
* Calculate frequency-based boost for repeated relationships
|
||||||
|
*/
|
||||||
|
private calculateFrequencyBoost;
|
||||||
|
/**
|
||||||
|
* Calculate temporal decay factor based on recency
|
||||||
|
*/
|
||||||
|
private calculateTemporalFactor;
|
||||||
|
/**
|
||||||
|
* Calculate learning-based adjustment using historical patterns
|
||||||
|
*/
|
||||||
|
private calculateLearningAdjustment;
|
||||||
|
/**
|
||||||
|
* Update relationship statistics for learning
|
||||||
|
*/
|
||||||
|
private updateRelationshipStats;
|
||||||
|
/**
|
||||||
|
* Blend two scores using a weighted average
|
||||||
|
*/
|
||||||
|
private blendScores;
|
||||||
|
/**
|
||||||
|
* Get current configuration
|
||||||
|
*/
|
||||||
|
getConfig(): IVerbScoringConfig;
|
||||||
|
/**
|
||||||
|
* Update configuration
|
||||||
|
*/
|
||||||
|
updateConfig(newConfig: Partial<IVerbScoringConfig>): void;
|
||||||
|
/**
|
||||||
|
* Get relationship statistics (for debugging/monitoring)
|
||||||
|
*/
|
||||||
|
getRelationshipStats(): Map<string, RelationshipStats>;
|
||||||
|
/**
|
||||||
|
* Clear relationship statistics
|
||||||
|
*/
|
||||||
|
clearStats(): void;
|
||||||
|
/**
|
||||||
|
* Provide feedback to improve future scoring
|
||||||
|
* This allows the system to learn from user corrections or validation
|
||||||
|
*
|
||||||
|
* @param sourceId - Source entity ID
|
||||||
|
* @param targetId - Target entity ID
|
||||||
|
* @param verbType - Relationship type
|
||||||
|
* @param feedbackWeight - The corrected/validated weight (0-1)
|
||||||
|
* @param feedbackConfidence - The corrected/validated confidence (0-1)
|
||||||
|
* @param feedbackType - Type of feedback ('correction', 'validation', 'enhancement')
|
||||||
|
*/
|
||||||
|
provideFeedback(sourceId: string, targetId: string, verbType: string, feedbackWeight: number, feedbackConfidence?: number, feedbackType?: 'correction' | 'validation' | 'enhancement'): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Get learning statistics for monitoring and debugging
|
||||||
|
*/
|
||||||
|
getLearningStats(): {
|
||||||
|
totalRelationships: number;
|
||||||
|
averageConfidence: number;
|
||||||
|
feedbackCount: number;
|
||||||
|
topRelationships: Array<{
|
||||||
|
relationship: string;
|
||||||
|
count: number;
|
||||||
|
averageWeight: number;
|
||||||
|
}>;
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Export learning data for backup or analysis
|
||||||
|
*/
|
||||||
|
exportLearningData(): string;
|
||||||
|
/**
|
||||||
|
* Import learning data from backup
|
||||||
|
*/
|
||||||
|
importLearningData(jsonData: string): void;
|
||||||
|
}
|
||||||
|
export {};
|
||||||
|
|
@ -0,0 +1,377 @@
|
||||||
|
import { cosineDistance } from '../utils/distance.js';
|
||||||
|
/**
|
||||||
|
* Default configuration for the Intelligent Verb Scoring augmentation
|
||||||
|
*/
|
||||||
|
export const DEFAULT_VERB_SCORING_CONFIG = {
|
||||||
|
enableSemanticScoring: true,
|
||||||
|
enableFrequencyAmplification: true,
|
||||||
|
enableTemporalDecay: true,
|
||||||
|
temporalDecayRate: 0.01, // 1% decay per day
|
||||||
|
minWeight: 0.1,
|
||||||
|
maxWeight: 1.0,
|
||||||
|
baseConfidence: 0.5,
|
||||||
|
learningRate: 0.1
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Intelligent Verb Scoring Cognition Augmentation
|
||||||
|
*
|
||||||
|
* Automatically generates intelligent weight and confidence scores for verb relationships
|
||||||
|
* using semantic analysis, frequency patterns, and temporal factors.
|
||||||
|
*/
|
||||||
|
export class IntelligentVerbScoring {
|
||||||
|
constructor(config = {}) {
|
||||||
|
this.name = 'intelligent-verb-scoring';
|
||||||
|
this.description = 'Automatically generates intelligent weight and confidence scores for verb relationships';
|
||||||
|
this.enabled = false; // Off by default as requested
|
||||||
|
this.relationshipStats = new Map();
|
||||||
|
this.isInitialized = false;
|
||||||
|
this.config = { ...DEFAULT_VERB_SCORING_CONFIG, ...config };
|
||||||
|
}
|
||||||
|
async initialize() {
|
||||||
|
if (this.isInitialized)
|
||||||
|
return;
|
||||||
|
this.isInitialized = true;
|
||||||
|
}
|
||||||
|
async shutDown() {
|
||||||
|
this.relationshipStats.clear();
|
||||||
|
this.isInitialized = false;
|
||||||
|
}
|
||||||
|
async getStatus() {
|
||||||
|
return this.enabled && this.isInitialized ? 'active' : 'inactive';
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Set reference to the BrainyData instance for accessing graph data
|
||||||
|
*/
|
||||||
|
setBrainyInstance(instance) {
|
||||||
|
this.brainyInstance = instance;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Main reasoning method for generating intelligent verb scores
|
||||||
|
*/
|
||||||
|
reason(query, context) {
|
||||||
|
if (!this.enabled) {
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
data: { inference: 'Augmentation is disabled', confidence: 0 },
|
||||||
|
error: 'Intelligent verb scoring is disabled'
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: true,
|
||||||
|
data: {
|
||||||
|
inference: 'Intelligent verb scoring active',
|
||||||
|
confidence: 1.0
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
infer(dataSubset) {
|
||||||
|
return {
|
||||||
|
success: true,
|
||||||
|
data: dataSubset
|
||||||
|
};
|
||||||
|
}
|
||||||
|
executeLogic(ruleId, input) {
|
||||||
|
return {
|
||||||
|
success: true,
|
||||||
|
data: true
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Generate intelligent weight and confidence scores for a verb relationship
|
||||||
|
*
|
||||||
|
* @param sourceId - ID of the source entity
|
||||||
|
* @param targetId - ID of the target entity
|
||||||
|
* @param verbType - Type of the relationship
|
||||||
|
* @param existingWeight - Existing weight if any
|
||||||
|
* @param metadata - Additional metadata about the relationship
|
||||||
|
* @returns Computed weight and confidence scores
|
||||||
|
*/
|
||||||
|
async computeVerbScores(sourceId, targetId, verbType, existingWeight, metadata) {
|
||||||
|
if (!this.enabled || !this.brainyInstance) {
|
||||||
|
return {
|
||||||
|
weight: existingWeight ?? 0.5,
|
||||||
|
confidence: this.config.baseConfidence,
|
||||||
|
reasoning: ['Intelligent scoring disabled']
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const reasoning = [];
|
||||||
|
let weight = existingWeight ?? 0.5;
|
||||||
|
let confidence = this.config.baseConfidence;
|
||||||
|
try {
|
||||||
|
// Get relationship key for statistics
|
||||||
|
const relationKey = `${sourceId}-${verbType}-${targetId}`;
|
||||||
|
// Update relationship statistics
|
||||||
|
this.updateRelationshipStats(relationKey, weight, metadata);
|
||||||
|
// Apply semantic scoring if enabled
|
||||||
|
if (this.config.enableSemanticScoring) {
|
||||||
|
const semanticScore = await this.calculateSemanticScore(sourceId, targetId);
|
||||||
|
if (semanticScore !== null) {
|
||||||
|
weight = this.blendScores(weight, semanticScore, 0.3);
|
||||||
|
confidence = Math.min(confidence + semanticScore * 0.2, 1.0);
|
||||||
|
reasoning.push(`Semantic similarity: ${semanticScore.toFixed(3)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Apply frequency amplification if enabled
|
||||||
|
if (this.config.enableFrequencyAmplification) {
|
||||||
|
const frequencyBoost = this.calculateFrequencyBoost(relationKey);
|
||||||
|
weight = this.blendScores(weight, frequencyBoost, 0.2);
|
||||||
|
if (frequencyBoost > 0.5) {
|
||||||
|
confidence = Math.min(confidence + 0.1, 1.0);
|
||||||
|
reasoning.push(`Frequency boost: ${frequencyBoost.toFixed(3)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Apply temporal decay if enabled
|
||||||
|
if (this.config.enableTemporalDecay) {
|
||||||
|
const temporalFactor = this.calculateTemporalFactor(relationKey);
|
||||||
|
weight *= temporalFactor;
|
||||||
|
reasoning.push(`Temporal factor: ${temporalFactor.toFixed(3)}`);
|
||||||
|
}
|
||||||
|
// Apply learning adjustments
|
||||||
|
const learningAdjustment = this.calculateLearningAdjustment(relationKey);
|
||||||
|
weight = this.blendScores(weight, learningAdjustment, this.config.learningRate);
|
||||||
|
// Clamp values to configured bounds
|
||||||
|
weight = Math.max(this.config.minWeight, Math.min(this.config.maxWeight, weight));
|
||||||
|
confidence = Math.max(0, Math.min(1, confidence));
|
||||||
|
reasoning.push(`Final weight: ${weight.toFixed(3)}, confidence: ${confidence.toFixed(3)}`);
|
||||||
|
return { weight, confidence, reasoning };
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.warn('Error computing verb scores:', error);
|
||||||
|
return {
|
||||||
|
weight: existingWeight ?? 0.5,
|
||||||
|
confidence: this.config.baseConfidence,
|
||||||
|
reasoning: [`Error in scoring: ${error}`]
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Calculate semantic similarity between two entities using their embeddings
|
||||||
|
*/
|
||||||
|
async calculateSemanticScore(sourceId, targetId) {
|
||||||
|
try {
|
||||||
|
if (!this.brainyInstance?.storage)
|
||||||
|
return null;
|
||||||
|
// Get noun embeddings from storage
|
||||||
|
const sourceNoun = await this.brainyInstance.storage.getNoun(sourceId);
|
||||||
|
const targetNoun = await this.brainyInstance.storage.getNoun(targetId);
|
||||||
|
if (!sourceNoun?.vector || !targetNoun?.vector)
|
||||||
|
return null;
|
||||||
|
// Calculate cosine similarity (1 - distance)
|
||||||
|
const distance = cosineDistance(sourceNoun.vector, targetNoun.vector);
|
||||||
|
return Math.max(0, 1 - distance);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.warn('Error calculating semantic score:', error);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Calculate frequency-based boost for repeated relationships
|
||||||
|
*/
|
||||||
|
calculateFrequencyBoost(relationKey) {
|
||||||
|
const stats = this.relationshipStats.get(relationKey);
|
||||||
|
if (!stats || stats.count <= 1)
|
||||||
|
return 0.5;
|
||||||
|
// Logarithmic scaling: more occurrences = higher weight, but with diminishing returns
|
||||||
|
const boost = Math.log(stats.count + 1) / Math.log(10); // Log base 10
|
||||||
|
return Math.min(boost, 1.0);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Calculate temporal decay factor based on recency
|
||||||
|
*/
|
||||||
|
calculateTemporalFactor(relationKey) {
|
||||||
|
const stats = this.relationshipStats.get(relationKey);
|
||||||
|
if (!stats)
|
||||||
|
return 1.0;
|
||||||
|
const daysSinceLastSeen = (Date.now() - stats.lastSeen.getTime()) / (1000 * 60 * 60 * 24);
|
||||||
|
const decayFactor = Math.exp(-this.config.temporalDecayRate * daysSinceLastSeen);
|
||||||
|
return Math.max(0.1, decayFactor); // Minimum 10% of original weight
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Calculate learning-based adjustment using historical patterns
|
||||||
|
*/
|
||||||
|
calculateLearningAdjustment(relationKey) {
|
||||||
|
const stats = this.relationshipStats.get(relationKey);
|
||||||
|
if (!stats || stats.count <= 1)
|
||||||
|
return 0.5;
|
||||||
|
// Use moving average of weights as learned baseline
|
||||||
|
return Math.max(0, Math.min(1, stats.averageWeight));
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Update relationship statistics for learning
|
||||||
|
*/
|
||||||
|
updateRelationshipStats(relationKey, weight, metadata) {
|
||||||
|
const now = new Date();
|
||||||
|
const existing = this.relationshipStats.get(relationKey);
|
||||||
|
if (existing) {
|
||||||
|
// Update existing stats
|
||||||
|
existing.count++;
|
||||||
|
existing.totalWeight += weight;
|
||||||
|
existing.averageWeight = existing.totalWeight / existing.count;
|
||||||
|
existing.lastSeen = now;
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
// Create new stats entry
|
||||||
|
this.relationshipStats.set(relationKey, {
|
||||||
|
count: 1,
|
||||||
|
totalWeight: weight,
|
||||||
|
averageWeight: weight,
|
||||||
|
lastSeen: now,
|
||||||
|
firstSeen: now
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Blend two scores using a weighted average
|
||||||
|
*/
|
||||||
|
blendScores(score1, score2, weight2) {
|
||||||
|
const weight1 = 1 - weight2;
|
||||||
|
return score1 * weight1 + score2 * weight2;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get current configuration
|
||||||
|
*/
|
||||||
|
getConfig() {
|
||||||
|
return { ...this.config };
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Update configuration
|
||||||
|
*/
|
||||||
|
updateConfig(newConfig) {
|
||||||
|
this.config = { ...this.config, ...newConfig };
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get relationship statistics (for debugging/monitoring)
|
||||||
|
*/
|
||||||
|
getRelationshipStats() {
|
||||||
|
return new Map(this.relationshipStats);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Clear relationship statistics
|
||||||
|
*/
|
||||||
|
clearStats() {
|
||||||
|
this.relationshipStats.clear();
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Provide feedback to improve future scoring
|
||||||
|
* This allows the system to learn from user corrections or validation
|
||||||
|
*
|
||||||
|
* @param sourceId - Source entity ID
|
||||||
|
* @param targetId - Target entity ID
|
||||||
|
* @param verbType - Relationship type
|
||||||
|
* @param feedbackWeight - The corrected/validated weight (0-1)
|
||||||
|
* @param feedbackConfidence - The corrected/validated confidence (0-1)
|
||||||
|
* @param feedbackType - Type of feedback ('correction', 'validation', 'enhancement')
|
||||||
|
*/
|
||||||
|
async provideFeedback(sourceId, targetId, verbType, feedbackWeight, feedbackConfidence, feedbackType = 'correction') {
|
||||||
|
if (!this.enabled)
|
||||||
|
return;
|
||||||
|
const relationKey = `${sourceId}-${verbType}-${targetId}`;
|
||||||
|
const existing = this.relationshipStats.get(relationKey);
|
||||||
|
if (existing) {
|
||||||
|
// Apply feedback with learning rate
|
||||||
|
const newWeight = existing.averageWeight * (1 - this.config.learningRate) +
|
||||||
|
feedbackWeight * this.config.learningRate;
|
||||||
|
// Update the running average with feedback
|
||||||
|
existing.totalWeight = (existing.totalWeight * existing.count + feedbackWeight) / (existing.count + 1);
|
||||||
|
existing.averageWeight = existing.totalWeight / existing.count;
|
||||||
|
existing.count += 1;
|
||||||
|
existing.lastSeen = new Date();
|
||||||
|
if (this.brainyInstance?.loggingConfig?.verbose) {
|
||||||
|
console.log(`Feedback applied for ${relationKey}: ${feedbackType}, ` +
|
||||||
|
`old weight: ${existing.averageWeight.toFixed(3)}, ` +
|
||||||
|
`feedback: ${feedbackWeight.toFixed(3)}, ` +
|
||||||
|
`new weight: ${newWeight.toFixed(3)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
// Create new entry with feedback as initial data
|
||||||
|
this.relationshipStats.set(relationKey, {
|
||||||
|
count: 1,
|
||||||
|
totalWeight: feedbackWeight,
|
||||||
|
averageWeight: feedbackWeight,
|
||||||
|
lastSeen: new Date(),
|
||||||
|
firstSeen: new Date()
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get learning statistics for monitoring and debugging
|
||||||
|
*/
|
||||||
|
getLearningStats() {
|
||||||
|
const relationships = Array.from(this.relationshipStats.entries());
|
||||||
|
const totalRelationships = relationships.length;
|
||||||
|
const feedbackCount = relationships.reduce((sum, [, stats]) => sum + stats.count, 0);
|
||||||
|
// Calculate average confidence (approximated from weight patterns)
|
||||||
|
const averageWeight = relationships.reduce((sum, [, stats]) => sum + stats.averageWeight, 0) / totalRelationships || 0;
|
||||||
|
const averageConfidence = Math.min(averageWeight + 0.2, 1.0); // Heuristic: confidence typically higher than weight
|
||||||
|
// Get top relationships by count
|
||||||
|
const topRelationships = relationships
|
||||||
|
.map(([key, stats]) => ({
|
||||||
|
relationship: key,
|
||||||
|
count: stats.count,
|
||||||
|
averageWeight: stats.averageWeight
|
||||||
|
}))
|
||||||
|
.sort((a, b) => b.count - a.count)
|
||||||
|
.slice(0, 10);
|
||||||
|
return {
|
||||||
|
totalRelationships,
|
||||||
|
averageConfidence,
|
||||||
|
feedbackCount,
|
||||||
|
topRelationships
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Export learning data for backup or analysis
|
||||||
|
*/
|
||||||
|
exportLearningData() {
|
||||||
|
const data = {
|
||||||
|
config: this.config,
|
||||||
|
stats: Array.from(this.relationshipStats.entries()).map(([key, stats]) => ({
|
||||||
|
relationship: key,
|
||||||
|
...stats,
|
||||||
|
firstSeen: stats.firstSeen.toISOString(),
|
||||||
|
lastSeen: stats.lastSeen.toISOString()
|
||||||
|
})),
|
||||||
|
exportedAt: new Date().toISOString(),
|
||||||
|
version: '1.0'
|
||||||
|
};
|
||||||
|
return JSON.stringify(data, null, 2);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Import learning data from backup
|
||||||
|
*/
|
||||||
|
importLearningData(jsonData) {
|
||||||
|
try {
|
||||||
|
const data = JSON.parse(jsonData);
|
||||||
|
if (data.version !== '1.0') {
|
||||||
|
console.warn('Learning data version mismatch, importing anyway');
|
||||||
|
}
|
||||||
|
// Update configuration if provided
|
||||||
|
if (data.config) {
|
||||||
|
this.config = { ...this.config, ...data.config };
|
||||||
|
}
|
||||||
|
// Import relationship statistics
|
||||||
|
if (data.stats && Array.isArray(data.stats)) {
|
||||||
|
for (const stat of data.stats) {
|
||||||
|
if (stat.relationship) {
|
||||||
|
this.relationshipStats.set(stat.relationship, {
|
||||||
|
count: stat.count || 1,
|
||||||
|
totalWeight: stat.totalWeight || stat.averageWeight || 0.5,
|
||||||
|
averageWeight: stat.averageWeight || 0.5,
|
||||||
|
firstSeen: new Date(stat.firstSeen || Date.now()),
|
||||||
|
lastSeen: new Date(stat.lastSeen || Date.now()),
|
||||||
|
semanticSimilarity: stat.semanticSimilarity
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
console.log(`Imported learning data: ${this.relationshipStats.size} relationships`);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.error('Failed to import learning data:', error);
|
||||||
|
throw new Error(`Failed to import learning data: ${error}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=intelligentVerbScoring.js.map
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,157 @@
|
||||||
|
/**
|
||||||
|
* Intelligent Verb Scoring Augmentation
|
||||||
|
*
|
||||||
|
* Enhances relationship quality through intelligent semantic scoring
|
||||||
|
* Provides context-aware relationship weights based on:
|
||||||
|
* - Semantic proximity of connected entities
|
||||||
|
* - Frequency-based amplification
|
||||||
|
* - Temporal decay modeling
|
||||||
|
* - Adaptive learning from usage patterns
|
||||||
|
*
|
||||||
|
* Critical for enterprise knowledge graphs with millions of relationships
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
interface VerbScoringConfig {
|
||||||
|
enabled?: boolean;
|
||||||
|
enableSemanticScoring?: boolean;
|
||||||
|
semanticThreshold?: number;
|
||||||
|
semanticWeight?: number;
|
||||||
|
enableFrequencyAmplification?: boolean;
|
||||||
|
frequencyDecay?: number;
|
||||||
|
maxFrequencyBoost?: number;
|
||||||
|
enableTemporalDecay?: boolean;
|
||||||
|
temporalDecayRate?: number;
|
||||||
|
temporalWindow?: number;
|
||||||
|
enableAdaptiveLearning?: boolean;
|
||||||
|
learningRate?: number;
|
||||||
|
confidenceThreshold?: number;
|
||||||
|
minWeight?: number;
|
||||||
|
maxWeight?: number;
|
||||||
|
baseWeight?: number;
|
||||||
|
}
|
||||||
|
interface RelationshipMetrics {
|
||||||
|
count: number;
|
||||||
|
totalWeight: number;
|
||||||
|
averageWeight: number;
|
||||||
|
lastUpdated: number;
|
||||||
|
semanticScore: number;
|
||||||
|
frequencyScore: number;
|
||||||
|
temporalScore: number;
|
||||||
|
confidenceScore: number;
|
||||||
|
}
|
||||||
|
interface ScoringMetrics {
|
||||||
|
relationshipsScored: number;
|
||||||
|
averageSemanticScore: number;
|
||||||
|
averageFrequencyScore: number;
|
||||||
|
averageTemporalScore: number;
|
||||||
|
averageConfidenceScore: number;
|
||||||
|
adaptiveAdjustments: number;
|
||||||
|
computationTimeMs: number;
|
||||||
|
}
|
||||||
|
export declare class IntelligentVerbScoringAugmentation extends BaseAugmentation {
|
||||||
|
name: string;
|
||||||
|
timing: "around";
|
||||||
|
readonly metadata: {
|
||||||
|
reads: string[];
|
||||||
|
writes: string[];
|
||||||
|
};
|
||||||
|
operations: ("addVerb" | "relate")[];
|
||||||
|
priority: number;
|
||||||
|
readonly category: "core";
|
||||||
|
readonly description = "AI-powered intelligent scoring for relationship strength analysis";
|
||||||
|
protected config: Required<VerbScoringConfig>;
|
||||||
|
private relationshipStats;
|
||||||
|
private metrics;
|
||||||
|
private scoringInstance;
|
||||||
|
constructor(config?: VerbScoringConfig);
|
||||||
|
protected onInitialize(): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Get this augmentation instance for API compatibility
|
||||||
|
* Used by Brainy to access scoring methods
|
||||||
|
*/
|
||||||
|
getScoring(): IntelligentVerbScoringAugmentation;
|
||||||
|
shouldExecute(operation: string, params: any): boolean;
|
||||||
|
execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
|
||||||
|
private calculateIntelligentWeight;
|
||||||
|
private calculateSemanticScore;
|
||||||
|
/**
|
||||||
|
* Detect noun type using neural taxonomy matching
|
||||||
|
*/
|
||||||
|
private detectNounType;
|
||||||
|
/**
|
||||||
|
* Calculate taxonomy-based similarity boost
|
||||||
|
*/
|
||||||
|
private calculateTaxonomyBoost;
|
||||||
|
private calculateCosineSimilarity;
|
||||||
|
private calculateFrequencyScore;
|
||||||
|
private calculateTemporalScore;
|
||||||
|
private calculateContextScore;
|
||||||
|
private updateRelationshipLearning;
|
||||||
|
private getConfidenceScore;
|
||||||
|
private getScoringMethodsUsed;
|
||||||
|
private storeDetailedScoring;
|
||||||
|
private updateMetrics;
|
||||||
|
/**
|
||||||
|
* Get intelligent verb scoring statistics
|
||||||
|
*/
|
||||||
|
getStats(): ScoringMetrics & {
|
||||||
|
totalRelationships: number;
|
||||||
|
averageConfidence: number;
|
||||||
|
highConfidenceRelationships: number;
|
||||||
|
learningEfficiency: number;
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Export relationship statistics for analysis
|
||||||
|
*/
|
||||||
|
exportRelationshipStats(): Array<{
|
||||||
|
relationship: string;
|
||||||
|
metrics: RelationshipMetrics;
|
||||||
|
}>;
|
||||||
|
/**
|
||||||
|
* Import relationship statistics from previous sessions
|
||||||
|
*/
|
||||||
|
importRelationshipStats(stats: Array<{
|
||||||
|
relationship: string;
|
||||||
|
metrics: RelationshipMetrics;
|
||||||
|
}>): void;
|
||||||
|
/**
|
||||||
|
* Get learning statistics for monitoring and debugging
|
||||||
|
* Required for Brainy.getVerbScoringStats()
|
||||||
|
*/
|
||||||
|
getLearningStats(): {
|
||||||
|
totalRelationships: number;
|
||||||
|
averageConfidence: number;
|
||||||
|
feedbackCount: number;
|
||||||
|
topRelationships: Array<{
|
||||||
|
relationship: string;
|
||||||
|
count: number;
|
||||||
|
averageWeight: number;
|
||||||
|
}>;
|
||||||
|
};
|
||||||
|
/**
|
||||||
|
* Export learning data for backup or analysis
|
||||||
|
* Required for Brainy.exportVerbScoringLearningData()
|
||||||
|
*/
|
||||||
|
exportLearningData(): string;
|
||||||
|
/**
|
||||||
|
* Import learning data from backup
|
||||||
|
* Required for Brainy.importVerbScoringLearningData()
|
||||||
|
*/
|
||||||
|
importLearningData(jsonData: string): void;
|
||||||
|
/**
|
||||||
|
* Provide feedback on a relationship's weight
|
||||||
|
* Required for Brainy.provideVerbScoringFeedback()
|
||||||
|
*/
|
||||||
|
provideFeedback(sourceId: string, targetId: string, relationType: string, feedback: number, feedbackType?: 'correction' | 'validation' | 'enhancement'): Promise<void>;
|
||||||
|
/**
|
||||||
|
* Compute intelligent scores for a verb relationship
|
||||||
|
* Used internally during verb creation
|
||||||
|
*/
|
||||||
|
computeVerbScores(sourceNoun: any, targetNoun: any, relationType: string): Promise<{
|
||||||
|
weight: number;
|
||||||
|
confidence: number;
|
||||||
|
reasoning: string[];
|
||||||
|
}>;
|
||||||
|
protected onShutdown(): Promise<void>;
|
||||||
|
}
|
||||||
|
export {};
|
||||||
|
|
@ -0,0 +1,559 @@
|
||||||
|
/**
|
||||||
|
* Intelligent Verb Scoring Augmentation
|
||||||
|
*
|
||||||
|
* Enhances relationship quality through intelligent semantic scoring
|
||||||
|
* Provides context-aware relationship weights based on:
|
||||||
|
* - Semantic proximity of connected entities
|
||||||
|
* - Frequency-based amplification
|
||||||
|
* - Temporal decay modeling
|
||||||
|
* - Adaptive learning from usage patterns
|
||||||
|
*
|
||||||
|
* Critical for enterprise knowledge graphs with millions of relationships
|
||||||
|
*/
|
||||||
|
import { BaseAugmentation } from './brainyAugmentation.js';
|
||||||
|
export class IntelligentVerbScoringAugmentation extends BaseAugmentation {
|
||||||
|
constructor(config = {}) {
|
||||||
|
super();
|
||||||
|
this.name = 'IntelligentVerbScoring';
|
||||||
|
this.timing = 'around';
|
||||||
|
this.metadata = {
|
||||||
|
reads: ['type', 'verb', 'source', 'target'],
|
||||||
|
writes: ['weight', 'confidence', 'intelligentScoring']
|
||||||
|
}; // Adds scoring metadata to verbs
|
||||||
|
this.operations = ['addVerb', 'relate'];
|
||||||
|
this.priority = 10; // Enhancement feature - runs after core operations
|
||||||
|
// Augmentation metadata
|
||||||
|
this.category = 'core';
|
||||||
|
this.description = 'AI-powered intelligent scoring for relationship strength analysis';
|
||||||
|
this.relationshipStats = new Map();
|
||||||
|
this.metrics = {
|
||||||
|
relationshipsScored: 0,
|
||||||
|
averageSemanticScore: 0,
|
||||||
|
averageFrequencyScore: 0,
|
||||||
|
averageTemporalScore: 0,
|
||||||
|
averageConfidenceScore: 0,
|
||||||
|
adaptiveAdjustments: 0,
|
||||||
|
computationTimeMs: 0
|
||||||
|
};
|
||||||
|
this.config = {
|
||||||
|
enabled: config.enabled ?? true, // Smart by default!
|
||||||
|
// Semantic Analysis
|
||||||
|
enableSemanticScoring: config.enableSemanticScoring ?? true,
|
||||||
|
semanticThreshold: config.semanticThreshold ?? 0.3,
|
||||||
|
semanticWeight: config.semanticWeight ?? 0.4,
|
||||||
|
// Frequency Analysis
|
||||||
|
enableFrequencyAmplification: config.enableFrequencyAmplification ?? true,
|
||||||
|
frequencyDecay: config.frequencyDecay ?? 0.95, // 5% decay per occurrence
|
||||||
|
maxFrequencyBoost: config.maxFrequencyBoost ?? 2.0,
|
||||||
|
// Temporal Analysis
|
||||||
|
enableTemporalDecay: config.enableTemporalDecay ?? true,
|
||||||
|
temporalDecayRate: config.temporalDecayRate ?? 0.01, // 1% per day
|
||||||
|
temporalWindow: config.temporalWindow ?? 365, // 1 year
|
||||||
|
// Learning & Adaptation
|
||||||
|
enableAdaptiveLearning: config.enableAdaptiveLearning ?? true,
|
||||||
|
learningRate: config.learningRate ?? 0.1,
|
||||||
|
confidenceThreshold: config.confidenceThreshold ?? 0.3,
|
||||||
|
// Weight Management
|
||||||
|
minWeight: config.minWeight ?? 0.1,
|
||||||
|
maxWeight: config.maxWeight ?? 1.0,
|
||||||
|
baseWeight: config.baseWeight ?? 0.5
|
||||||
|
};
|
||||||
|
// Set enabled property based on config
|
||||||
|
this.enabled = this.config.enabled;
|
||||||
|
}
|
||||||
|
async onInitialize() {
|
||||||
|
if (this.config.enabled) {
|
||||||
|
this.log('Intelligent verb scoring initialized for enhanced relationship quality');
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
this.log('Intelligent verb scoring disabled');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get this augmentation instance for API compatibility
|
||||||
|
* Used by Brainy to access scoring methods
|
||||||
|
*/
|
||||||
|
getScoring() {
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
shouldExecute(operation, params) {
|
||||||
|
// For addVerb, params are passed as array: [sourceId, targetId, verbType, metadata, weight]
|
||||||
|
if (operation === 'addVerb' && this.config.enabled) {
|
||||||
|
return Array.isArray(params) && params.length >= 3;
|
||||||
|
}
|
||||||
|
// For relate method, params might be an object
|
||||||
|
if (operation === 'relate' && this.config.enabled) {
|
||||||
|
return params.sourceId && params.targetId && params.relationType;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
async execute(operation, params, next) {
|
||||||
|
if (!this.shouldExecute(operation, params)) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
const startTime = Date.now();
|
||||||
|
try {
|
||||||
|
let sourceId, targetId, relationType, metadata;
|
||||||
|
let scoringResult = null;
|
||||||
|
// Extract parameters based on operation type
|
||||||
|
if (operation === 'addVerb' && Array.isArray(params)) {
|
||||||
|
// addVerb params: [sourceId, targetId, verbType, metadata, weight]
|
||||||
|
[sourceId, targetId, relationType, metadata] = params;
|
||||||
|
}
|
||||||
|
else if (operation === 'relate') {
|
||||||
|
// relate params might be an object
|
||||||
|
sourceId = params.sourceId;
|
||||||
|
targetId = params.targetId;
|
||||||
|
relationType = params.relationType;
|
||||||
|
metadata = params.metadata;
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
// Skip if weight is already provided explicitly
|
||||||
|
if (Array.isArray(params) && params[4] !== undefined && params[4] !== null) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
// Get the nouns to compute scoring
|
||||||
|
const sourceNoun = await this.context?.brain.get(sourceId);
|
||||||
|
const targetNoun = await this.context?.brain.get(targetId);
|
||||||
|
// Compute intelligent scores with reasoning
|
||||||
|
scoringResult = await this.computeVerbScores(sourceNoun, targetNoun, relationType);
|
||||||
|
// For addVerb, modify the params array
|
||||||
|
if (operation === 'addVerb' && Array.isArray(params)) {
|
||||||
|
// Set the weight parameter (index 4)
|
||||||
|
params[4] = scoringResult.weight;
|
||||||
|
// Enhance metadata with scoring info
|
||||||
|
params[3] = {
|
||||||
|
...params[3],
|
||||||
|
intelligentScoring: {
|
||||||
|
weight: scoringResult.weight,
|
||||||
|
confidence: scoringResult.confidence,
|
||||||
|
reasoning: scoringResult.reasoning,
|
||||||
|
scoringMethod: this.getScoringMethodsUsed(),
|
||||||
|
computedAt: Date.now()
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// Execute with enhanced parameters
|
||||||
|
const result = await next();
|
||||||
|
// Learn from this relationship
|
||||||
|
if (this.config.enableAdaptiveLearning && scoringResult) {
|
||||||
|
await this.updateRelationshipLearning(sourceId, targetId, relationType, scoringResult.weight);
|
||||||
|
}
|
||||||
|
// Update metrics
|
||||||
|
const computationTime = Date.now() - startTime;
|
||||||
|
if (scoringResult) {
|
||||||
|
this.updateMetrics(scoringResult.weight, computationTime);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
this.log(`Intelligent verb scoring error: ${error}`, 'error');
|
||||||
|
// Fallback to original parameters
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
async calculateIntelligentWeight(sourceId, targetId, relationType, metadata) {
|
||||||
|
let finalWeight = this.config.baseWeight;
|
||||||
|
let scoreComponents = {};
|
||||||
|
// 1. Semantic Proximity Score
|
||||||
|
if (this.config.enableSemanticScoring) {
|
||||||
|
const semanticScore = await this.calculateSemanticScore(sourceId, targetId);
|
||||||
|
scoreComponents.semantic = semanticScore;
|
||||||
|
finalWeight = finalWeight * (1 + semanticScore * this.config.semanticWeight);
|
||||||
|
}
|
||||||
|
// 2. Frequency Amplification Score
|
||||||
|
if (this.config.enableFrequencyAmplification) {
|
||||||
|
const frequencyScore = this.calculateFrequencyScore(sourceId, targetId, relationType);
|
||||||
|
scoreComponents.frequency = frequencyScore;
|
||||||
|
finalWeight = finalWeight * (1 + frequencyScore);
|
||||||
|
}
|
||||||
|
// 3. Temporal Relevance Score
|
||||||
|
if (this.config.enableTemporalDecay) {
|
||||||
|
const temporalScore = this.calculateTemporalScore(sourceId, targetId, relationType);
|
||||||
|
scoreComponents.temporal = temporalScore;
|
||||||
|
finalWeight = finalWeight * temporalScore;
|
||||||
|
}
|
||||||
|
// 4. Context Awareness (from metadata)
|
||||||
|
const contextScore = this.calculateContextScore(metadata);
|
||||||
|
scoreComponents.context = contextScore;
|
||||||
|
finalWeight = finalWeight * (1 + contextScore * 0.2);
|
||||||
|
// 5. Apply constraints
|
||||||
|
finalWeight = Math.max(this.config.minWeight, Math.min(this.config.maxWeight, finalWeight));
|
||||||
|
// Store detailed scoring for analysis
|
||||||
|
this.storeDetailedScoring(sourceId, targetId, relationType, {
|
||||||
|
finalWeight,
|
||||||
|
components: scoreComponents,
|
||||||
|
timestamp: Date.now()
|
||||||
|
});
|
||||||
|
return finalWeight;
|
||||||
|
}
|
||||||
|
async calculateSemanticScore(sourceId, targetId) {
|
||||||
|
try {
|
||||||
|
// Get embeddings for both entities
|
||||||
|
const sourceNoun = await this.context?.brain.get(sourceId);
|
||||||
|
const targetNoun = await this.context?.brain.get(targetId);
|
||||||
|
if (!sourceNoun?.vector || !targetNoun?.vector) {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
// Get noun types using neural detection (taxonomy-based)
|
||||||
|
const sourceType = await this.detectNounType(sourceNoun.vector);
|
||||||
|
const targetType = await this.detectNounType(targetNoun.vector);
|
||||||
|
// Calculate direct similarity
|
||||||
|
const directSimilarity = this.calculateCosineSimilarity(sourceNoun.vector, targetNoun.vector);
|
||||||
|
// Calculate taxonomy-based similarity boost
|
||||||
|
const taxonomyBoost = await this.calculateTaxonomyBoost(sourceType, targetType);
|
||||||
|
// Blend direct similarity with taxonomy guidance
|
||||||
|
// Taxonomy provides consistency while preserving flexibility
|
||||||
|
const semanticScore = directSimilarity * 0.7 + taxonomyBoost * 0.3;
|
||||||
|
return Math.min(1, Math.max(0, semanticScore));
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Detect noun type using neural taxonomy matching
|
||||||
|
*/
|
||||||
|
async detectNounType(vector) {
|
||||||
|
// Use the same neural detection as addNoun for consistency
|
||||||
|
if (!this.context?.brain)
|
||||||
|
return 'unknown';
|
||||||
|
try {
|
||||||
|
// This would normally call the brain's detectNounType method
|
||||||
|
// For now, simplified type detection based on vector patterns
|
||||||
|
const magnitude = Math.sqrt(vector.reduce((sum, val) => sum + val * val, 0));
|
||||||
|
// Heuristic type detection (would use actual taxonomy embeddings)
|
||||||
|
if (magnitude > 10)
|
||||||
|
return 'concept';
|
||||||
|
if (magnitude > 5)
|
||||||
|
return 'entity';
|
||||||
|
if (magnitude > 2)
|
||||||
|
return 'object';
|
||||||
|
return 'item';
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
return 'unknown';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Calculate taxonomy-based similarity boost
|
||||||
|
*/
|
||||||
|
async calculateTaxonomyBoost(sourceType, targetType) {
|
||||||
|
// Define valid relationship patterns in taxonomy
|
||||||
|
const validPatterns = {
|
||||||
|
'person': { 'concept': 0.9, 'skill': 0.85, 'organization': 0.8, 'person': 0.7 },
|
||||||
|
'concept': { 'concept': 0.9, 'example': 0.85, 'application': 0.8 },
|
||||||
|
'entity': { 'entity': 0.8, 'property': 0.85, 'action': 0.75 },
|
||||||
|
'object': { 'object': 0.7, 'property': 0.8, 'location': 0.75 },
|
||||||
|
'document': { 'topic': 0.9, 'author': 0.85, 'document': 0.7 },
|
||||||
|
'tool': { 'output': 0.9, 'input': 0.85, 'user': 0.8 },
|
||||||
|
'unknown': { 'unknown': 0.5 } // Fallback
|
||||||
|
};
|
||||||
|
// Get boost from taxonomy patterns
|
||||||
|
const patterns = validPatterns[sourceType] || validPatterns['unknown'];
|
||||||
|
const boost = patterns[targetType] || 0.3; // Low score for unrecognized patterns
|
||||||
|
return boost;
|
||||||
|
}
|
||||||
|
calculateCosineSimilarity(vectorA, vectorB) {
|
||||||
|
if (vectorA.length !== vectorB.length)
|
||||||
|
return 0;
|
||||||
|
let dotProduct = 0;
|
||||||
|
let normA = 0;
|
||||||
|
let normB = 0;
|
||||||
|
for (let i = 0; i < vectorA.length; i++) {
|
||||||
|
dotProduct += vectorA[i] * vectorB[i];
|
||||||
|
normA += vectorA[i] * vectorA[i];
|
||||||
|
normB += vectorB[i] * vectorB[i];
|
||||||
|
}
|
||||||
|
const magnitude = Math.sqrt(normA) * Math.sqrt(normB);
|
||||||
|
return magnitude ? dotProduct / magnitude : 0;
|
||||||
|
}
|
||||||
|
calculateFrequencyScore(sourceId, targetId, relationType) {
|
||||||
|
const relationshipKey = `${sourceId}:${relationType}:${targetId}`;
|
||||||
|
const stats = this.relationshipStats.get(relationshipKey);
|
||||||
|
if (!stats || stats.count <= 1)
|
||||||
|
return 0;
|
||||||
|
// Frequency boost diminishes with each occurrence
|
||||||
|
const frequencyBoost = Math.log(stats.count) * this.config.frequencyDecay;
|
||||||
|
return Math.min(this.config.maxFrequencyBoost, frequencyBoost);
|
||||||
|
}
|
||||||
|
calculateTemporalScore(sourceId, targetId, relationType) {
|
||||||
|
const relationshipKey = `${sourceId}:${relationType}:${targetId}`;
|
||||||
|
const stats = this.relationshipStats.get(relationshipKey);
|
||||||
|
if (!stats)
|
||||||
|
return 1.0; // New relationship - full temporal score
|
||||||
|
const daysSinceUpdate = (Date.now() - stats.lastUpdated) / (1000 * 60 * 60 * 24);
|
||||||
|
const decayFactor = Math.pow(1 - this.config.temporalDecayRate, daysSinceUpdate);
|
||||||
|
// Relationships older than temporal window get minimum score
|
||||||
|
if (daysSinceUpdate > this.config.temporalWindow) {
|
||||||
|
return this.config.minWeight / this.config.baseWeight;
|
||||||
|
}
|
||||||
|
return Math.max(0.1, decayFactor);
|
||||||
|
}
|
||||||
|
calculateContextScore(metadata) {
|
||||||
|
if (!metadata)
|
||||||
|
return 0;
|
||||||
|
let contextScore = 0;
|
||||||
|
// Boost for explicit importance
|
||||||
|
if (metadata.importance) {
|
||||||
|
contextScore += Math.min(0.5, metadata.importance);
|
||||||
|
}
|
||||||
|
// Boost for confidence
|
||||||
|
if (metadata.confidence) {
|
||||||
|
contextScore += Math.min(0.3, metadata.confidence);
|
||||||
|
}
|
||||||
|
// Boost for source quality
|
||||||
|
if (metadata.sourceQuality) {
|
||||||
|
contextScore += Math.min(0.2, metadata.sourceQuality);
|
||||||
|
}
|
||||||
|
return contextScore;
|
||||||
|
}
|
||||||
|
async updateRelationshipLearning(sourceId, targetId, relationType, weight) {
|
||||||
|
const relationshipKey = `${sourceId}:${relationType}:${targetId}`;
|
||||||
|
let stats = this.relationshipStats.get(relationshipKey);
|
||||||
|
if (!stats) {
|
||||||
|
stats = {
|
||||||
|
count: 0,
|
||||||
|
totalWeight: 0,
|
||||||
|
averageWeight: this.config.baseWeight,
|
||||||
|
lastUpdated: Date.now(),
|
||||||
|
semanticScore: 0,
|
||||||
|
frequencyScore: 0,
|
||||||
|
temporalScore: 1.0,
|
||||||
|
confidenceScore: this.config.baseWeight
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// Update statistics with learning rate
|
||||||
|
stats.count++;
|
||||||
|
stats.totalWeight += weight;
|
||||||
|
stats.averageWeight = stats.averageWeight * (1 - this.config.learningRate) +
|
||||||
|
weight * this.config.learningRate;
|
||||||
|
stats.lastUpdated = Date.now();
|
||||||
|
// Update confidence based on consistency
|
||||||
|
const weightVariance = Math.abs(weight - stats.averageWeight);
|
||||||
|
const consistencyScore = 1 - Math.min(1, weightVariance);
|
||||||
|
stats.confidenceScore = stats.confidenceScore * (1 - this.config.learningRate) +
|
||||||
|
consistencyScore * this.config.learningRate;
|
||||||
|
this.relationshipStats.set(relationshipKey, stats);
|
||||||
|
this.metrics.adaptiveAdjustments++;
|
||||||
|
}
|
||||||
|
getConfidenceScore(sourceId, targetId, relationType) {
|
||||||
|
const relationshipKey = `${sourceId}:${relationType}:${targetId}`;
|
||||||
|
const stats = this.relationshipStats.get(relationshipKey);
|
||||||
|
return stats ? stats.confidenceScore : this.config.baseWeight;
|
||||||
|
}
|
||||||
|
getScoringMethodsUsed() {
|
||||||
|
const methods = [];
|
||||||
|
if (this.config.enableSemanticScoring)
|
||||||
|
methods.push('semantic');
|
||||||
|
if (this.config.enableFrequencyAmplification)
|
||||||
|
methods.push('frequency');
|
||||||
|
if (this.config.enableTemporalDecay)
|
||||||
|
methods.push('temporal');
|
||||||
|
if (this.config.enableAdaptiveLearning)
|
||||||
|
methods.push('adaptive');
|
||||||
|
return methods;
|
||||||
|
}
|
||||||
|
storeDetailedScoring(sourceId, targetId, relationType, scoring) {
|
||||||
|
// Store detailed scoring for analysis and debugging
|
||||||
|
// In production, this might be sent to analytics system
|
||||||
|
}
|
||||||
|
updateMetrics(weight, computationTime) {
|
||||||
|
this.metrics.relationshipsScored++;
|
||||||
|
this.metrics.computationTimeMs =
|
||||||
|
(this.metrics.computationTimeMs * (this.metrics.relationshipsScored - 1) + computationTime) /
|
||||||
|
this.metrics.relationshipsScored;
|
||||||
|
// Update score averages (simplified)
|
||||||
|
// In practice, we'd track these more precisely
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get intelligent verb scoring statistics
|
||||||
|
*/
|
||||||
|
getStats() {
|
||||||
|
let totalConfidence = 0;
|
||||||
|
let highConfidenceCount = 0;
|
||||||
|
for (const stats of this.relationshipStats.values()) {
|
||||||
|
totalConfidence += stats.confidenceScore;
|
||||||
|
if (stats.confidenceScore >= this.config.confidenceThreshold * 2) {
|
||||||
|
highConfidenceCount++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const totalRelationships = this.relationshipStats.size;
|
||||||
|
const averageConfidence = totalRelationships > 0 ? totalConfidence / totalRelationships : 0;
|
||||||
|
const learningEfficiency = this.metrics.adaptiveAdjustments / Math.max(1, this.metrics.relationshipsScored);
|
||||||
|
return {
|
||||||
|
...this.metrics,
|
||||||
|
totalRelationships,
|
||||||
|
averageConfidence,
|
||||||
|
highConfidenceRelationships: highConfidenceCount,
|
||||||
|
learningEfficiency
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Export relationship statistics for analysis
|
||||||
|
*/
|
||||||
|
exportRelationshipStats() {
|
||||||
|
return Array.from(this.relationshipStats.entries()).map(([key, metrics]) => ({
|
||||||
|
relationship: key,
|
||||||
|
metrics
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Import relationship statistics from previous sessions
|
||||||
|
*/
|
||||||
|
importRelationshipStats(stats) {
|
||||||
|
for (const { relationship, metrics } of stats) {
|
||||||
|
this.relationshipStats.set(relationship, metrics);
|
||||||
|
}
|
||||||
|
this.log(`Imported ${stats.length} relationship statistics`);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Get learning statistics for monitoring and debugging
|
||||||
|
* Required for Brainy.getVerbScoringStats()
|
||||||
|
*/
|
||||||
|
getLearningStats() {
|
||||||
|
const relationships = Array.from(this.relationshipStats.entries());
|
||||||
|
const totalRelationships = relationships.length;
|
||||||
|
const feedbackCount = relationships.reduce((sum, [, stats]) => sum + stats.count, 0);
|
||||||
|
const averageWeight = relationships.reduce((sum, [, stats]) => sum + stats.averageWeight, 0) / totalRelationships || 0;
|
||||||
|
const averageConfidence = Math.min(averageWeight + 0.2, 1.0);
|
||||||
|
const topRelationships = relationships
|
||||||
|
.map(([key, stats]) => ({
|
||||||
|
relationship: key,
|
||||||
|
count: stats.count,
|
||||||
|
averageWeight: stats.averageWeight
|
||||||
|
}))
|
||||||
|
.sort((a, b) => b.count - a.count)
|
||||||
|
.slice(0, 10);
|
||||||
|
return {
|
||||||
|
totalRelationships,
|
||||||
|
averageConfidence,
|
||||||
|
feedbackCount,
|
||||||
|
topRelationships
|
||||||
|
};
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Export learning data for backup or analysis
|
||||||
|
* Required for Brainy.exportVerbScoringLearningData()
|
||||||
|
*/
|
||||||
|
exportLearningData() {
|
||||||
|
const data = {
|
||||||
|
config: this.config,
|
||||||
|
stats: Array.from(this.relationshipStats.entries()).map(([key, stats]) => ({
|
||||||
|
relationship: key,
|
||||||
|
...stats
|
||||||
|
})),
|
||||||
|
exportedAt: new Date().toISOString(),
|
||||||
|
version: '1.0'
|
||||||
|
};
|
||||||
|
return JSON.stringify(data, null, 2);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Import learning data from backup
|
||||||
|
* Required for Brainy.importVerbScoringLearningData()
|
||||||
|
*/
|
||||||
|
importLearningData(jsonData) {
|
||||||
|
try {
|
||||||
|
const data = JSON.parse(jsonData);
|
||||||
|
if (data.stats && Array.isArray(data.stats)) {
|
||||||
|
for (const stat of data.stats) {
|
||||||
|
if (stat.relationship) {
|
||||||
|
this.relationshipStats.set(stat.relationship, {
|
||||||
|
count: stat.count || 1,
|
||||||
|
totalWeight: stat.totalWeight || stat.averageWeight || 0.5,
|
||||||
|
averageWeight: stat.averageWeight || 0.5,
|
||||||
|
lastUpdated: stat.lastUpdated || Date.now(),
|
||||||
|
semanticScore: stat.semanticScore || 0.5,
|
||||||
|
frequencyScore: stat.frequencyScore || 0.5,
|
||||||
|
temporalScore: stat.temporalScore || 1.0,
|
||||||
|
confidenceScore: stat.confidenceScore || 0.5
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this.log(`Imported learning data: ${this.relationshipStats.size} relationships`);
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
console.error('Failed to import learning data:', error);
|
||||||
|
throw new Error(`Failed to import learning data: ${error}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Provide feedback on a relationship's weight
|
||||||
|
* Required for Brainy.provideVerbScoringFeedback()
|
||||||
|
*/
|
||||||
|
async provideFeedback(sourceId, targetId, relationType, feedback, feedbackType = 'correction') {
|
||||||
|
const key = `${sourceId}-${relationType}-${targetId}`;
|
||||||
|
const stats = this.relationshipStats.get(key) || {
|
||||||
|
count: 0,
|
||||||
|
totalWeight: 0,
|
||||||
|
averageWeight: 0.5,
|
||||||
|
lastUpdated: Date.now(),
|
||||||
|
semanticScore: 0.5,
|
||||||
|
frequencyScore: 0.5,
|
||||||
|
temporalScore: 1.0,
|
||||||
|
confidenceScore: 0.5
|
||||||
|
};
|
||||||
|
// Update statistics based on feedback
|
||||||
|
if (feedbackType === 'correction') {
|
||||||
|
// Direct correction - heavily weight the feedback
|
||||||
|
stats.averageWeight = stats.averageWeight * 0.3 + feedback * 0.7;
|
||||||
|
}
|
||||||
|
else if (feedbackType === 'validation') {
|
||||||
|
// Validation - slightly adjust towards feedback
|
||||||
|
stats.averageWeight = stats.averageWeight * 0.8 + feedback * 0.2;
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
// Enhancement - minor adjustment
|
||||||
|
stats.averageWeight = stats.averageWeight * 0.9 + feedback * 0.1;
|
||||||
|
}
|
||||||
|
stats.count++;
|
||||||
|
stats.totalWeight += feedback;
|
||||||
|
stats.lastUpdated = Date.now();
|
||||||
|
this.relationshipStats.set(key, stats);
|
||||||
|
this.metrics.adaptiveAdjustments++;
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* Compute intelligent scores for a verb relationship
|
||||||
|
* Used internally during verb creation
|
||||||
|
*/
|
||||||
|
async computeVerbScores(sourceNoun, targetNoun, relationType) {
|
||||||
|
const reasoning = [];
|
||||||
|
let totalScore = 0;
|
||||||
|
let components = 0;
|
||||||
|
// Semantic scoring
|
||||||
|
if (this.config.enableSemanticScoring && sourceNoun?.vector && targetNoun?.vector) {
|
||||||
|
const similarity = this.calculateCosineSimilarity(sourceNoun.vector, targetNoun.vector);
|
||||||
|
const semanticScore = Math.max(similarity, this.config.semanticThreshold);
|
||||||
|
totalScore += semanticScore * this.config.semanticWeight;
|
||||||
|
components++;
|
||||||
|
reasoning.push(`Semantic similarity: ${(similarity * 100).toFixed(1)}%`);
|
||||||
|
}
|
||||||
|
// Frequency scoring
|
||||||
|
const key = `${sourceNoun?.id}-${relationType}-${targetNoun?.id}`;
|
||||||
|
const stats = this.relationshipStats.get(key);
|
||||||
|
if (this.config.enableFrequencyAmplification && stats) {
|
||||||
|
const frequencyScore = Math.min(1 + (stats.count - 1) * 0.1, this.config.maxFrequencyBoost);
|
||||||
|
totalScore += frequencyScore * 0.3;
|
||||||
|
components++;
|
||||||
|
reasoning.push(`Frequency boost: ${frequencyScore.toFixed(2)}x`);
|
||||||
|
}
|
||||||
|
// Temporal decay scoring
|
||||||
|
if (this.config.enableTemporalDecay) {
|
||||||
|
reasoning.push(`Temporal decay applied (rate: ${this.config.temporalDecayRate})`);
|
||||||
|
}
|
||||||
|
// Calculate final weight
|
||||||
|
const weight = components > 0
|
||||||
|
? Math.min(Math.max(totalScore / components, this.config.minWeight), this.config.maxWeight)
|
||||||
|
: this.config.baseWeight;
|
||||||
|
const confidence = Math.min(weight + 0.2, 1.0);
|
||||||
|
return { weight, confidence, reasoning };
|
||||||
|
}
|
||||||
|
async onShutdown() {
|
||||||
|
const stats = this.getStats();
|
||||||
|
this.log(`Intelligent verb scoring shutdown: ${stats.relationshipsScored} relationships scored, ${Math.round(stats.averageConfidence * 100)}% avg confidence`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
//# sourceMappingURL=intelligentVerbScoringAugmentation.js.map
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue