Compare commits
No commits in common. "feature/enterprise-improvements" and "main" have entirely different histories.
feature/en
...
main
1137 changed files with 237515 additions and 148438 deletions
12
.aiignore
Normal file
12
.aiignore
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
# An .aiignore file follows the same syntax as a .gitignore file.
|
||||||
|
# .gitignore documentation: https://git-scm.com/docs/gitignore
|
||||||
|
|
||||||
|
# you can ignore files
|
||||||
|
.DS_Store
|
||||||
|
*.log
|
||||||
|
*.tmp
|
||||||
|
|
||||||
|
# or folders
|
||||||
|
dist/
|
||||||
|
build/
|
||||||
|
out/
|
||||||
170
.claude/skills/architecture.md
Normal file
170
.claude/skills/architecture.md
Normal file
|
|
@ -0,0 +1,170 @@
|
||||||
|
# Brainy Architecture Reference
|
||||||
|
|
||||||
|
## What Is Brainy
|
||||||
|
|
||||||
|
@soulcraft/brainy (v7.17.0) is a Universal Knowledge Protocol -- a Triple Intelligence database combining vector search, graph traversal, and metadata filtering in a single library. Published to npm as a public MIT-licensed package.
|
||||||
|
|
||||||
|
## Core Architecture
|
||||||
|
|
||||||
|
### Storage Layer (`src/storage/`)
|
||||||
|
- **StorageAdapter interface** (`src/coreTypes.ts:576`): The contract ALL storage backends implement. ALWAYS check this interface before adding storage methods.
|
||||||
|
- **BaseStorage** (`src/storage/baseStorage.ts`): Base implementation with built-in type-aware partitioning (TypeAwareStorageAdapter was removed -- functionality merged into BaseStorage).
|
||||||
|
- **Adapters** (`src/storage/adapters/`):
|
||||||
|
- `fileSystemStorage.ts` -- local filesystem
|
||||||
|
- `memoryStorage.ts` -- in-memory
|
||||||
|
- `baseStorageAdapter.ts` -- shared adapter base (counts, batch ops)
|
||||||
|
- Cloud + OPFS adapters were removed in 8.0 (cloud backup is operator tooling)
|
||||||
|
- **Generational MVCC / Db API** (`src/db/`): immutable `Db` values over generation-stamped records
|
||||||
|
- `db.ts` (the `Db` value), `generationStore.ts` (record layer + commit protocol), `types.ts`, `errors.ts`, `whereMatcher.ts`
|
||||||
|
- Design record: `docs/ADR-001-generational-mvcc.md`; replaced the pre-8.0 COW branching + versioning subsystems
|
||||||
|
|
||||||
|
### Vector Search (`src/hnsw/`)
|
||||||
|
- `hnswIndex.ts` -- HNSW-based approximate nearest neighbor search
|
||||||
|
- `typeAwareHNSWIndex.ts` -- type-partitioned vector search
|
||||||
|
- NOT in `src/intelligence/` (that directory does not exist)
|
||||||
|
|
||||||
|
### Graph Engine (`src/graph/`)
|
||||||
|
- `graphAdjacencyIndex.ts` -- adjacency-based graph representation
|
||||||
|
- `pathfinding.ts` -- relationship traversal and pathfinding
|
||||||
|
- `lsm/` -- LSM tree implementation for graph storage
|
||||||
|
|
||||||
|
### Metadata Index (`src/utils/metadataIndex.ts`)
|
||||||
|
- O(1) exact match via hash indexes
|
||||||
|
- O(log n) range queries via sorted indexes
|
||||||
|
- Roaring bitmap set operations for efficient filtering
|
||||||
|
- Adaptive chunking strategy (`metadataIndexChunking.ts`)
|
||||||
|
- Caching layer (`metadataIndexCache.ts`)
|
||||||
|
|
||||||
|
### Triple Intelligence (`src/triple/`)
|
||||||
|
- `TripleIntelligenceSystem.ts` -- combines vector + graph + metadata into unified queries
|
||||||
|
- Lazy-loaded indexes (loaded on first use, not at startup)
|
||||||
|
|
||||||
|
### Neural/AI Components (`src/neural/`)
|
||||||
|
- Smart Importers (`src/importers/`): CSV, Excel, PDF, DOCX, YAML, JSON, Markdown, Orchestrator
|
||||||
|
- `SmartExtractor.ts` -- entity extraction from unstructured data
|
||||||
|
- `SmartRelationshipExtractor.ts` -- relationship detection
|
||||||
|
- `NeuralEntityExtractor.ts` -- ML-based entity recognition
|
||||||
|
- Natural language processing utilities
|
||||||
|
|
||||||
|
### Distributed Systems (`src/distributed/`)
|
||||||
|
- Distributed Coordinator for multi-node operation
|
||||||
|
- Shard Manager for data partitioning
|
||||||
|
- Cache Synchronization across nodes
|
||||||
|
- Read/Write separation
|
||||||
|
- Network and HTTP transport layers
|
||||||
|
- Storage discovery and shard migration
|
||||||
|
|
||||||
|
### Transaction Management (`src/transaction/`)
|
||||||
|
- TransactionManager for ACID operations
|
||||||
|
- Operations: SaveNoun, AddToHNSW, UpdateMetadata, etc.
|
||||||
|
- Distributed transaction support
|
||||||
|
|
||||||
|
### Integration Hub (`src/integrations/`)
|
||||||
|
- Google Sheets integration
|
||||||
|
- OData (Open Data Protocol)
|
||||||
|
- Server-Sent Events (SSE)
|
||||||
|
- Webhooks
|
||||||
|
- Event bus system
|
||||||
|
|
||||||
|
### Virtual Filesystem (`src/vfs/`)
|
||||||
|
- `VirtualFileSystem.ts` -- full VFS implementation (87 KB)
|
||||||
|
- `PathResolver.ts`, `FSCompat.ts`, `MimeTypeDetector.ts`, `TreeUtils.ts`
|
||||||
|
- Subdirectories: `semantic/` (semantic search), `streams/` (streaming), `importers/`
|
||||||
|
|
||||||
|
### MCP Support (`src/mcp/`)
|
||||||
|
- BrainyMCPAdapter, MCPAugmentationToolset, BrainyMCPService
|
||||||
|
- Model Control Protocol request/response handling
|
||||||
|
|
||||||
|
### Aggregation Engine (`src/aggregation/`)
|
||||||
|
- **AggregationIndex** (`AggregationIndex.ts`): Write-time incremental aggregation — SUM, COUNT, AVG, MIN, MAX with GROUP BY and time windows
|
||||||
|
- **Time Windows** (`timeWindows.ts`): ISO 8601 bucketing — hour, day, week, month, quarter, year, custom intervals
|
||||||
|
- **Materializer** (`materializer.ts`): Debounced writes of aggregate results as `NounType.Measurement` entities
|
||||||
|
- Integrates into `brain.find({ aggregate })` for unified query API
|
||||||
|
- Write hooks in `add()`, `update()`, `delete()` for O(1) incremental updates
|
||||||
|
- `'aggregation'` provider key enables native plugin acceleration
|
||||||
|
|
||||||
|
### Additional Systems
|
||||||
|
- **CLI** (`src/cli/`): Complete command-line tool with interactive mode and catalog system
|
||||||
|
- **Migration** (`src/migration/`): MigrationRunner for database schema migrations
|
||||||
|
- **Embeddings** (`src/embeddings/`): Embedding manager with Candle-WASM Rust source
|
||||||
|
- **Streaming** (`src/streaming/`): Pipeline support with adaptive backpressure
|
||||||
|
- **Versioning** (`src/versioning/`): VersioningAPI for data versioning
|
||||||
|
- **Plugin System**: Registry-based plugin architecture
|
||||||
|
- **Patterns** (`src/patterns/`): 7 pattern library JSON files
|
||||||
|
|
||||||
|
## Type System
|
||||||
|
- **NounType** (42 types, `src/types/graphTypes.ts:850-893`): Person, Organization, Concept, Collection, Document, Task, Project, etc.
|
||||||
|
- **VerbType** (127 types, `src/types/graphTypes.ts:900-1087`): Contains, RelatedTo, PartOf, Creates, DependsOn, MemberOf, etc.
|
||||||
|
- All types in `src/types/`
|
||||||
|
|
||||||
|
## Module Exports (`src/index.ts`)
|
||||||
|
38+ named exports including: Brainy class, configuration types, neural APIs (NeuralImport, NeuralEntityExtractor, SmartExtractor, SmartRelationshipExtractor), distance functions, plugin system, migration system, embedding functions, storage adapters, COW infrastructure, pipeline utilities, graph types, MCP components, integration hub, OData utilities, and more.
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
```
|
||||||
|
src/
|
||||||
|
├── index.ts # 38+ public exports
|
||||||
|
├── brainy.ts # Main Brainy class (6,500+ lines)
|
||||||
|
├── setup.ts # Initialization polyfills
|
||||||
|
├── coreTypes.ts # StorageAdapter interface + core types
|
||||||
|
├── storage/
|
||||||
|
│ ├── baseStorage.ts # Base storage (includes type-aware)
|
||||||
|
│ ├── adapters/ # All storage backends + cloud adapters
|
||||||
|
│ └── cow/ # Copy-on-Write versioning
|
||||||
|
├── hnsw/ # HNSW vector search
|
||||||
|
├── graph/ # Graph engine + pathfinding + LSM
|
||||||
|
├── triple/ # Triple Intelligence system
|
||||||
|
├── neural/ # Smart extractors + NLP
|
||||||
|
├── importers/ # File format importers (8 types)
|
||||||
|
├── distributed/ # Distributed database (16 files)
|
||||||
|
├── transaction/ # ACID transactions (6 files)
|
||||||
|
├── integrations/ # Sheets, OData, SSE, Webhooks
|
||||||
|
├── vfs/ # Virtual filesystem + semantic search
|
||||||
|
├── mcp/ # Model Control Protocol
|
||||||
|
├── cli/ # Command-line interface
|
||||||
|
├── migration/ # Schema migrations
|
||||||
|
├── embeddings/ # Embedding manager + Candle-WASM
|
||||||
|
├── streaming/ # Pipeline + backpressure
|
||||||
|
├── versioning/ # Versioning API
|
||||||
|
├── types/ # TypeScript type definitions
|
||||||
|
├── utils/ # Metadata index, logging, etc.
|
||||||
|
├── config/ # Configuration system
|
||||||
|
├── patterns/ # Pattern library
|
||||||
|
├── api/ # API layer
|
||||||
|
├── interfaces/ # Interface definitions
|
||||||
|
├── shared/ # Shared utilities
|
||||||
|
├── data/ # Data utilities
|
||||||
|
├── errors/ # Error handling
|
||||||
|
├── critical/ # Critical error handling
|
||||||
|
├── universal/ # Universal utilities
|
||||||
|
├── import/ # Import functionality
|
||||||
|
└── scripts/ # Build scripts
|
||||||
|
```
|
||||||
|
|
||||||
|
## Initialization
|
||||||
|
`brainy.ts` `init()` method performs initialization cascade:
|
||||||
|
1. Load plugins
|
||||||
|
2. Initialize storage
|
||||||
|
3. Enable COW (Copy-on-Write)
|
||||||
|
4. Set up embeddings
|
||||||
|
5. Initialize caches
|
||||||
|
6. Set up graph indexes
|
||||||
|
7. Initialize VFS
|
||||||
|
8. Set up transaction manager
|
||||||
|
9. Initialize distributed components (if enabled)
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
- Framework: Vitest
|
||||||
|
- Run: `npm test`
|
||||||
|
- Test directories:
|
||||||
|
- `tests/unit/` -- unit tests
|
||||||
|
- `tests/integration/` -- integration tests
|
||||||
|
- `tests/benchmarks/` -- performance benchmarks (NOT tests/performance/)
|
||||||
|
- `tests/comprehensive/` -- comprehensive test suites
|
||||||
|
- `tests/api/` -- API tests
|
||||||
|
- `tests/helpers/` -- test utilities
|
||||||
|
|
||||||
|
## Release
|
||||||
|
- `npm run release:patch/minor/major` -- fully automated via `scripts/release.sh`
|
||||||
|
- `npm run release:dry` -- preview without changes
|
||||||
|
- Uses conventional commits for changelog generation
|
||||||
57
.dockerignore
Normal file
57
.dockerignore
Normal file
|
|
@ -0,0 +1,57 @@
|
||||||
|
# Git
|
||||||
|
.git
|
||||||
|
.gitignore
|
||||||
|
|
||||||
|
# Development
|
||||||
|
.vscode
|
||||||
|
.idea
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
.DS_Store
|
||||||
|
|
||||||
|
# Node
|
||||||
|
node_modules
|
||||||
|
npm-debug.log*
|
||||||
|
yarn-debug.log*
|
||||||
|
yarn-error.log*
|
||||||
|
|
||||||
|
# Testing
|
||||||
|
tests
|
||||||
|
*.test.ts
|
||||||
|
*.test.js
|
||||||
|
coverage
|
||||||
|
.nyc_output
|
||||||
|
|
||||||
|
# Documentation (keep only essentials)
|
||||||
|
docs
|
||||||
|
*.md
|
||||||
|
!README.md
|
||||||
|
!LICENSE
|
||||||
|
|
||||||
|
# Build artifacts (will be built in Docker)
|
||||||
|
dist
|
||||||
|
build
|
||||||
|
*.tsbuildinfo
|
||||||
|
|
||||||
|
# Environment
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
|
||||||
|
# Strategy and private docs
|
||||||
|
.strategy
|
||||||
|
CLAUDE.md
|
||||||
|
|
||||||
|
# Development files
|
||||||
|
docker-compose.yml
|
||||||
|
Dockerfile
|
||||||
|
.dockerignore
|
||||||
|
|
||||||
|
# Data (should be mounted, not baked in)
|
||||||
|
data
|
||||||
|
*.db
|
||||||
|
*.sqlite
|
||||||
|
|
||||||
|
# Models (should be downloaded at runtime or mounted)
|
||||||
|
models
|
||||||
|
*.onnx
|
||||||
|
*.bin
|
||||||
18
.github/pull_request_template.md
vendored
18
.github/pull_request_template.md
vendored
|
|
@ -1,18 +0,0 @@
|
||||||
# 🚨 CRITICAL: Public Repository Checklist
|
|
||||||
|
|
||||||
## Before merging this PR, confirm:
|
|
||||||
|
|
||||||
- [ ] **No business strategy** - No pricing, revenue plans, or commercial roadmaps
|
|
||||||
- [ ] **No customer info** - No specific customer details or enterprise discussions
|
|
||||||
- [ ] **No infrastructure details** - No Google Cloud project IDs or internal URLs
|
|
||||||
- [ ] **No SkillVill content** - All SkillVill planning belongs in brain-cloud repo
|
|
||||||
- [ ] **Open source focused** - Content appropriate for public consumption
|
|
||||||
- [ ] **Registry references OK** - Only registry.soulcraft.com endpoint references allowed
|
|
||||||
|
|
||||||
## ✅ This PR contains only:
|
|
||||||
- [ ] Open source Brainy library improvements
|
|
||||||
- [ ] Generic documentation updates
|
|
||||||
- [ ] CLI enhancements (registry integration OK)
|
|
||||||
- [ ] Community-focused features
|
|
||||||
|
|
||||||
**🛡️ When in doubt, move content to brain-cloud private repo instead!**
|
|
||||||
40
.github/workflows/ci.yml
vendored
Normal file
40
.github/workflows/ci.yml
vendored
Normal file
|
|
@ -0,0 +1,40 @@
|
||||||
|
name: CI
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
pull_request:
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
node:
|
||||||
|
name: Node ${{ matrix.node-version }}
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
strategy:
|
||||||
|
fail-fast: false
|
||||||
|
matrix:
|
||||||
|
node-version: ['22', '24']
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: ${{ matrix.node-version }}
|
||||||
|
cache: npm
|
||||||
|
- run: npm ci
|
||||||
|
- run: npm run test:unit
|
||||||
|
|
||||||
|
bun:
|
||||||
|
name: Bun (latest)
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: '22'
|
||||||
|
cache: npm
|
||||||
|
- uses: oven-sh/setup-bun@v2
|
||||||
|
with:
|
||||||
|
bun-version: latest
|
||||||
|
- run: npm ci
|
||||||
|
# test:bun imports the built dist/, so build first.
|
||||||
|
- run: npm run build
|
||||||
|
# Bun as a runtime is the supported Bun story (`bun add` / `bun run`).
|
||||||
|
- run: npm run test:bun
|
||||||
43
.github/workflows/repo-audit.yml
vendored
43
.github/workflows/repo-audit.yml
vendored
|
|
@ -1,43 +0,0 @@
|
||||||
name: 🚨 Repository Content Audit
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [ main ]
|
|
||||||
pull_request:
|
|
||||||
branches: [ main ]
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
audit-content:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: 🔍 Scan for commercial content
|
|
||||||
run: |
|
|
||||||
echo "🚨 Scanning for prohibited content..."
|
|
||||||
|
|
||||||
# Scan for business terms
|
|
||||||
if grep -r -i "skillvill\|revenue\|pricing.*\$[0-9]\|enterprise.*plan\|business.*plan.*\$" . --exclude-dir=.git --exclude-dir=node_modules; then
|
|
||||||
echo "❌ FOUND BUSINESS CONTENT IN PUBLIC REPO!"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Scan for infrastructure details
|
|
||||||
if grep -r "476163328636\|brain-cloud-api.*\.run\.app" . --exclude-dir=.git --exclude-dir=node_modules; then
|
|
||||||
echo "❌ FOUND INFRASTRUCTURE DETAILS IN PUBLIC REPO!"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Scan for customer info
|
|
||||||
if grep -r -i "customer.*\$\|client.*enterprise\|acme.*corp.*\$" . --exclude-dir=.git --exclude-dir=node_modules; then
|
|
||||||
echo "❌ FOUND CUSTOMER INFO IN PUBLIC REPO!"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "✅ Content audit passed - repository is clean!"
|
|
||||||
|
|
||||||
- name: 📝 Verify allowed content
|
|
||||||
run: |
|
|
||||||
echo "✅ Checking for required open source content..."
|
|
||||||
ls -la src/ bin/ README.md package.json
|
|
||||||
echo "✅ All good - open source content present"
|
|
||||||
147
.gitignore
vendored
147
.gitignore
vendored
|
|
@ -1,65 +1,116 @@
|
||||||
# Dependencies
|
# Dependencies
|
||||||
node_modules/
|
node_modules/
|
||||||
.pnp
|
|
||||||
.pnp.js
|
|
||||||
|
|
||||||
# Testing
|
|
||||||
coverage/
|
|
||||||
*.lcov
|
|
||||||
.nyc_output
|
|
||||||
|
|
||||||
# Production build
|
|
||||||
dist/
|
|
||||||
|
|
||||||
# Models (downloaded at runtime)
|
|
||||||
models/
|
|
||||||
|
|
||||||
# Runtime data
|
|
||||||
brainy-data/
|
|
||||||
*.log
|
|
||||||
npm-debug.log*
|
npm-debug.log*
|
||||||
yarn-debug.log*
|
yarn-debug.log*
|
||||||
yarn-error.log*
|
yarn-error.log*
|
||||||
lerna-debug.log*
|
|
||||||
|
|
||||||
# OS files
|
# Build outputs
|
||||||
.DS_Store
|
dist/
|
||||||
Thumbs.db
|
build/
|
||||||
|
*.tsbuildinfo
|
||||||
|
|
||||||
# IDE
|
# Environment variables
|
||||||
|
.env
|
||||||
|
.env.local
|
||||||
|
.env.development.local
|
||||||
|
.env.test.local
|
||||||
|
.env.production.local
|
||||||
|
|
||||||
|
# Runtime data
|
||||||
|
brainy-data/
|
||||||
|
.brainy/
|
||||||
|
*.log
|
||||||
|
*.pid
|
||||||
|
*.seed
|
||||||
|
*.pid.lock
|
||||||
|
|
||||||
|
# Coverage directory used by tools like istanbul
|
||||||
|
coverage/
|
||||||
|
*.lcov
|
||||||
|
|
||||||
|
# Test results
|
||||||
|
tests/results/
|
||||||
|
|
||||||
|
# Filesystem test artifacts (created by integration tests)
|
||||||
|
test-*/
|
||||||
|
|
||||||
|
# IDE files
|
||||||
.vscode/
|
.vscode/
|
||||||
.idea/
|
.idea/
|
||||||
*.swp
|
*.swp
|
||||||
*.swo
|
*.swo
|
||||||
*~
|
*~
|
||||||
|
|
||||||
# Environment
|
# OS files
|
||||||
.env
|
.DS_Store
|
||||||
.env.local
|
Thumbs.db
|
||||||
.env.*.local
|
|
||||||
|
|
||||||
# Debug
|
|
||||||
npm-debug.log*
|
|
||||||
yarn-debug.log*
|
|
||||||
yarn-error.log*
|
|
||||||
|
|
||||||
# Cache
|
|
||||||
.npm
|
|
||||||
.eslintcache
|
|
||||||
.cache/
|
|
||||||
|
|
||||||
# Optional npm cache directory
|
|
||||||
.npm
|
|
||||||
|
|
||||||
# Optional eslint cache
|
|
||||||
.eslintcache
|
|
||||||
|
|
||||||
# Temporary files
|
# Temporary files
|
||||||
|
tmp/
|
||||||
|
temp/
|
||||||
*.tmp
|
*.tmp
|
||||||
*.temp
|
|
||||||
/tmp/
|
|
||||||
|
|
||||||
# Test artifacts
|
# Planning and instruction files
|
||||||
test-results/
|
plan.md
|
||||||
tests/results/models-cache/
|
|
||||||
CLAUDE.md
|
# Package files
|
||||||
|
*.tgz
|
||||||
|
|
||||||
|
# Private/confidential files
|
||||||
|
PLAN.md
|
||||||
|
INTERNAL_NOTES.md
|
||||||
|
TODO_PRIVATE.md
|
||||||
|
*.tar.gz
|
||||||
|
|
||||||
|
# Strategy and planning documents (private)
|
||||||
|
.strategy/
|
||||||
|
# Removed: PRODUCTION_*.md (now these should be public documentation)
|
||||||
|
DISTRIBUTED_*.md
|
||||||
|
*_ASSESSMENT.md
|
||||||
|
*_ANALYSIS.md
|
||||||
|
*_TRUTH*.md
|
||||||
|
|
||||||
|
# Models (downloaded at runtime)
|
||||||
|
models/
|
||||||
|
models-cache/
|
||||||
|
|
||||||
|
# But include bundled WASM model assets
|
||||||
|
!assets/models/
|
||||||
|
|
||||||
|
# Development planning files (not for commit)
|
||||||
|
PLAN.md
|
||||||
|
|
||||||
|
# Backup folders
|
||||||
|
backup-*
|
||||||
|
backup/
|
||||||
|
|
||||||
|
# Internal documentation
|
||||||
|
docs/internal/
|
||||||
|
|
||||||
|
# Cache files
|
||||||
|
*.cache
|
||||||
|
|
||||||
|
# Rust/Cargo build artifacts
|
||||||
|
src/embeddings/candle-wasm/target/
|
||||||
|
src/embeddings/candle-wasm/Cargo.lock
|
||||||
|
|
||||||
|
# Ignore the wasm-pack output dir's CONTENTS (note the `/*`, not `/`, so the
|
||||||
|
# re-includes below can take effect — git cannot re-include a file whose parent
|
||||||
|
# DIRECTORY is excluded). Keep the pre-built WASM committed: it ships in the npm
|
||||||
|
# package anyway, it lets consumers + CI build without a Rust/wasm-pack toolchain,
|
||||||
|
# and versioning it makes the shipped artifact reproducible (not "whatever the
|
||||||
|
# maintainer last built").
|
||||||
|
src/embeddings/wasm/pkg/*
|
||||||
|
!src/embeddings/wasm/pkg/*.wasm
|
||||||
|
!src/embeddings/wasm/pkg/*.js
|
||||||
|
!src/embeddings/wasm/pkg/*.d.ts
|
||||||
|
|
||||||
|
# Log files (redundant but explicit)
|
||||||
|
*.log
|
||||||
|
|
||||||
|
# Temporary files (redundant but explicit)
|
||||||
|
*.tmp
|
||||||
|
/.junie/guidelines.md
|
||||||
|
|
||||||
|
# Claude Code harness state
|
||||||
|
.claude/scheduled_tasks.lock
|
||||||
|
|
|
||||||
84
.npmignore
Normal file
84
.npmignore
Normal file
|
|
@ -0,0 +1,84 @@
|
||||||
|
# Source files (not needed in package)
|
||||||
|
src/
|
||||||
|
tests/
|
||||||
|
scripts/
|
||||||
|
coverage/
|
||||||
|
|
||||||
|
# Model files (downloaded on first use, not bundled)
|
||||||
|
models/
|
||||||
|
models-cache/
|
||||||
|
|
||||||
|
# Development and backup files
|
||||||
|
backup-*
|
||||||
|
backup-*/
|
||||||
|
docs/backup*/
|
||||||
|
|
||||||
|
# Documentation (except essentials)
|
||||||
|
*.md
|
||||||
|
!README.md
|
||||||
|
!LICENSE
|
||||||
|
!CHANGELOG.md
|
||||||
|
!MIGRATION.md
|
||||||
|
|
||||||
|
# Configuration files
|
||||||
|
.gitignore
|
||||||
|
.npmignore
|
||||||
|
tsconfig.json
|
||||||
|
vitest.config.ts
|
||||||
|
vitest.config.mts
|
||||||
|
*.config.js
|
||||||
|
*.config.ts
|
||||||
|
.eslintrc*
|
||||||
|
.prettierrc*
|
||||||
|
|
||||||
|
# Test files
|
||||||
|
test-*.js
|
||||||
|
test-*.ts
|
||||||
|
*.test.ts
|
||||||
|
*.test.js
|
||||||
|
*.spec.ts
|
||||||
|
*.spec.js
|
||||||
|
|
||||||
|
# Temporary and log files
|
||||||
|
*.log
|
||||||
|
*.tmp
|
||||||
|
tmp/
|
||||||
|
temp/
|
||||||
|
brainy-data/
|
||||||
|
|
||||||
|
# Git and CI files
|
||||||
|
.git/
|
||||||
|
.github/
|
||||||
|
.gitlab-ci.yml
|
||||||
|
.travis.yml
|
||||||
|
|
||||||
|
# IDE files
|
||||||
|
.vscode/
|
||||||
|
.idea/
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
|
||||||
|
# OS files
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
# Private files
|
||||||
|
PLAN.md
|
||||||
|
CLAUDE.md
|
||||||
|
INTERNAL_NOTES.md
|
||||||
|
TODO_PRIVATE.md
|
||||||
|
*-ANALYSIS.md
|
||||||
|
*-PLAN.md
|
||||||
|
|
||||||
|
# Build artifacts not needed
|
||||||
|
*.tsbuildinfo
|
||||||
|
*.map
|
||||||
|
|
||||||
|
# Development environment
|
||||||
|
.env*
|
||||||
|
.nvm*
|
||||||
|
.node-version
|
||||||
|
|
||||||
|
# Keep dist/ for the compiled code
|
||||||
|
# Keep bin/ for the CLI
|
||||||
|
# Keep package.json, package-lock.json
|
||||||
1
.nvmrc
Normal file
1
.nvmrc
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
22
|
||||||
4720
CHANGELOG.md
4720
CHANGELOG.md
File diff suppressed because it is too large
Load diff
208
CLAUDE.md
Normal file
208
CLAUDE.md
Normal file
|
|
@ -0,0 +1,208 @@
|
||||||
|
# Brainy - Claude Code Project Guide
|
||||||
|
|
||||||
|
This file provides guidance for Claude Code (and human contributors) when working on the Brainy codebase.
|
||||||
|
|
||||||
|
## Cross-Project Coordination
|
||||||
|
|
||||||
|
Handoff file: `/home/dpsifr/.strategy/PLATFORM-HANDOFF.md`
|
||||||
|
|
||||||
|
**At session START:** Read the handoff. Find rows where Owner = Brainy. Act on those first.
|
||||||
|
|
||||||
|
**At session END:** Mark completed actions ✅, delete rows you finished, delete threads with zero remaining actions. File must not grow. **If you shipped anything consumers need to know about, update `RELEASES.md` before closing.**
|
||||||
|
|
||||||
|
**Brainy's current open actions:** None. MIT open-source — no platform-specific actions.
|
||||||
|
|
||||||
|
**Current version:** `@soulcraft/brainy@7.31.5` (latest published; 8.0.0 release candidate on `feat/8.0-u64-ids`)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
Brainy is a Universal Knowledge Protocol -- a Triple Intelligence database that combines vector similarity search, graph traversal, and metadata filtering into a single TypeScript library. Published as `@soulcraft/brainy` on npm under the MIT license.
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm install # Install dependencies
|
||||||
|
npm run build # Build the project
|
||||||
|
npm test # Run test suite (Vitest)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
Full architecture reference: `.claude/skills/architecture.md`
|
||||||
|
|
||||||
|
### Core Systems
|
||||||
|
- **Storage** (`src/storage/`): Pluggable storage backends via StorageAdapter interface (`src/coreTypes.ts`)
|
||||||
|
- **Vector Search** (`src/hnsw/`): HNSW approximate nearest neighbor search
|
||||||
|
- **Graph Engine** (`src/graph/`): Relationship traversal with adjacency index and pathfinding
|
||||||
|
- **Metadata Index** (`src/utils/metadataIndex.ts`): O(1) exact match, O(log n) range queries
|
||||||
|
- **Triple Intelligence** (`src/triple/`): Unified query combining all three intelligence types
|
||||||
|
- **Aggregation Engine** (`src/aggregation/`): Write-time incremental SUM/COUNT/AVG/MIN/MAX with GROUP BY and time windows
|
||||||
|
- **Virtual Filesystem** (`src/vfs/`): Full VFS with semantic search
|
||||||
|
|
||||||
|
### Type System
|
||||||
|
- **NounType** (42 types): Entity classification -- Person, Concept, Collection, Document, Task, etc.
|
||||||
|
- **VerbType** (127 types): Relationship types -- Contains, RelatedTo, PartOf, Creates, DependsOn, etc.
|
||||||
|
- Defined in `src/types/graphTypes.ts`
|
||||||
|
|
||||||
|
## Code Standards
|
||||||
|
|
||||||
|
### TypeScript
|
||||||
|
- Strict mode enabled
|
||||||
|
- Target: ES2020, NodeNext module resolution
|
||||||
|
- All new code must be TypeScript
|
||||||
|
- Follow existing patterns -- read related code before writing
|
||||||
|
|
||||||
|
### Quality
|
||||||
|
- All code must compile without errors
|
||||||
|
- All code must have working tests that exercise real behavior
|
||||||
|
- No stub returns (`return {} as any`)
|
||||||
|
- No incomplete implementations with TODO comments
|
||||||
|
- If something can't be fully implemented, throw an explicit error rather than faking it
|
||||||
|
|
||||||
|
### Verification Before Code Changes
|
||||||
|
1. Check that interfaces and methods actually exist before using them
|
||||||
|
2. Check that type properties are in the type definitions
|
||||||
|
3. Run `npm test` -- tests must pass
|
||||||
|
4. Run `npm run build` -- build must succeed
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
- Framework: Vitest
|
||||||
|
- Tests in `tests/` (unit, integration, benchmarks, comprehensive)
|
||||||
|
- Use in-memory storage for speed where possible
|
||||||
|
- Tests must exercise real behavior, not mock it
|
||||||
|
- Benchmarks are in `tests/benchmarks/` (not tests/performance/)
|
||||||
|
|
||||||
|
## Commit Conventions
|
||||||
|
|
||||||
|
Use [Conventional Commits](https://www.conventionalcommits.org/):
|
||||||
|
|
||||||
|
```
|
||||||
|
feat: add new feature (minor version bump)
|
||||||
|
fix: resolve bug (patch version bump)
|
||||||
|
docs: update documentation (patch version bump)
|
||||||
|
perf: improve performance (patch version bump)
|
||||||
|
refactor: restructure code (patch version bump)
|
||||||
|
test: add/update tests (patch version bump)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Important:** Never use `BREAKING CHANGE` in commit messages. Major version bumps are manual decisions only (`npm run release:major`).
|
||||||
|
|
||||||
|
## Docs Pipeline — soulcraft.com/docs
|
||||||
|
|
||||||
|
Docs in `docs/**/*.md` are published with the npm package (included in `files`) and synced to soulcraft.com/docs on every portal deploy. Frontmatter controls what appears publicly.
|
||||||
|
|
||||||
|
### Docs check triggers
|
||||||
|
|
||||||
|
Run the docs check whenever the user says ANY of:
|
||||||
|
- "commit, publish, release" / "release" / "publish"
|
||||||
|
- "update the docs" / "make sure docs are accurate" / "check the docs"
|
||||||
|
- "review docs" / "clean up docs"
|
||||||
|
|
||||||
|
### Pre-release docs check (MANDATORY before every release)
|
||||||
|
|
||||||
|
When the user says "commit, publish, release" or any variation, **before committing**:
|
||||||
|
|
||||||
|
1. **Scan all files changed in this session** (and any recently added `docs/*.md` files)
|
||||||
|
2. For each changed/new doc, decide: is this useful to external users?
|
||||||
|
- **Yes** → ensure it has complete frontmatter (add or update it)
|
||||||
|
- **No** (internal, migration, dev-only) → ensure it has no frontmatter or `public: false`
|
||||||
|
3. For docs that already have frontmatter, verify:
|
||||||
|
- `description` still matches the actual content
|
||||||
|
- `next` links still exist and are still the right follow-up pages
|
||||||
|
- `title` matches the doc's h1
|
||||||
|
4. Include frontmatter changes in the commit
|
||||||
|
|
||||||
|
### Frontmatter format
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
title: Human-readable title
|
||||||
|
slug: category/page-name # URL: soulcraft.com/docs/category/page-name
|
||||||
|
public: true # false or absent = not published
|
||||||
|
category: getting-started | concepts | guides | api
|
||||||
|
template: guide | concept | api # controls layout on soulcraft.com
|
||||||
|
order: 1 # sidebar position within category (lower = first)
|
||||||
|
description: One sentence. What this doc covers and why it matters.
|
||||||
|
next: # "Next steps" links shown at bottom of page
|
||||||
|
- category/other-slug
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
### Category guide
|
||||||
|
|
||||||
|
| category | use for |
|
||||||
|
|----------|---------|
|
||||||
|
| `getting-started` | installation, quick start, first steps |
|
||||||
|
| `concepts` | how the system works, mental models |
|
||||||
|
| `guides` | how to do specific things, recipes |
|
||||||
|
| `api` | method reference, signatures, parameters |
|
||||||
|
|
||||||
|
### What stays internal (no frontmatter / `public: false`)
|
||||||
|
|
||||||
|
- Release guides, developer learning paths
|
||||||
|
- Migration guides for old versions (v3→v4, v5.11)
|
||||||
|
- Architecture analysis docs (clustering algorithms, etc.)
|
||||||
|
- Anything in `docs/internal/`
|
||||||
|
- Deployment/ops/cost docs (cloud-run, kubernetes, cost-optimization)
|
||||||
|
|
||||||
|
## Release Process
|
||||||
|
|
||||||
|
Fully automated via `scripts/release.sh`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run release:dry # Preview (no changes)
|
||||||
|
npm run release:patch # Bug fixes
|
||||||
|
npm run release:minor # New features
|
||||||
|
npm run release:major # Breaking changes (rare, manual decision)
|
||||||
|
```
|
||||||
|
|
||||||
|
The script: verifies clean git state, builds, tests, bumps version, updates CHANGELOG.md, commits, tags, pushes, publishes to npm, and creates a GitHub release.
|
||||||
|
|
||||||
|
After a successful release, remind the user:
|
||||||
|
> "Published. Deploy portal to pick up the new docs → go to the portal project and deploy."
|
||||||
|
|
||||||
|
Do NOT deploy portal from here. Portal is always deployed separately from within the portal project.
|
||||||
|
|
||||||
|
## Closed-Source Product Names — HARD RULE
|
||||||
|
|
||||||
|
Brainy is the only Soulcraft open-source project. Nothing in this repo — code, JSDoc, tests,
|
||||||
|
docs, RELEASES.md, CHANGELOG.md, commit messages — may reference closed-source Soulcraft
|
||||||
|
products by name (Workshop, Venue, Memory, Muse, Hall, Forge, Academy, Pulse, Heart,
|
||||||
|
Collective, SDK) or by their specific class/method names (`BookingDraftService`,
|
||||||
|
`getDemandHeatmap`, `systemKind`, etc.).
|
||||||
|
|
||||||
|
When recording a consumer-reported bug, regression scenario, or release note:
|
||||||
|
- Refer to "a consumer", "a downstream application", "a production deployment", or "an
|
||||||
|
internal report" — never name the product.
|
||||||
|
- All doc examples must use generic domain values (`'employee'`, `'customer'`, `'invoice'`,
|
||||||
|
`'milestone'`, `OrderService`, `/orders/...`), not product-specific schemas.
|
||||||
|
- Internal session artifacts (`.strategy/`, `~/.claude/plans/`, handoff files outside the
|
||||||
|
repo) MAY name products — those are not public.
|
||||||
|
|
||||||
|
If you catch yourself typing a product name into a tracked file, stop and rephrase.
|
||||||
|
|
||||||
|
## Performance Claims
|
||||||
|
|
||||||
|
When documenting performance characteristics:
|
||||||
|
- **MEASURED**: Cite the test file and line number
|
||||||
|
- **PROJECTED**: Clearly label as extrapolated from tested scale
|
||||||
|
- Never claim a performance figure without context or evidence
|
||||||
|
|
||||||
|
## Debugging
|
||||||
|
|
||||||
|
When a bug persists through 2+ fix attempts, switch to systematic debugging:
|
||||||
|
1. Add comprehensive logging at every step
|
||||||
|
2. Test with production-like data
|
||||||
|
3. Trace the complete execution path
|
||||||
|
4. Check both library code and consumer code
|
||||||
|
5. Verify with actual test execution before declaring fixed
|
||||||
|
|
||||||
|
## Key Paths
|
||||||
|
|
||||||
|
- Main class: `src/brainy.ts`
|
||||||
|
- Public API: `src/index.ts` (38+ exports)
|
||||||
|
- Storage interface: `src/coreTypes.ts`
|
||||||
|
- Type definitions: `src/types/`
|
||||||
|
- Strategy/planning docs: `.strategy/` (gitignored, not public)
|
||||||
298
CONTRIBUTING.md
Normal file
298
CONTRIBUTING.md
Normal file
|
|
@ -0,0 +1,298 @@
|
||||||
|
# Contributing to Brainy
|
||||||
|
|
||||||
|
Thank you for your interest in contributing to Brainy! This document provides guidelines and instructions for contributing to the project.
|
||||||
|
|
||||||
|
## Code of Conduct
|
||||||
|
|
||||||
|
By participating in this project, you agree to abide by our Code of Conduct:
|
||||||
|
- Be respectful and inclusive
|
||||||
|
- Welcome newcomers and help them get started
|
||||||
|
- Focus on constructive criticism
|
||||||
|
- Respect differing viewpoints and experiences
|
||||||
|
|
||||||
|
## How to Contribute
|
||||||
|
|
||||||
|
### Reporting Issues
|
||||||
|
|
||||||
|
Before creating an issue, please check existing issues to avoid duplicates.
|
||||||
|
|
||||||
|
When creating an issue, include:
|
||||||
|
- Clear, descriptive title
|
||||||
|
- Detailed description of the problem
|
||||||
|
- Steps to reproduce
|
||||||
|
- Expected vs actual behavior
|
||||||
|
- System information (OS, Node version, Brainy version)
|
||||||
|
- Code examples if applicable
|
||||||
|
|
||||||
|
### Suggesting Features
|
||||||
|
|
||||||
|
Feature requests are welcome! Please provide:
|
||||||
|
- Clear use case
|
||||||
|
- Proposed API/interface
|
||||||
|
- Examples of how it would work
|
||||||
|
- Any potential challenges or considerations
|
||||||
|
|
||||||
|
### Pull Requests
|
||||||
|
|
||||||
|
#### Before Starting
|
||||||
|
|
||||||
|
1. Check existing issues and PRs
|
||||||
|
2. Open an issue to discuss significant changes
|
||||||
|
3. Fork the repository
|
||||||
|
4. Create a feature branch from `main`
|
||||||
|
|
||||||
|
#### Development Setup
|
||||||
|
|
||||||
|
**Quick Setup (Recommended):**
|
||||||
|
```bash
|
||||||
|
# Clone your fork
|
||||||
|
git clone https://github.com/your-username/brainy.git
|
||||||
|
cd brainy
|
||||||
|
|
||||||
|
# Run setup script (installs all dependencies including Rust)
|
||||||
|
./scripts/setup-dev.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
**Manual Setup:**
|
||||||
|
```bash
|
||||||
|
# Clone your fork
|
||||||
|
git clone https://github.com/your-username/brainy.git
|
||||||
|
cd brainy
|
||||||
|
|
||||||
|
# Install system dependencies (Ubuntu/Debian)
|
||||||
|
sudo apt-get install -y build-essential pkg-config libssl-dev
|
||||||
|
|
||||||
|
# Install Rust (for WASM embedding engine)
|
||||||
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
||||||
|
source ~/.cargo/env
|
||||||
|
rustup target add wasm32-unknown-unknown
|
||||||
|
cargo install wasm-pack
|
||||||
|
|
||||||
|
# Install Node.js dependencies
|
||||||
|
npm install
|
||||||
|
|
||||||
|
# Build Candle WASM embedding engine
|
||||||
|
npm run build:candle
|
||||||
|
|
||||||
|
# Build TypeScript
|
||||||
|
npm run build
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
npm test
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Making Changes
|
||||||
|
|
||||||
|
1. **Follow the code style**
|
||||||
|
- TypeScript for all source code
|
||||||
|
- Clear variable and function names
|
||||||
|
- Comments for complex logic
|
||||||
|
- JSDoc for public APIs
|
||||||
|
|
||||||
|
2. **Write tests**
|
||||||
|
- Add tests for new features
|
||||||
|
- Update tests for changes
|
||||||
|
- Ensure all tests pass
|
||||||
|
|
||||||
|
3. **Update documentation**
|
||||||
|
- Update README if needed
|
||||||
|
- Add/update API documentation
|
||||||
|
- Include examples
|
||||||
|
|
||||||
|
#### Commit Guidelines
|
||||||
|
|
||||||
|
Follow conventional commits format:
|
||||||
|
|
||||||
|
```
|
||||||
|
type(scope): description
|
||||||
|
|
||||||
|
[optional body]
|
||||||
|
|
||||||
|
[optional footer]
|
||||||
|
```
|
||||||
|
|
||||||
|
Types:
|
||||||
|
- `feat`: New feature
|
||||||
|
- `fix`: Bug fix
|
||||||
|
- `docs`: Documentation changes
|
||||||
|
- `style`: Code style changes
|
||||||
|
- `refactor`: Code refactoring
|
||||||
|
- `perf`: Performance improvements
|
||||||
|
- `test`: Test changes
|
||||||
|
- `chore`: Build/tooling changes
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
```bash
|
||||||
|
feat(triple): add graph traversal depth limit
|
||||||
|
fix(storage): handle concurrent write conflicts
|
||||||
|
docs(api): update search method documentation
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Submitting PR
|
||||||
|
|
||||||
|
1. Push to your fork
|
||||||
|
2. Create PR against `main` branch
|
||||||
|
3. Fill out PR template
|
||||||
|
4. Ensure CI checks pass
|
||||||
|
5. Wait for review
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
|
||||||
|
#### Running Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run all tests
|
||||||
|
npm test
|
||||||
|
|
||||||
|
# Run specific test file
|
||||||
|
npm test tests/core.test.ts
|
||||||
|
|
||||||
|
# Run with coverage
|
||||||
|
npm run test:coverage
|
||||||
|
|
||||||
|
# Watch mode
|
||||||
|
npm run test:watch
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Writing Tests
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { describe, it, expect } from 'vitest'
|
||||||
|
import { Brainy } from '../src'
|
||||||
|
|
||||||
|
describe('Feature Name', () => {
|
||||||
|
it('should do something specific', async () => {
|
||||||
|
const brain = new Brainy()
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
// Test implementation
|
||||||
|
const result = await brain.search("test")
|
||||||
|
|
||||||
|
expect(result).toBeDefined()
|
||||||
|
expect(result.length).toBeGreaterThan(0)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture Guidelines
|
||||||
|
|
||||||
|
### Adding New Features
|
||||||
|
|
||||||
|
1. **Check existing functionality**
|
||||||
|
- Review `ARCHITECTURE.md`
|
||||||
|
- Check if similar features exist
|
||||||
|
- Consider if it should be an augmentation
|
||||||
|
|
||||||
|
2. **Design considerations**
|
||||||
|
- Maintain backward compatibility
|
||||||
|
- Consider performance impact
|
||||||
|
- Think about all storage adapters
|
||||||
|
- Plan for extensibility
|
||||||
|
|
||||||
|
3. **Implementation checklist**
|
||||||
|
- [ ] Core functionality
|
||||||
|
- [ ] Tests (unit and integration)
|
||||||
|
- [ ] Documentation
|
||||||
|
- [ ] TypeScript types
|
||||||
|
- [ ] Examples
|
||||||
|
- [ ] Performance benchmarks (if applicable)
|
||||||
|
|
||||||
|
### Creating Augmentations
|
||||||
|
|
||||||
|
Augmentations extend Brainy's functionality:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { BrainyAugmentation } from '../types'
|
||||||
|
|
||||||
|
export class MyAugmentation extends BrainyAugmentation {
|
||||||
|
name = 'MyAugmentation'
|
||||||
|
|
||||||
|
async onInit(brain: Brainy): Promise<void> {
|
||||||
|
// Initialize augmentation
|
||||||
|
}
|
||||||
|
|
||||||
|
async onAdd(item: any, brain: Brainy): Promise<any> {
|
||||||
|
// Process before adding
|
||||||
|
return item
|
||||||
|
}
|
||||||
|
|
||||||
|
async onSearch(query: any, results: any[], brain: Brainy): Promise<any[]> {
|
||||||
|
// Process search results
|
||||||
|
return results
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Performance Considerations
|
||||||
|
|
||||||
|
- Use batch operations where possible
|
||||||
|
- Implement caching strategically
|
||||||
|
- Consider memory usage
|
||||||
|
- Profile performance impacts
|
||||||
|
- Add benchmarks for critical paths
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
### API Documentation
|
||||||
|
|
||||||
|
Use JSDoc for all public APIs:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
/**
|
||||||
|
* Searches for similar items using vector similarity
|
||||||
|
* @param query - Search query (text or vector)
|
||||||
|
* @param options - Search options
|
||||||
|
* @returns Array of search results with scores
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* const results = await brain.search("machine learning", { limit: 10 })
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
async search(query: string | Vector, options?: SearchOptions): Promise<SearchResult[]> {
|
||||||
|
// Implementation
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Examples
|
||||||
|
|
||||||
|
Add examples for new features:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// examples/feature-name.ts
|
||||||
|
import { Brainy } from 'brainy'
|
||||||
|
|
||||||
|
async function exampleUsage() {
|
||||||
|
const brain = new Brainy()
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
// Show feature usage
|
||||||
|
// Include comments explaining what's happening
|
||||||
|
// Handle errors appropriately
|
||||||
|
}
|
||||||
|
|
||||||
|
exampleUsage().catch(console.error)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Release Process
|
||||||
|
|
||||||
|
1. **Version bump**: Follow semantic versioning
|
||||||
|
2. **Update CHANGELOG**: Document all changes
|
||||||
|
3. **Run tests**: Ensure all tests pass
|
||||||
|
4. **Build**: Generate distribution files
|
||||||
|
5. **Tag**: Create git tag for version
|
||||||
|
6. **Publish**: Release to npm
|
||||||
|
|
||||||
|
## Getting Help
|
||||||
|
|
||||||
|
- **Discord**: Join our community
|
||||||
|
- **Issues**: Ask questions on GitHub
|
||||||
|
- **Discussions**: Share ideas and get feedback
|
||||||
|
|
||||||
|
## Recognition
|
||||||
|
|
||||||
|
Contributors will be recognized in:
|
||||||
|
- CHANGELOG.md for their contributions
|
||||||
|
- README.md contributors section
|
||||||
|
- GitHub contributors page
|
||||||
|
|
||||||
|
Thank you for contributing to Brainy! 🧠
|
||||||
72
Dockerfile
Normal file
72
Dockerfile
Normal file
|
|
@ -0,0 +1,72 @@
|
||||||
|
# Multi-stage Dockerfile for Brainy
|
||||||
|
# Optimized for production deployment with minimal image size
|
||||||
|
|
||||||
|
# Stage 1: Build stage
|
||||||
|
FROM node:22-alpine AS builder
|
||||||
|
|
||||||
|
# Install build dependencies
|
||||||
|
RUN apk add --no-cache python3 make g++
|
||||||
|
|
||||||
|
# Set working directory
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Copy package files
|
||||||
|
COPY package*.json ./
|
||||||
|
|
||||||
|
# Install all dependencies (including dev dependencies for building)
|
||||||
|
RUN npm ci
|
||||||
|
|
||||||
|
# Copy source code
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
# Build the TypeScript code
|
||||||
|
RUN npm run build
|
||||||
|
|
||||||
|
# Remove dev dependencies and only keep production ones
|
||||||
|
RUN npm prune --production
|
||||||
|
|
||||||
|
# Stage 2: Production stage
|
||||||
|
FROM node:22-alpine
|
||||||
|
|
||||||
|
# Install production dependencies only
|
||||||
|
RUN apk add --no-cache tini
|
||||||
|
|
||||||
|
# Create non-root user for security
|
||||||
|
RUN addgroup -g 1001 -S nodejs && \
|
||||||
|
adduser -S nodejs -u 1001
|
||||||
|
|
||||||
|
# Set working directory
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Copy package files
|
||||||
|
COPY package*.json ./
|
||||||
|
|
||||||
|
# Copy built application from builder stage
|
||||||
|
COPY --from=builder --chown=nodejs:nodejs /app/node_modules ./node_modules
|
||||||
|
COPY --from=builder --chown=nodejs:nodejs /app/dist ./dist
|
||||||
|
|
||||||
|
# Copy necessary static files
|
||||||
|
COPY --chown=nodejs:nodejs README.md LICENSE ./
|
||||||
|
|
||||||
|
# Create data directory for file-based storage
|
||||||
|
RUN mkdir -p /app/data && chown -R nodejs:nodejs /app/data
|
||||||
|
|
||||||
|
# Switch to non-root user
|
||||||
|
USER nodejs
|
||||||
|
|
||||||
|
# Expose default port (can be overridden)
|
||||||
|
EXPOSE 3000
|
||||||
|
|
||||||
|
# Set environment variables for production
|
||||||
|
ENV NODE_ENV=production
|
||||||
|
ENV BRAINY_STORAGE_PATH=/app/data
|
||||||
|
|
||||||
|
# Health check endpoint
|
||||||
|
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
|
||||||
|
CMD node -e "require('http').get('http://localhost:3000/health', (r) => r.statusCode === 200 ? process.exit(0) : process.exit(1))"
|
||||||
|
|
||||||
|
# Use tini to handle signals properly
|
||||||
|
ENTRYPOINT ["/sbin/tini", "--"]
|
||||||
|
|
||||||
|
# Default command (can be overridden)
|
||||||
|
CMD ["node", "dist/index.js"]
|
||||||
|
|
@ -1,48 +0,0 @@
|
||||||
# Test Suite Fix Strategy
|
|
||||||
|
|
||||||
## Current Status
|
|
||||||
- **34 test files** remaining after cleanup
|
|
||||||
- **83 passing, 2 failing** in critical tests
|
|
||||||
- Timeouts in some test files
|
|
||||||
|
|
||||||
## Issues Identified
|
|
||||||
|
|
||||||
### 1. Test Timeouts
|
|
||||||
**Cause**: Tests waiting for real network/file operations
|
|
||||||
**Fix**:
|
|
||||||
- Use vi.useFakeTimers() consistently
|
|
||||||
- Mock all external dependencies
|
|
||||||
- Set reasonable test timeouts
|
|
||||||
|
|
||||||
### 2. Wrong Expectations
|
|
||||||
**Cause**: Tests expect old behavior (hard delete by default)
|
|
||||||
**Fix**: Update expectations to match soft delete default
|
|
||||||
|
|
||||||
### 3. Missing Methods
|
|
||||||
**Cause**: Tests calling destroy() that doesn't exist
|
|
||||||
**Fix**: Remove cleanup calls or implement disposal pattern
|
|
||||||
|
|
||||||
## Action Plan
|
|
||||||
|
|
||||||
### Phase 1: Fix Critical Tests (DONE)
|
|
||||||
✅ core.test.ts - PASSING
|
|
||||||
✅ unified-api.test.ts - PASSING
|
|
||||||
✅ cli.test.ts - PASSING
|
|
||||||
✅ edge-cases.test.ts - PASSING
|
|
||||||
|
|
||||||
### Phase 2: Fix Storage Tests
|
|
||||||
- storage-adapter-coverage.test.ts - Update delete expectations
|
|
||||||
- regression.test.ts - Remove destroy() calls
|
|
||||||
|
|
||||||
### Phase 3: Skip/Remove Slow Tests
|
|
||||||
- metadata-performance.test.ts - Skip or reduce dataset size
|
|
||||||
- s3-comprehensive.test.ts - Ensure mocks are working
|
|
||||||
|
|
||||||
### Phase 4: Final Validation
|
|
||||||
- Run all tests with --bail to stop on first failure
|
|
||||||
- Ensure no test takes > 30 seconds
|
|
||||||
|
|
||||||
## Expected Outcome
|
|
||||||
- All tests pass within 60 seconds total
|
|
||||||
- ~400 meaningful tests (after removing redundant)
|
|
||||||
- 100% pass rate
|
|
||||||
2
LICENSE
2
LICENSE
|
|
@ -1,6 +1,6 @@
|
||||||
MIT License
|
MIT License
|
||||||
|
|
||||||
Copyright (c) 2023 Soulcraft Research
|
Copyright (c) 2024 Brainy Data Contributors
|
||||||
|
|
||||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
of this software and associated documentation files (the "Software"), to deal
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
|
|
||||||
37
MIGRATION.md
37
MIGRATION.md
|
|
@ -1,37 +0,0 @@
|
||||||
# Migration Guide: Brainy 0.x → 1.0
|
|
||||||
|
|
||||||
## Breaking Changes
|
|
||||||
|
|
||||||
### 1. Soft Delete by Default
|
|
||||||
- `delete()` now performs soft delete by default
|
|
||||||
- Items are marked with `deleted: true` metadata instead of being removed
|
|
||||||
- To perform hard delete: `delete(id, { soft: false })`
|
|
||||||
|
|
||||||
### 2. Enhanced Search Filtering
|
|
||||||
- Soft deleted items are automatically excluded from search results
|
|
||||||
- No code changes needed - this happens automatically
|
|
||||||
|
|
||||||
### 3. Configuration Storage
|
|
||||||
- New encrypted configuration storage API
|
|
||||||
- Use `setConfig()` and `getConfig()` for secure configuration
|
|
||||||
|
|
||||||
## New Features
|
|
||||||
|
|
||||||
### 1. Unified API
|
|
||||||
- 5 core methods for all operations
|
|
||||||
- `add()`, `search()`, `import()`, `addNoun()`, `addVerb()`
|
|
||||||
|
|
||||||
### 2. Encryption Support
|
|
||||||
- Built-in encryption for sensitive data
|
|
||||||
- `encryptData()` and `decryptData()` methods
|
|
||||||
|
|
||||||
### 3. Augmentation System
|
|
||||||
- Professional augmentation catalog
|
|
||||||
- Registry integration at registry.soulcraft.com
|
|
||||||
|
|
||||||
## Upgrade Steps
|
|
||||||
|
|
||||||
1. Update package: `npm install @soulcraft/brainy@latest`
|
|
||||||
2. Review delete operations if expecting hard delete
|
|
||||||
3. Update tests to expect soft delete behavior
|
|
||||||
4. Leverage new encryption features for sensitive data
|
|
||||||
|
|
@ -1,124 +0,0 @@
|
||||||
# Brainy Model Management Strategy
|
|
||||||
|
|
||||||
## Critical Requirement
|
|
||||||
The Xenova/all-MiniLM-L6-v2 transformer model (87MB) is **essential** for Brainy operations. It must be available and never change to ensure consistent embeddings across all deployments.
|
|
||||||
|
|
||||||
## Current Approach: Hybrid Model Management
|
|
||||||
|
|
||||||
### 1. **NPM Package** (Default)
|
|
||||||
- Models are NOT included in the NPM package (keeps it small at 643KB)
|
|
||||||
- Models download automatically on first use
|
|
||||||
- Cached locally after first download
|
|
||||||
- Perfect for: Development, most deployments
|
|
||||||
|
|
||||||
### 2. **Docker/CI** (Production)
|
|
||||||
```dockerfile
|
|
||||||
# Download models during build when internet is available
|
|
||||||
RUN npm install @soulcraft/brainy
|
|
||||||
RUN npm run download-models # Downloads to ./models/
|
|
||||||
# Models are now part of the container image
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. **CDN Fallback** (Future)
|
|
||||||
- Host models on cdn.soulcraft.com
|
|
||||||
- Provides reliable fallback if Hugging Face is down
|
|
||||||
- Ensures we control model availability
|
|
||||||
|
|
||||||
## File Structure
|
|
||||||
```
|
|
||||||
models/
|
|
||||||
├── Xenova/
|
|
||||||
│ └── all-MiniLM-L6-v2/
|
|
||||||
│ ├── config.json (650 bytes)
|
|
||||||
│ ├── tokenizer.json (695 KB)
|
|
||||||
│ ├── tokenizer_config.json (366 bytes)
|
|
||||||
│ └── onnx/
|
|
||||||
│ └── model.onnx (87 MB)
|
|
||||||
└── .brainy-models-bundled (marker file)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Why NOT in Git Repository
|
|
||||||
|
|
||||||
1. **Size**: 87MB is too large for comfortable Git operations
|
|
||||||
2. **Git LFS Complexity**: Requires additional setup, costs money
|
|
||||||
3. **Flexibility**: Different deployment strategies need different approaches
|
|
||||||
4. **NPM Package Size**: Would bloat package from 643KB to 88MB+
|
|
||||||
|
|
||||||
## Deployment Strategies
|
|
||||||
|
|
||||||
### A. Standard Web App
|
|
||||||
```bash
|
|
||||||
npm install @soulcraft/brainy
|
|
||||||
# Models download on first use, cached forever
|
|
||||||
```
|
|
||||||
|
|
||||||
### B. Serverless/Lambda
|
|
||||||
```javascript
|
|
||||||
// Pre-download in Lambda layer
|
|
||||||
const modelLayer = '/opt/models'
|
|
||||||
process.env.TRANSFORMERS_CACHE = modelLayer
|
|
||||||
```
|
|
||||||
|
|
||||||
### C. Kubernetes
|
|
||||||
```yaml
|
|
||||||
# Init container downloads models
|
|
||||||
initContainers:
|
|
||||||
- name: download-models
|
|
||||||
command: ['npm', 'run', 'download-models']
|
|
||||||
volumeMounts:
|
|
||||||
- name: models
|
|
||||||
mountPath: /app/models
|
|
||||||
```
|
|
||||||
|
|
||||||
### D. Offline Environment
|
|
||||||
```bash
|
|
||||||
# Download during build/packaging
|
|
||||||
npm run download-models
|
|
||||||
tar -czf models.tar.gz models/
|
|
||||||
# Deploy tar file with application
|
|
||||||
```
|
|
||||||
|
|
||||||
## Model Integrity
|
|
||||||
|
|
||||||
The model MUST remain unchanged. We ensure this by:
|
|
||||||
|
|
||||||
1. **Pinned Version**: Always use Xenova/all-MiniLM-L6-v2
|
|
||||||
2. **Hash Verification**: Check SHA256 of model.onnx
|
|
||||||
3. **Size Verification**: Ensure model.onnx is exactly 90,555,481 bytes
|
|
||||||
4. **Local Cache**: Once downloaded, never re-download
|
|
||||||
|
|
||||||
## Implementation in Code
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// src/embeddings/index.ts
|
|
||||||
import { env } from '@huggingface/transformers'
|
|
||||||
|
|
||||||
// Configure model location (in order of preference)
|
|
||||||
env.localModelPath = [
|
|
||||||
'./models', // Local bundled models
|
|
||||||
'/opt/models', // Lambda layer
|
|
||||||
process.env.MODELS_PATH, // Custom path
|
|
||||||
env.cacheDir // Default cache
|
|
||||||
].find(p => p && fs.existsSync(path.join(p, 'Xenova')))
|
|
||||||
|
|
||||||
// Disable remote models in production
|
|
||||||
if (process.env.NODE_ENV === 'production') {
|
|
||||||
env.allowRemoteModels = false
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Verification Script
|
|
||||||
|
|
||||||
Run `npm run verify-models` to check:
|
|
||||||
- ✅ All required model files exist
|
|
||||||
- ✅ File sizes match expected
|
|
||||||
- ✅ SHA256 hashes match (optional)
|
|
||||||
- ✅ Model can be loaded successfully
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
- **Development**: Models auto-download on first use
|
|
||||||
- **Production**: Models pre-downloaded during build
|
|
||||||
- **Distribution**: NPM package stays small (643KB)
|
|
||||||
- **Reliability**: Models always available, never change
|
|
||||||
- **Flexibility**: Multiple deployment strategies supported
|
|
||||||
|
|
@ -1,56 +0,0 @@
|
||||||
# Offline Models
|
|
||||||
|
|
||||||
Brainy uses Transformers.js with ONNX Runtime for **true offline operation** - no more TensorFlow.js dependency hell!
|
|
||||||
|
|
||||||
## How it works
|
|
||||||
|
|
||||||
Brainy automatically figures out the best approach:
|
|
||||||
|
|
||||||
1. **First use**: Downloads models once (~87 MB) to local cache
|
|
||||||
2. **Subsequent use**: Loads from cache (completely offline, zero network calls)
|
|
||||||
3. **Smart detection**: Automatically finds models in cache, bundled, or downloads as needed
|
|
||||||
|
|
||||||
## Standard usage
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm install @soulcraft/brainy
|
|
||||||
# Use immediately - models download automatically on first use
|
|
||||||
```
|
|
||||||
|
|
||||||
## Docker with production egress restrictions
|
|
||||||
|
|
||||||
For environments where production has no internet but build does:
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
FROM node:24-slim
|
|
||||||
WORKDIR /app
|
|
||||||
COPY package*.json ./
|
|
||||||
RUN npm install @soulcraft/brainy
|
|
||||||
RUN npm run download-models # Download during build (when internet available)
|
|
||||||
COPY . .
|
|
||||||
# Production container now works completely offline
|
|
||||||
```
|
|
||||||
|
|
||||||
## Development with immediate offline
|
|
||||||
|
|
||||||
If you want models available immediately for development:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm install @soulcraft/brainy
|
|
||||||
npm run download-models # Optional: download now instead of on first use
|
|
||||||
```
|
|
||||||
|
|
||||||
## Key benefits vs TensorFlow.js
|
|
||||||
|
|
||||||
- ✅ **95% smaller package** - 643 kB vs 12.5 MB
|
|
||||||
- ✅ **84% smaller models** - 87 MB vs 525 MB
|
|
||||||
- ✅ **True offline** - Zero network calls after initial download
|
|
||||||
- ✅ **No dependency issues** - 5 deps vs 47+, no more --legacy-peer-deps
|
|
||||||
- ✅ **Better performance** - ONNX Runtime beats TensorFlow.js
|
|
||||||
- ✅ **Same API** - Drop-in replacement
|
|
||||||
|
|
||||||
## Philosophy
|
|
||||||
|
|
||||||
**Install and use. Brainy handles the rest.**
|
|
||||||
|
|
||||||
No configuration files, no environment variables, no complex setup. Brainy detects your environment and does the right thing automatically.
|
|
||||||
874
README.md
874
README.md
|
|
@ -1,779 +1,217 @@
|
||||||
<div align="center">
|
<p align="center">
|
||||||
|
<img src="https://raw.githubusercontent.com/soulcraftlabs/brainy/main/brainy.png" alt="Brainy" width="180">
|
||||||
|
</p>
|
||||||
|
|
||||||

|
<h1 align="center">Brainy</h1>
|
||||||
|
|
||||||
[](https://badge.fury.io/js/%40soulcraft%2Fbrainy)
|
<p align="center">
|
||||||
[](https://github.com/soulcraftlabs/brainy/releases/tag/v1.0.0)
|
<b>Three database paradigms. One API. Zero configuration.</b><br>
|
||||||
[](https://opensource.org/licenses/MIT)
|
The in-process knowledge database for TypeScript — vector search, graph traversal,<br>
|
||||||
[](https://soulcraft.com)
|
and metadata filtering unified in a single query.
|
||||||
[](https://soulcraft.com)
|
</p>
|
||||||
[](https://nodejs.org/)
|
|
||||||
[](https://www.typescriptlang.org/)
|
|
||||||
|
|
||||||
# The World's First Multi-Dimensional AI Database™
|
<p align="center">
|
||||||
|
<a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/v/@soulcraft/brainy.svg" alt="npm version"></a>
|
||||||
|
<a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/dm/@soulcraft/brainy.svg" alt="npm downloads"></a>
|
||||||
|
<a href="https://github.com/soulcraftlabs/brainy/actions/workflows/ci.yml"><img src="https://github.com/soulcraftlabs/brainy/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
|
||||||
|
<a href="https://soulcraft.com/docs"><img src="https://img.shields.io/badge/docs-soulcraft.com-blue.svg" alt="Documentation"></a>
|
||||||
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License"></a>
|
||||||
|
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/%3C%2F%3E-TypeScript-%230074c1.svg" alt="TypeScript"></a>
|
||||||
|
</p>
|
||||||
|
|
||||||
*Vector similarity • Graph relationships • Metadata facets • Neural understanding*
|
<p align="center">
|
||||||
|
<a href="#quick-start">Quick start</a> ·
|
||||||
## 🎉 **From Browser to Billions - Same Simple Code!**
|
<a href="#one-query-three-engines">One query</a> ·
|
||||||
|
<a href="#feature-tour">Features</a> ·
|
||||||
**Start in seconds. Scale to millions. Never rewrite.**
|
<a href="#from-laptop-to-hundreds-of-millions">Scale with Cor</a> ·
|
||||||
|
<a href="#documentation">Docs</a>
|
||||||
```javascript
|
</p>
|
||||||
// This ONE LINE scales from browser playground to enterprise:
|
|
||||||
const brain = new BrainyData()
|
|
||||||
|
|
||||||
// That's it. Seriously. 🚀
|
|
||||||
```
|
|
||||||
|
|
||||||
✨ **No config files** • **No complexity** • **No limits**
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 💖 **Support Brainy's Development**
|
Built because we were tired of stitching a vector store to a graph database to a document store — and spending weeks on plumbing before writing a line of business logic. Brainy indexes every fact **three ways at once** and lets one call query them together:
|
||||||
|
|
||||||
<div align="center">
|
| You write | Brainy indexes it as | You query it with |
|
||||||
|
|---|---|---|
|
||||||
|
| `data: 'Ada wrote the first program'` | a **384-dim vector** (local embedding — no API key) | `find({ query: 'computing pioneers' })` |
|
||||||
|
| `metadata: { field: 'CS', year: 1843 }` | **structured fields** (O(1) exact, O(log n) range) | `find({ where: { year: { lessThan: 1900 } } })` |
|
||||||
|
| `relate({ from: ada, to: babbage })` | a **typed, directed graph edge** | `find({ connected: { to: babbage, depth: 2 } })` |
|
||||||
|
|
||||||
**Brainy is 100% open source and free forever!** Help us keep it that way:
|
It runs **inside your process** — no server, no Docker, nothing to operate — and persists to plain files you can snapshot with a hard link.
|
||||||
|
|
||||||
[](https://github.com/soulcraftlabs/brainy)
|
**New here?** → **[What is Brainy? — plain-language overview, no jargon](docs/eli5.md)**
|
||||||
[](https://soulcraft.com)
|
|
||||||
[](https://github.com/soulcraftlabs/brainy)
|
|
||||||
|
|
||||||
**Every sponsorship helps us:** Build more features • Fix bugs faster • Keep Brainy free
|
## Quick start
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎉 **NEW: Brainy 1.0 - The Unified API**
|
|
||||||
|
|
||||||
**The Great Cleanup is complete!** Brainy 1.0 introduces the **unified API** - ONE way to do everything with just **9 core methods**:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Install Brainy 1.0
|
bun add @soulcraft/brainy # Bun ≥ 1.1 — recommended
|
||||||
npm install @soulcraft/brainy
|
npm install @soulcraft/brainy # Node.js ≥ 22
|
||||||
```
|
```
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import { BrainyData, NounType, VerbType } from '@soulcraft/brainy'
|
import { Brainy, NounType, VerbType } from '@soulcraft/brainy'
|
||||||
|
|
||||||
const brain = new BrainyData()
|
const brain = new Brainy() // in-memory; one line swaps to disk
|
||||||
await brain.init()
|
await brain.init()
|
||||||
|
|
||||||
// 🎯 THE 9 UNIFIED METHODS - One way to do everything!
|
// Text auto-embeds locally; metadata auto-indexes
|
||||||
await brain.add("Smart data") // 1. Smart addition
|
const react = await brain.add({
|
||||||
await brain.search("query", 10) // 2. Unified search
|
data: 'React is a JavaScript library for building user interfaces',
|
||||||
await brain.import(["data1", "data2"]) // 3. Bulk import
|
type: NounType.Concept,
|
||||||
await brain.addNoun("John", NounType.Person) // 4. Typed entities
|
subtype: 'library',
|
||||||
await brain.addVerb(id1, id2, VerbType.Knows) // 5. Relationships
|
metadata: { category: 'frontend', year: 2013 }
|
||||||
await brain.update(id, "new data") // 6. Smart updates
|
|
||||||
await brain.delete(id) // 7. Soft delete
|
|
||||||
await brain.export({ format: 'json' }) // 8. Export data
|
|
||||||
brain.augment(myAugmentation) // 9. Extend infinitely! ♾️
|
|
||||||
|
|
||||||
// NEW: Type-safe augmentation management via brain.augmentations
|
|
||||||
brain.augmentations.list() // See all augmentations
|
|
||||||
brain.augmentations.enable(name) // Enable/disable dynamically
|
|
||||||
```
|
|
||||||
|
|
||||||
### ✨ **What's New in 1.5:**
|
|
||||||
- **🔥 40+ methods consolidated** → 9 unified methods
|
|
||||||
- **♾️ The 9th method** - `augment()` lets you extend Brainy infinitely!
|
|
||||||
- **🧠 Smart by default** - `add()` auto-detects and processes intelligently
|
|
||||||
- **🔐 Universal encryption** - Built-in encryption for sensitive data
|
|
||||||
- **🐳 Container ready** - Model preloading for production deployments
|
|
||||||
- **📦 16% smaller package** despite major new features
|
|
||||||
- **🔄 Soft delete default** - Better performance, no reindexing needed
|
|
||||||
- **🎯 NEW: Intelligent Verb Scoring** - Relationships automatically scored by AI
|
|
||||||
- **💾 NEW: Write-Ahead Log (WAL)** - Zero data loss guarantee, always on
|
|
||||||
- **⚡ NEW: Request Deduplication** - 3x performance for concurrent requests
|
|
||||||
|
|
||||||
**Breaking Changes:** See [MIGRATION.md](MIGRATION.md) for complete upgrade guide.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✅ 100% Free & Open Source
|
|
||||||
|
|
||||||
**Brainy is completely free. No license keys. No limits. No catch.**
|
|
||||||
|
|
||||||
Every feature you see here works without any payment or registration:
|
|
||||||
- ✓ Full vector database
|
|
||||||
- ✓ Graph relationships
|
|
||||||
- ✓ Semantic search
|
|
||||||
- ✓ All storage adapters
|
|
||||||
- ✓ Complete API
|
|
||||||
- ✓ Forever free
|
|
||||||
|
|
||||||
> 🌩️ **Brain Cloud** is our optional cloud service that helps sustain Brainy's development. Currently in early access at [soulcraft.com](https://soulcraft.com).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 💫 Why Brainy? The Problem We Solve
|
|
||||||
|
|
||||||
### ❌ **The Old Way: Database Frankenstein**
|
|
||||||
```
|
|
||||||
Pinecone ($750/mo) + Neo4j ($500/mo) + Elasticsearch ($300/mo) +
|
|
||||||
Sync nightmares + 3 different APIs + Vendor lock-in = 😱💸
|
|
||||||
```
|
|
||||||
|
|
||||||
### ✅ **The Brainy Way: One Brain, All Dimensions**
|
|
||||||
```
|
|
||||||
Vector + Graph + Search + AI = Brainy (Free & Open Source) = 🧠✨
|
|
||||||
```
|
|
||||||
|
|
||||||
**Your data gets superpowers. Your wallet stays happy.**
|
|
||||||
|
|
||||||
### 🧠 **Why Developers Love Brainy 1.0**
|
|
||||||
|
|
||||||
#### **⚡ One API to Rule Them All**
|
|
||||||
```javascript
|
|
||||||
// Before: Learning 10+ different database APIs
|
|
||||||
pinecone.upsert(), neo4j.run(), elasticsearch.search()
|
|
||||||
supabase.insert(), mongodb.find(), redis.set()
|
|
||||||
|
|
||||||
// After: 9 methods handle EVERYTHING
|
|
||||||
brain.add(), brain.search(), brain.import()
|
|
||||||
brain.addNoun(), brain.addVerb(), brain.update()
|
|
||||||
brain.delete(), brain.export(), brain.augment()
|
|
||||||
|
|
||||||
// Why 9? The 9th method (augment) gives you methods 10 → ∞!
|
|
||||||
```
|
|
||||||
|
|
||||||
#### **🤯 Mind-Blowing Features Out of the Box**
|
|
||||||
- **Smart by Default**: `add()` automatically understands your data
|
|
||||||
- **Graph + Vector**: Relationships AND semantic similarity in one query
|
|
||||||
- **Zero Config**: Works instantly, optimizes itself
|
|
||||||
- **Universal Encryption**: Secure everything with one flag
|
|
||||||
- **Perfect Memory**: Nothing ever gets lost or forgotten
|
|
||||||
|
|
||||||
#### **💰 Cost Comparison**
|
|
||||||
| Traditional Stack | Monthly Cost | Brainy 1.0 |
|
|
||||||
|------------------|--------------|-------------|
|
|
||||||
| Pinecone + Neo4j + Search | $1,500+ | **$0** |
|
|
||||||
| 3 different APIs to learn | Weeks | **Minutes** |
|
|
||||||
| Sync complexity | High | **None** |
|
|
||||||
| Vendor lock-in | Yes | **MIT License** |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 What Can You Build?
|
|
||||||
|
|
||||||
### 💬 **AI Chat Apps** - That Actually Remember
|
|
||||||
```javascript
|
|
||||||
// Your users' conversations persist across sessions
|
|
||||||
const brain = new BrainyData()
|
|
||||||
await brain.add("User prefers dark mode")
|
|
||||||
await brain.add("User is learning Spanish")
|
|
||||||
|
|
||||||
// Later sessions remember everything
|
|
||||||
const context = await brain.search("user preferences")
|
|
||||||
// AI knows: dark mode + Spanish learning preference
|
|
||||||
```
|
|
||||||
|
|
||||||
### 🤖 **Smart Assistants** - With Real Knowledge Graphs
|
|
||||||
```javascript
|
|
||||||
// Build assistants that understand relationships (NEW 1.0 API!)
|
|
||||||
import { BrainyData, NounType, VerbType } from '@soulcraft/brainy'
|
|
||||||
|
|
||||||
const brain = new BrainyData()
|
|
||||||
await brain.init()
|
|
||||||
|
|
||||||
// Create typed entities
|
|
||||||
const sarahId = await brain.addNoun("Sarah Thompson", NounType.Person)
|
|
||||||
const johnId = await brain.addNoun("John Davis", NounType.Person)
|
|
||||||
const projectId = await brain.addNoun("Project Apollo", NounType.Project)
|
|
||||||
|
|
||||||
// Create relationships with metadata
|
|
||||||
await brain.addVerb(sarahId, johnId, VerbType.ReportsTo, {
|
|
||||||
role: "Design Manager",
|
|
||||||
startDate: "2024-01-15"
|
|
||||||
})
|
|
||||||
await brain.addVerb(sarahId, projectId, VerbType.WorksWith, {
|
|
||||||
responsibility: "Lead Designer",
|
|
||||||
allocation: "75%"
|
|
||||||
})
|
})
|
||||||
|
|
||||||
// Query complex relationships with graph traversal
|
const next = await brain.add({
|
||||||
const sarahData = await brain.getNounWithVerbs(sarahId)
|
data: 'Next.js is a React framework with server-side rendering',
|
||||||
// Returns: complete graph view with all relationships and metadata
|
type: NounType.Concept,
|
||||||
```
|
subtype: 'framework',
|
||||||
|
metadata: { category: 'frontend', year: 2016 }
|
||||||
### 📊 **RAG Applications** - Without the Complexity
|
|
||||||
```javascript
|
|
||||||
// Retrieval-Augmented Generation in 3 lines
|
|
||||||
await brain.add(companyDocs) // Add your knowledge base
|
|
||||||
const relevant = await brain.search(userQuery, 10) // Find relevant context
|
|
||||||
const answer = await llm.generate(relevant + userQuery) // Generate with context
|
|
||||||
```
|
|
||||||
|
|
||||||
### 🔍 **Semantic Search** - That Just Works
|
|
||||||
```javascript
|
|
||||||
// No embeddings API needed - it's built in!
|
|
||||||
await brain.add("The iPhone 15 Pro has a titanium design")
|
|
||||||
await brain.add("Samsung Galaxy S24 features AI photography")
|
|
||||||
|
|
||||||
const results = await brain.search("smartphones with metal build")
|
|
||||||
// Returns: iPhone (titanium matches "metal build" semantically)
|
|
||||||
```
|
|
||||||
|
|
||||||
### 🎯 **Recommendation Engines** - With Graph Intelligence
|
|
||||||
```javascript
|
|
||||||
// Netflix-style recommendations with 1.0 unified API
|
|
||||||
import { BrainyData, NounType, VerbType } from '@soulcraft/brainy'
|
|
||||||
|
|
||||||
const brain = new BrainyData()
|
|
||||||
await brain.init()
|
|
||||||
|
|
||||||
// Create entities and relationships
|
|
||||||
const userId = await brain.addNoun("User123", NounType.Person)
|
|
||||||
const movieId = await brain.addNoun("Inception", NounType.Content)
|
|
||||||
|
|
||||||
// Track user behavior with metadata
|
|
||||||
await brain.addVerb(userId, movieId, VerbType.InteractedWith, {
|
|
||||||
action: "watched",
|
|
||||||
rating: 5,
|
|
||||||
timestamp: new Date(),
|
|
||||||
genre: "sci-fi"
|
|
||||||
})
|
})
|
||||||
|
|
||||||
// Get intelligent recommendations based on relationships
|
await brain.relate({ from: next, to: react, type: VerbType.DependsOn, subtype: 'runtime' })
|
||||||
const recommendations = await brain.getNounWithVerbs(userId, {
|
|
||||||
verbTypes: [VerbType.InteractedWith],
|
|
||||||
depth: 2
|
|
||||||
})
|
|
||||||
// Returns: Similar movies based on rating patterns and genre preferences
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### 🤖 **Multi-Agent AI Systems** - With Shared Memory
|
## One query, three engines
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
// Multiple AI agents sharing the same brain
|
const results = await brain.find({
|
||||||
const sharedBrain = new BrainyData({ instance: 'multi-agent-brain' })
|
query: 'modern frontend frameworks', // vector — what it means
|
||||||
await sharedBrain.init()
|
where: { year: { greaterThan: 2015 } }, // metadata — what it is
|
||||||
|
connected: { to: react, depth: 2 } // graph — what it touches
|
||||||
// Sales Agent adds customer intelligence
|
|
||||||
const customerId = await sharedBrain.addNoun("Acme Corp", NounType.Organization)
|
|
||||||
await sharedBrain.addVerb(customerId, "business-plan", VerbType.InterestedIn, {
|
|
||||||
priority: "high",
|
|
||||||
timeline: "Q2 2025"
|
|
||||||
})
|
})
|
||||||
|
|
||||||
// Support Agent instantly sees the context
|
|
||||||
const customerData = await sharedBrain.getNounWithVerbs(customerId)
|
|
||||||
// Support knows: customer interested in business plan
|
|
||||||
|
|
||||||
// Marketing Agent learns from both
|
|
||||||
const insights = await sharedBrain.search("business customers Q2", 10)
|
|
||||||
// Marketing can create targeted campaigns for similar prospects
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### 🏥 **Customer Support Bots** - With Perfect Memory
|
Every clause is optional; any combination composes. Under the hood Brainy plans the query across an HNSW vector index, a roaring-bitmap field index, and an adjacency graph index — and re-validates every result against your predicate before returning it, so a corrupt index can never hand you a wrong answer.
|
||||||
|
|
||||||
|
## Feature tour
|
||||||
|
|
||||||
|
### The database is a value
|
||||||
|
|
||||||
|
Pin it, rewind it, fork it. Snapshot isolation without a server.
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
// Support bot that remembers every interaction
|
const db = brain.now() // pin current state — O(1)
|
||||||
const customerId = await brain.addNoun("Customer_456", NounType.Person)
|
|
||||||
|
|
||||||
// Track support history with rich metadata
|
await brain.transact([ // atomic all-or-nothing, CAS-guarded
|
||||||
await brain.addVerb(customerId, "password-reset", VerbType.RequestedHelp, {
|
{ op: 'update', id: order, metadata: { status: 'paid' } },
|
||||||
issue: "Password reset",
|
{ op: 'relate', from: invoice, to: order, type: VerbType.References, subtype: 'billing' }
|
||||||
resolved: true,
|
], { ifAtGeneration: db.generation })
|
||||||
date: "2025-01-10",
|
|
||||||
satisfaction: 5,
|
|
||||||
agent: "Sarah"
|
|
||||||
})
|
|
||||||
|
|
||||||
// Next conversation - bot instantly knows history
|
await db.get(order) // still 'pending' — pinned forever
|
||||||
const history = await brain.getNounWithVerbs(customerId)
|
await brain.get(order) // 'paid' — live
|
||||||
// Bot: "I see you had a password issue last week. Everything working smoothly now?"
|
|
||||||
|
|
||||||
// Proactive insights
|
const lastWeek = await brain.asOf(Date.now() - 7 * 86_400_000) // full query surface, past state
|
||||||
const commonIssues = await brain.search("password reset common issues", 5)
|
const whatIf = await db.with([{ op: 'remove', id: order }]) // speculative — never touches disk
|
||||||
// Bot offers preventive tips before problems occur
|
await brain.now().persist('/backups/today') // instant hard-link snapshot
|
||||||
```
|
```
|
||||||
|
|
||||||
### ❌ **The Old Way: Database Frankenstein**
|
**[Consistency model](docs/concepts/consistency-model.md)** · **[Snapshots & time travel](docs/guides/snapshots-and-time-travel.md)**
|
||||||
```
|
|
||||||
Pinecone ($750/mo) + Neo4j ($500/mo) + Elasticsearch ($300/mo) +
|
### Local embeddings — no API keys
|
||||||
Sync nightmares + 3 different APIs + Vendor lock-in = 😱💸
|
|
||||||
|
Strings embed on-device with a bundled MiniLM model (WASM). Semantic search works offline, in CI, and on air-gapped machines, at zero cost per call. Hybrid keyword + semantic ranking is the default:
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
await brain.find({ query: 'David Smith' }) // auto: text + semantic
|
||||||
|
await brain.find({ query: 'AI concepts', searchMode: 'semantic' }) // semantic only
|
||||||
```
|
```
|
||||||
|
|
||||||
### ✅ **The Brainy Way: One Brain, All Dimensions**
|
### A typed graph, not a bag of edges
|
||||||
```
|
|
||||||
Vector + Graph + Search + AI = Brainy (Free & Open Source) = 🧠✨
|
42 entity types × 127 relationship types form a shared vocabulary for any domain — healthcare (`Patient → diagnoses → Condition`), finance (`Account → transfers → Transaction`), yours. Your own taxonomy layers on with `subtype`, enforced at write time:
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
await brain.add({ data: 'Avery Brooks', type: NounType.Person, subtype: 'employee' })
|
||||||
|
|
||||||
|
brain.counts.bySubtype(NounType.Person) // O(1) — { employee: 12, customer: 847 }
|
||||||
|
brain.requireSubtype(NounType.Person, { values: ['employee', 'customer'], required: true })
|
||||||
```
|
```
|
||||||
|
|
||||||
**Your data gets superpowers. Your wallet stays happy.**
|
**[Type system](docs/architecture/noun-verb-taxonomy.md)** · **[Subtypes & facets](docs/guides/subtypes-and-facets.md)**
|
||||||
|
|
||||||
|
### Graph analytics built in
|
||||||
|
|
||||||
## ⚡ Zero to Production in 60 Seconds!
|
```javascript
|
||||||
|
await brain.graph.rank() // which entities matter most (centrality)
|
||||||
|
await brain.graph.communities() // natural clusters
|
||||||
|
await brain.graph.path(a, b) // how two things connect
|
||||||
|
await brain.graph.subgraph([seed], { depth: 2 }) // bounded neighborhood → { nodes, edges }
|
||||||
|
await brain.graph.export() // whole graph, one O(N+E) streaming pass
|
||||||
|
```
|
||||||
|
|
||||||
**The same code runs everywhere** - from your browser playground to enterprise production. No rewrites. No complexity. Just scale.
|
### Write-time aggregations
|
||||||
|
|
||||||
### 🎯 **Start Simple** (30 seconds)
|
`SUM` / `COUNT` / `AVG` / `MIN` / `MAX` with `GROUP BY` and time windows, maintained incrementally on every write — reads are O(1) lookups, not scans. **[Aggregation guide](docs/guides/aggregation.md)**
|
||||||
|
|
||||||
|
### Import anything
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
await brain.import('customers.csv')
|
||||||
|
await brain.import('sales.xlsx') // every sheet
|
||||||
|
await brain.import('research-paper.pdf') // tables extracted
|
||||||
|
await brain.import('https://api.example.com/data.json')
|
||||||
|
```
|
||||||
|
|
||||||
|
Entities auto-classify on the way in; `brain.extractEntities(text)` exposes the same NER ensemble directly. **[Import guide](docs/guides/import-anything.md)**
|
||||||
|
|
||||||
|
### A filesystem that understands content
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
await brain.vfs.writeFile('/docs/readme.md', 'Project documentation')
|
||||||
|
await brain.vfs.search('React components with hooks') // semantic file search
|
||||||
|
```
|
||||||
|
|
||||||
|
**[VFS quick start](docs/vfs/QUICK_START.md)**
|
||||||
|
|
||||||
|
### Operations-grade by default
|
||||||
|
|
||||||
|
- **Single-writer, many-reader** — an exclusive lock protects the data directory; `Brainy.openReadOnly()` and the `brainy inspect` CLI examine a live brain from another process, safely.
|
||||||
|
- **Self-upgrading data files** — a 7.x brain opens under 8.x and migrates itself behind an observable lock (`getIndexStatus().migration`), with an automatic pre-upgrade backup. No migration scripts.
|
||||||
|
- **No silent wrong answers** — cold-open guards self-heal or throw typed errors (`MetadataIndexNotReadyError`, `GraphIndexNotReadyError`); they never return `[]` for data that exists.
|
||||||
|
|
||||||
|
**[Multi-process model](docs/concepts/multi-process.md)** · **[Inspection guide](docs/guides/inspection.md)**
|
||||||
|
|
||||||
|
## From laptop to hundreds of millions
|
||||||
|
|
||||||
|
Brainy's TypeScript engines take you a long way. When you outgrow them, add the native engine — **the API doesn't change**:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm install @soulcraft/brainy
|
npm install @soulcraft/cor
|
||||||
```
|
```
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import { BrainyData } from '@soulcraft/brainy'
|
const brain = new Brainy({ storage: { type: 'filesystem', path: './data' } })
|
||||||
|
await brain.init() // @soulcraft/cor detected — same code, native engines underneath
|
||||||
// That's it! No config files. No setup. It just works! 🎉
|
|
||||||
const brain = new BrainyData()
|
|
||||||
await brain.init()
|
|
||||||
|
|
||||||
// Your data becomes intelligent instantly
|
|
||||||
await brain.add("Apple released the iPhone in 2007")
|
|
||||||
const results = await brain.search("smartphone history")
|
|
||||||
// Returns Apple info - it understands meaning!
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### 🚀 **Scale to Millions** (same code!)
|
Installing the package is the opt-in: if `@soulcraft/cor` is present, it loads and announces itself in the init log; if it's present but broken, `init()` **throws** — an installed accelerator never silently vanishes behind the JS engines. Opt out with `plugins: []`, or pin exactly what loads with `plugins: ['@soulcraft/cor']`. [`@soulcraft/cor`](https://www.npmjs.com/package/@soulcraft/cor) (Brainy 8.x ↔ Cor 3.x, version-matched) registers Rust implementations behind every provider seam: SIMD distance kernels, memory-mapped storage, a disk-native vector index that doesn't need your dataset in RAM, durable LSM field/graph indexes that serve cold opens instantly, and native aggregation. Recall@10 measured **0.99 / 0.96 / 0.96 at 1M / 10M / 100M vectors** in Cor's release gate.
|
||||||
```javascript
|
|
||||||
// THE EXACT SAME CODE scales to enterprise!
|
|
||||||
const brain = new BrainyData({
|
|
||||||
storage: { s3Storage: { bucketName: 'my-data' }} // Just add storage
|
|
||||||
})
|
|
||||||
|
|
||||||
// Now handling millions of records with:
|
Open core, commercial accelerator: Brainy is MIT and complete on its own; Cor is licensed and funds both.
|
||||||
// ✅ Automatic connection pooling (20x throughput)
|
|
||||||
// ✅ Write-ahead logging (zero data loss)
|
|
||||||
// ✅ Streaming import (unlimited size)
|
|
||||||
// ✅ Intelligent caching (sub-100ms queries)
|
|
||||||
await brain.import(millionRecords) // Streams automatically!
|
|
||||||
```
|
|
||||||
|
|
||||||
### 🌍 **Works Everywhere** (really!)
|
## Performance
|
||||||
```javascript
|
|
||||||
// In the browser (uses OPFS - no server needed!)
|
|
||||||
const brain = new BrainyData()
|
|
||||||
|
|
||||||
// On your laptop (uses local files)
|
- JS distance kernels: **~6× faster cosine, ~1.4× euclidean** than 7.x (measured: [`tests/benchmarks/distance-microbench.mjs`](tests/benchmarks/distance-microbench.mjs), 384-dim, median of 41).
|
||||||
const brain = new BrainyData()
|
- Whole-graph reads are single **O(N + E)** cursor walks — a consumer-measured 19k-edge export dropped from ~27 s of per-node calls to one scan.
|
||||||
|
- Full numbers and capacity planning: **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)** · **[docs/SCALING.md](docs/SCALING.md)**
|
||||||
|
|
||||||
// In production (uses S3/cloud)
|
## Use cases
|
||||||
const brain = new BrainyData({ storage: { s3Storage: {...} }})
|
|
||||||
|
|
||||||
// THE SAME API EVERYWHERE! 🎯
|
**AI agent memory** — persistent semantic recall with relationship tracking · **Knowledge bases** — auto-linking and meaning-aware navigation · **Semantic search** over codebases, documents, media · **Enterprise data** — CRM, catalogs, institutional memory · **Games & simulations** — worlds and characters that remember.
|
||||||
```
|
|
||||||
|
|
||||||
### ☁️ Brain Cloud (AI Memory + Agent Coordination)
|
## Documentation
|
||||||
```bash
|
|
||||||
# Auto-setup with cloud instance provisioning (RECOMMENDED)
|
|
||||||
brainy cloud setup --email your@email.com
|
|
||||||
|
|
||||||
# Sign up at app.soulcraft.com (free trial)
|
| Start | Core | Going deeper |
|
||||||
brainy cloud auth # Auto-configures based on your plan
|
|---|---|---|
|
||||||
```
|
| [Brainy explained simply](docs/eli5.md) | [API reference](docs/api/README.md) | [Architecture overview](docs/architecture/overview.md) |
|
||||||
|
| [Installation](docs/guides/installation.md) | [Data model](docs/DATA_MODEL.md) | [Consistency model](docs/concepts/consistency-model.md) |
|
||||||
|
| [Natural-language queries](docs/guides/natural-language.md) | [Query operators](docs/QUERY_OPERATORS.md) | [Multi-process model](docs/concepts/multi-process.md) |
|
||||||
|
| | [Find system](docs/FIND_SYSTEM.md) | [Scaling](docs/SCALING.md) |
|
||||||
|
|
||||||
```javascript
|
## Requirements
|
||||||
import { BrainyData, Cortex } from '@soulcraft/brainy'
|
|
||||||
// After authentication, augmentations auto-load
|
|
||||||
// No imports needed - they're managed by your account!
|
|
||||||
|
|
||||||
const brain = new BrainyData()
|
**Bun ≥ 1.1** (recommended) or **Node.js ≥ 22**. Brainy 8.x is server-only; the 7.x line remains on npm for browser use.
|
||||||
const cortex = new Cortex()
|
|
||||||
|
|
||||||
// Add augmentations to extend functionality
|
## Contributing & license
|
||||||
brain.register(new CustomAugmentation())
|
|
||||||
|
|
||||||
// Now your AI remembers everything across all sessions!
|
Contributions welcome — see **[CONTRIBUTING.md](CONTRIBUTING.md)**. MIT © Brainy Contributors.
|
||||||
await brain.add("User prefers TypeScript over JavaScript")
|
|
||||||
// This memory persists and syncs across all devices
|
|
||||||
// Returns: Microsoft and Anthropic with relevance scores
|
|
||||||
|
|
||||||
// Query relationships
|
|
||||||
const companies = await brain.getRelated("Sundar Pichai", { verb: "leads" })
|
|
||||||
// Returns: Google, Alphabet
|
|
||||||
|
|
||||||
// Filter with metadata
|
|
||||||
const recent = await brain.search("companies", 10, {
|
|
||||||
filter: { founded: { $gte: 2000 } }
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🧩 Augmentation System - Extend Your Brain
|
|
||||||
|
|
||||||
Brainy is **100% open source** with a powerful augmentation system. Choose what you need:
|
|
||||||
|
|
||||||
### 🆓 **Built-in Augmentations** (Always Free)
|
|
||||||
```javascript
|
|
||||||
import { NeuralImport } from '@soulcraft/brainy'
|
|
||||||
|
|
||||||
// AI-powered data understanding - included in every install
|
|
||||||
const neural = new NeuralImport(brain)
|
|
||||||
await neural.neuralImport('data.csv') // Automatically extracts entities & relationships
|
|
||||||
```
|
|
||||||
|
|
||||||
**Included augmentations:**
|
|
||||||
- ✅ **Neural Import** - AI understands your data structure
|
|
||||||
- ✅ **Basic Memory** - Persistent storage
|
|
||||||
- ✅ **Simple Search** - Text and vector search
|
|
||||||
- ✅ **Graph Traversal** - Relationship queries
|
|
||||||
|
|
||||||
### 🌟 **Community Augmentations** (Coming Soon!)
|
|
||||||
```javascript
|
|
||||||
// 🚧 FUTURE: Community augmentations will be available soon!
|
|
||||||
// These are examples of what the community could build:
|
|
||||||
|
|
||||||
// Example: Sentiment Analysis (not yet available)
|
|
||||||
// npm install brainy-sentiment
|
|
||||||
// brain.register(new SentimentAnalyzer())
|
|
||||||
|
|
||||||
// Example: Translation (not yet available)
|
|
||||||
// npm install brainy-translate
|
|
||||||
// brain.register(new Translator())
|
|
||||||
```
|
|
||||||
|
|
||||||
**Ideas for Community Augmentations:**
|
|
||||||
*Want to build one of these? We'll help promote it!*
|
|
||||||
- 🎭 Sentiment Analysis - Analyze emotional tone
|
|
||||||
- 🌍 Translation - Multi-language support
|
|
||||||
- 📧 Email Parser - Extract structured data from emails
|
|
||||||
- 🔗 URL Extractor - Find and validate URLs
|
|
||||||
- 📊 Data Visualizer - Generate charts from data
|
|
||||||
- 🎨 Image Understanding - Analyze image content
|
|
||||||
|
|
||||||
**Be the First!** Create an augmentation and we'll feature it here.
|
|
||||||
[See how to build augmentations →](UNIFIED-API.md#creating-your-own-augmentation)
|
|
||||||
|
|
||||||
### ☁️ **Brain Cloud** - Optional Cloud Services (Early Access) 🎆
|
|
||||||
|
|
||||||
**Currently in Early Access** - Join at [soulcraft.com](https://soulcraft.com)
|
|
||||||
|
|
||||||
**Available Tiers:**
|
|
||||||
|
|
||||||
#### 🆓 **Free Forever** - Local Database
|
|
||||||
- ✓ Full multi-dimensional database
|
|
||||||
- ✓ Works offline
|
|
||||||
- ✓ No API keys required
|
|
||||||
- ✓ Your data stays private
|
|
||||||
|
|
||||||
#### ☁️ **Cloud Sync** - $19/month
|
|
||||||
- ✓ Everything in Free tier
|
|
||||||
- ✓ Team collaboration
|
|
||||||
- ✓ Cross-device synchronization
|
|
||||||
- ✓ Automatic backups
|
|
||||||
- ✓ Real-time sync
|
|
||||||
|
|
||||||
#### 🏢 **Enterprise** - $99/month
|
|
||||||
- ✓ Everything in Cloud Sync
|
|
||||||
- ✓ Dedicated infrastructure
|
|
||||||
- ✓ Service Level Agreement (SLA)
|
|
||||||
- ✓ Priority support
|
|
||||||
- ✓ Custom integrations
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// Brain Cloud integration (when available):
|
|
||||||
const brain = new BrainyData({
|
|
||||||
cloud: {
|
|
||||||
enabled: true, // Enable cloud sync
|
|
||||||
apiKey: process.env.BRAIN_CLOUD_KEY // Optional for premium features
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
// Works perfectly without cloud too:
|
|
||||||
const brain = new BrainyData()
|
|
||||||
await brain.init()
|
|
||||||
// Full database functionality, locally!
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
### 🌐 **Why Brain Cloud?**
|
|
||||||
|
|
||||||
Brain Cloud adds optional cloud services to sustain Brainy's development:
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// Connect to Brain Cloud - your brain in the cloud
|
|
||||||
await brain.connect('brain-cloud.soulcraft.com', {
|
|
||||||
instance: 'my-team-brain',
|
|
||||||
apiKey: process.env.BRAIN_CLOUD_KEY
|
|
||||||
})
|
|
||||||
|
|
||||||
// Now your brain persists across:
|
|
||||||
// - Multiple developers
|
|
||||||
// - Different environments
|
|
||||||
// - AI agents
|
|
||||||
// - Sessions
|
|
||||||
```
|
|
||||||
|
|
||||||
**Brain Cloud features:**
|
|
||||||
- 🔄 Auto-sync across team
|
|
||||||
- 💾 Managed backups
|
|
||||||
- 🚀 Auto-scaling
|
|
||||||
- 🔒 Enterprise security
|
|
||||||
- 📊 Analytics dashboard
|
|
||||||
- 🤖 Multi-agent coordination
|
|
||||||
|
|
||||||
## 📝 Create Your Own Augmentation
|
|
||||||
|
|
||||||
### We ❤️ Open Source
|
|
||||||
|
|
||||||
**Brainy will ALWAYS be open source.** We believe in:
|
|
||||||
- 🌍 Community first
|
|
||||||
- 🔓 No vendor lock-in
|
|
||||||
- 🎁 Free forever core
|
|
||||||
- 🤝 Sustainable open source
|
|
||||||
|
|
||||||
### Build & Share Your Augmentation
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import { IAugmentation } from '@soulcraft/brainy'
|
|
||||||
|
|
||||||
export class MovieRecommender implements IAugmentation {
|
|
||||||
name = 'movie-recommender'
|
|
||||||
type = 'cognition' // sense|conduit|cognition|memory
|
|
||||||
description = 'AI-powered movie recommendations'
|
|
||||||
enabled = true
|
|
||||||
|
|
||||||
async processRawData(data: any) {
|
|
||||||
// Your recommendation logic
|
|
||||||
const movies = await this.analyzePreferences(data)
|
|
||||||
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: {
|
|
||||||
recommendations: movies,
|
|
||||||
confidence: 0.95
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Register with Brainy
|
|
||||||
const brain = new BrainyData()
|
|
||||||
brain.register(new MovieRecommender())
|
|
||||||
```
|
|
||||||
|
|
||||||
**Share with the community:**
|
|
||||||
```bash
|
|
||||||
npm publish brainy-movie-recommender
|
|
||||||
```
|
|
||||||
|
|
||||||
**Earn from your creation:**
|
|
||||||
- 💚 Keep it free (we'll promote it!)
|
|
||||||
- 💰 Sell licenses (we'll help distribute!)
|
|
||||||
- 🤝 Join our partner program
|
|
||||||
|
|
||||||
## 🎯 Real-World Examples
|
|
||||||
|
|
||||||
### Customer Support Bot with Memory
|
|
||||||
```javascript
|
|
||||||
// Your bot remembers every interaction
|
|
||||||
await brain.add({
|
|
||||||
customerId: "user_123",
|
|
||||||
issue: "Password reset",
|
|
||||||
resolved: true,
|
|
||||||
date: new Date()
|
|
||||||
})
|
|
||||||
|
|
||||||
// Next interaction knows the history
|
|
||||||
const history = await brain.search(`customer user_123`, 10)
|
|
||||||
// Bot says: "I see you had a password issue last week. All working now?"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Knowledge Base that Understands Context
|
|
||||||
```javascript
|
|
||||||
// Add your documentation
|
|
||||||
await brain.add("To deploy Brainy, run npm install @soulcraft/brainy")
|
|
||||||
await brain.add("Brainy requires Node.js 24.4.1 or higher")
|
|
||||||
await brain.add("For production, use Brain Cloud for scaling")
|
|
||||||
|
|
||||||
// Natural language queries work
|
|
||||||
const answer = await brain.search("how do I deploy to production?")
|
|
||||||
// Returns relevant docs about Brain Cloud and scaling
|
|
||||||
```
|
|
||||||
|
|
||||||
### Multi-Agent AI Systems
|
|
||||||
```javascript
|
|
||||||
// Agents share the same brain
|
|
||||||
const agentBrain = new BrainyData({ instance: 'shared-brain' })
|
|
||||||
|
|
||||||
// Sales Agent adds knowledge
|
|
||||||
await agentBrain.add("Customer interested in enterprise plan")
|
|
||||||
|
|
||||||
// Support Agent sees it instantly
|
|
||||||
const context = await agentBrain.search("customer plan interest")
|
|
||||||
|
|
||||||
// Marketing Agent learns from both
|
|
||||||
const insights = await agentBrain.getRelated("enterprise plan")
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🏗️ Architecture - Unified & Simple
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────────────┐
|
|
||||||
│ 🎯 YOUR APP - One Simple API │
|
|
||||||
│ brain.add() brain.search() brain.addVerb() │
|
|
||||||
└─────────────────┬───────────────────────────┘
|
|
||||||
│
|
|
||||||
┌─────────────────▼───────────────────────────┐
|
|
||||||
│ 🧠 BRAINY 1.0 - THE UNIFIED BRAIN │
|
|
||||||
│ │
|
|
||||||
│ ┌─────────────┐ ┌─────────────┐ ┌────────┐ │
|
|
||||||
│ │ Vector │ │ Graph │ │ Facets │ │
|
|
||||||
│ │ Search │ │Relationships│ │Metadata│ │
|
|
||||||
│ └─────────────┘ └─────────────┘ └────────┘ │
|
|
||||||
│ │
|
|
||||||
│ ┌─────────────┐ ┌─────────────┐ ┌────────┐ │
|
|
||||||
│ │ Encryption │ │ Memory │ │ Cache │ │
|
|
||||||
│ │ Universal │ │ Management │ │ 3-Tier │ │
|
|
||||||
│ └─────────────┘ └─────────────┘ └────────┘ │
|
|
||||||
└─────────────────┬───────────────────────────┘
|
|
||||||
│
|
|
||||||
┌─────────────────▼───────────────────────────┐
|
|
||||||
│ 💾 STORAGE - Universal Adapters │
|
|
||||||
│ Memory • FileSystem • S3 • OPFS • Custom │
|
|
||||||
└─────────────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
### **What Makes 1.0 Different:**
|
|
||||||
- **🎯 One API**: 9 methods handle everything (was 40+ methods)
|
|
||||||
- **🧠 Smart Core**: Automatic data understanding and processing
|
|
||||||
- **🔗 Graph Built-in**: Relationships are first-class citizens
|
|
||||||
- **🔐 Security Native**: Encryption integrated, not bolted-on
|
|
||||||
- **🧩 Extensible**: Augment with custom capabilities
|
|
||||||
- **📤 Portable**: Export in any format (json, csv, graph)
|
|
||||||
- **⚡ Zero Config**: Works perfectly out of the box
|
|
||||||
|
|
||||||
### **The Magic:**
|
|
||||||
1. **You call** `brain.add("complex data")`
|
|
||||||
2. **Brainy understands** → detects type, extracts meaning
|
|
||||||
3. **Brainy stores** → vector + graph + metadata simultaneously
|
|
||||||
4. **Brainy optimizes** → indexes, caches, tunes performance
|
|
||||||
5. **You get superpowers** → semantic search + graph traversal + more
|
|
||||||
|
|
||||||
## 🏢 Enterprise Features (NEW in 1.5!)
|
|
||||||
|
|
||||||
### 💾 **Write-Ahead Log (WAL) - Always On**
|
|
||||||
Zero data loss guarantee with intelligent configuration:
|
|
||||||
- **FileSystem/OPFS**: Aggressive durability (1-minute checkpoints)
|
|
||||||
- **S3/Cloud**: Cost-optimized (5-minute checkpoints, larger batches)
|
|
||||||
- **Memory**: Operation tracking for debugging
|
|
||||||
- **Automatic recovery** on startup from crashes
|
|
||||||
|
|
||||||
### 🎯 **Intelligent Verb Scoring - Now Default!**
|
|
||||||
AI-powered relationship quality:
|
|
||||||
```javascript
|
|
||||||
// Just add relationships - scoring happens automatically!
|
|
||||||
await brain.addVerb(person1, person2, "collaborates_with")
|
|
||||||
// Automatically scored based on:
|
|
||||||
// - Semantic similarity of entities
|
|
||||||
// - Frequency patterns
|
|
||||||
// - Temporal decay
|
|
||||||
// - Adaptive learning from usage
|
|
||||||
```
|
|
||||||
|
|
||||||
### ⚡ **Request Deduplication - 3x Performance**
|
|
||||||
Concurrent identical requests share results:
|
|
||||||
```javascript
|
|
||||||
// These fire simultaneously but only one executes
|
|
||||||
const [r1, r2, r3] = await Promise.all([
|
|
||||||
brain.search("AI"),
|
|
||||||
brain.search("AI"), // Returns instantly from first
|
|
||||||
brain.search("AI") // Returns instantly from first
|
|
||||||
])
|
|
||||||
```
|
|
||||||
|
|
||||||
### 🚀 **Auto-Tuning Everything**
|
|
||||||
- **Cache sizes** adjust to your usage patterns
|
|
||||||
- **Index parameters** optimize based on data
|
|
||||||
- **Write buffers** adapt to load
|
|
||||||
- **Connection pools** scale automatically
|
|
||||||
|
|
||||||
## 💡 Core Features
|
|
||||||
|
|
||||||
### 🔍 Multi-Dimensional Search
|
|
||||||
- **Vector**: Semantic similarity (meaning-based)
|
|
||||||
- **Graph**: Relationship traversal (connection-based)
|
|
||||||
- **Faceted**: Metadata filtering (property-based)
|
|
||||||
- **Hybrid**: All combined (maximum power)
|
|
||||||
|
|
||||||
### ⚡ Performance - Production Ready
|
|
||||||
- **Speed**: 100,000+ ops/second (faster with 1.0 optimizations)
|
|
||||||
- **Scale**: Millions of entities + relationships
|
|
||||||
- **Memory**: ~100MB for 1M vectors (16% smaller than 0.x)
|
|
||||||
- **Latency**: <10ms searches with 3-tier caching
|
|
||||||
- **Intelligence**: Auto-tuning learns from your usage patterns
|
|
||||||
|
|
||||||
### 🔒 Production Ready
|
|
||||||
- **Encryption**: End-to-end available
|
|
||||||
- **Persistence**: Multiple storage backends
|
|
||||||
- **Reliability**: 99.9% uptime in production
|
|
||||||
- **Security**: SOC2 compliant architecture
|
|
||||||
|
|
||||||
## 📚 Documentation
|
|
||||||
|
|
||||||
### Getting Started
|
|
||||||
- [**Quick Start Guide**](docs/getting-started/quick-start.md) - Get up and running in 60 seconds
|
|
||||||
- [**Installation**](docs/getting-started/installation.md) - Detailed installation instructions
|
|
||||||
- [**Architecture Overview**](PHILOSOPHY.md) - Design principles and philosophy
|
|
||||||
|
|
||||||
### Core Documentation
|
|
||||||
- [**API Reference**](docs/api/BRAINY-API-REFERENCE.md) - Complete API documentation
|
|
||||||
- [**Augmentation Guide**](docs/augmentations/README.md) - Build your own augmentations
|
|
||||||
- [**CLI Reference**](docs/brainy-cli.md) - Command-line interface
|
|
||||||
- [**All Documentation**](docs/README.md) - Browse all docs
|
|
||||||
|
|
||||||
### Guides
|
|
||||||
- [**Search & Metadata**](docs/user-guides/SEARCH_AND_METADATA_GUIDE.md) - Advanced search
|
|
||||||
- [**Performance Optimization**](docs/optimization-guides/large-scale-optimizations.md) - Scale Brainy
|
|
||||||
- [**Production Deployment**](docs/deployment/DEPLOYMENT-GUIDE.md) - Deploy to production
|
|
||||||
- [**Contributing Guidelines**](CONTRIBUTING.md) - Join the community
|
|
||||||
|
|
||||||
## 🤝 Our Promise to the Community
|
|
||||||
|
|
||||||
1. **Brainy core will ALWAYS be open source** (MIT License)
|
|
||||||
2. **No feature will ever move from free to paid**
|
|
||||||
3. **Community augmentations always welcome**
|
|
||||||
4. **We'll actively promote community creators**
|
|
||||||
5. **Commercial success funds open source development**
|
|
||||||
|
|
||||||
## 🙏 Join the Movement
|
|
||||||
|
|
||||||
### Ways to Contribute
|
|
||||||
- 🐛 Report bugs
|
|
||||||
- 💡 Suggest features
|
|
||||||
- 🔧 Submit PRs
|
|
||||||
- 📦 Create augmentations
|
|
||||||
- 📖 Improve docs
|
|
||||||
- ⭐ Star the repo
|
|
||||||
- 📢 Spread the word
|
|
||||||
|
|
||||||
### Get Help & Connect
|
|
||||||
- 📧 [Email Support](mailto:support@soulcraft.com)
|
|
||||||
- 🐛 [GitHub Issues](https://github.com/soulcraftlabs/brainy/issues)
|
|
||||||
- 💬 [GitHub Discussions](https://github.com/soulcraftlabs/brainy/discussions)
|
|
||||||
|
|
||||||
## 📈 Who's Using Brainy?
|
|
||||||
|
|
||||||
- 🚀 **Startups**: Building AI-first products
|
|
||||||
- 🏢 **Enterprises**: Replacing expensive databases
|
|
||||||
- 🎓 **Researchers**: Exploring knowledge graphs
|
|
||||||
- 👨💻 **Developers**: Creating smart applications
|
|
||||||
- 🤖 **AI Engineers**: Building RAG systems
|
|
||||||
|
|
||||||
## 📄 License
|
|
||||||
|
|
||||||
**MIT License** - Use it anywhere, build anything!
|
|
||||||
|
|
||||||
Premium augmentations available at [soulcraft.com](https://soulcraft.com)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
<div align="center">
|
|
||||||
|
|
||||||
### 🧠⚛️ **Give Your Data a Brain Upgrade**
|
|
||||||
|
|
||||||
**[Get Started](docs/getting-started/quick-start-1.0.md)** •
|
|
||||||
**[Examples](examples/)** •
|
|
||||||
**[API Docs](UNIFIED-API.md)** •
|
|
||||||
**[GitHub](https://github.com/soulcraftlabs/brainy)**
|
|
||||||
|
|
||||||
⭐ **Star us on GitHub to support open source AI!** ⭐
|
|
||||||
|
|
||||||
*Created and maintained by [SoulCraft](https://soulcraft.com) • Powered by our amazing open source community*
|
|
||||||
|
|
||||||
**SoulCraft** builds and maintains Brainy as open source (MIT License) because we believe AI infrastructure should be accessible to everyone.
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
|
||||||
2901
RELEASES.md
Normal file
2901
RELEASES.md
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -1,45 +0,0 @@
|
||||||
# Test Suite Cleanup Plan
|
|
||||||
|
|
||||||
## Tests to Remove (Redundant/Outdated)
|
|
||||||
|
|
||||||
### 1. Debug/Development Tests
|
|
||||||
- `metadata-filter-debug.test.ts` - Debug test, not needed in production
|
|
||||||
- `filter-discovery.test.ts` - Experimental/discovery test
|
|
||||||
|
|
||||||
### 2. Redundant Tests (Keep Best One)
|
|
||||||
- Keep `metadata-filter.test.ts`, remove `metadata-filter-environments.test.ts`
|
|
||||||
- Keep `s3-comprehensive.test.ts`, remove `s3-storage.test.ts`
|
|
||||||
- Keep `statistics.test.ts`, remove `statistics-storage.test.ts`
|
|
||||||
- Keep `performance.test.ts`, remove `performance-improvements.test.ts`
|
|
||||||
- Keep `storage-adapter-coverage.test.ts`, remove `storage-adapters.test.ts`
|
|
||||||
|
|
||||||
### 3. Outdated/Broken Tests
|
|
||||||
- `frozen-flag.test.ts` - Feature might be removed
|
|
||||||
- `distributed-config-migration.test.ts` - Old migration test
|
|
||||||
- `package-install.test.ts` - CI/CD concern, not unit test
|
|
||||||
|
|
||||||
## Tests to Fix
|
|
||||||
|
|
||||||
### 1. Missing destroy() method
|
|
||||||
- `regression.test.ts` - Remove destroy() calls or implement cleanup method
|
|
||||||
|
|
||||||
### 2. Update Expectations
|
|
||||||
- Storage tests expecting hard delete by default
|
|
||||||
- Statistics tests expecting exact counts
|
|
||||||
|
|
||||||
## Tests to Keep (Critical)
|
|
||||||
1. `core.test.ts` ✅
|
|
||||||
2. `unified-api.test.ts` ✅
|
|
||||||
3. `cli.test.ts` ✅
|
|
||||||
4. `vector-operations.test.ts`
|
|
||||||
5. `edge-cases.test.ts`
|
|
||||||
6. `error-handling.test.ts`
|
|
||||||
7. `environment.*.test.ts`
|
|
||||||
8. `opfs-storage.test.ts`
|
|
||||||
9. `brainy-chat.test.ts`
|
|
||||||
10. `intelligent-verb-scoring.test.ts`
|
|
||||||
|
|
||||||
## Expected Result
|
|
||||||
- Remove ~15 redundant/outdated tests
|
|
||||||
- Fix ~5 tests with wrong expectations
|
|
||||||
- Final count: ~400-450 meaningful tests instead of 600+
|
|
||||||
|
|
@ -1,10 +1,9 @@
|
||||||
{
|
{
|
||||||
"_name_or_path": "sentence-transformers/all-MiniLM-L6-v2",
|
"_name_or_path": "nreimers/MiniLM-L6-H384-uncased",
|
||||||
"architectures": [
|
"architectures": [
|
||||||
"BertModel"
|
"BertModel"
|
||||||
],
|
],
|
||||||
"attention_probs_dropout_prob": 0.1,
|
"attention_probs_dropout_prob": 0.1,
|
||||||
"classifier_dropout": null,
|
|
||||||
"gradient_checkpointing": false,
|
"gradient_checkpointing": false,
|
||||||
"hidden_act": "gelu",
|
"hidden_act": "gelu",
|
||||||
"hidden_dropout_prob": 0.1,
|
"hidden_dropout_prob": 0.1,
|
||||||
|
|
@ -18,7 +17,7 @@
|
||||||
"num_hidden_layers": 6,
|
"num_hidden_layers": 6,
|
||||||
"pad_token_id": 0,
|
"pad_token_id": 0,
|
||||||
"position_embedding_type": "absolute",
|
"position_embedding_type": "absolute",
|
||||||
"transformers_version": "4.29.2",
|
"transformers_version": "4.8.2",
|
||||||
"type_vocab_size": 2,
|
"type_vocab_size": 2,
|
||||||
"use_cache": true,
|
"use_cache": true,
|
||||||
"vocab_size": 30522
|
"vocab_size": 30522
|
||||||
BIN
assets/models/all-MiniLM-L6-v2/model.safetensors
Normal file
BIN
assets/models/all-MiniLM-L6-v2/model.safetensors
Normal file
Binary file not shown.
1
assets/models/all-MiniLM-L6-v2/tokenizer.json
Normal file
1
assets/models/all-MiniLM-L6-v2/tokenizer.json
Normal file
File diff suppressed because one or more lines are too long
564
bin/brainy-interactive.js
Normal file
564
bin/brainy-interactive.js
Normal file
|
|
@ -0,0 +1,564 @@
|
||||||
|
#!/usr/bin/env node
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Brainy Interactive Mode
|
||||||
|
*
|
||||||
|
* Professional, guided CLI experience for beginners
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { program } from 'commander'
|
||||||
|
import { Brainy } from '../dist/index.js'
|
||||||
|
import chalk from 'chalk'
|
||||||
|
import inquirer from 'inquirer'
|
||||||
|
import ora from 'ora'
|
||||||
|
import Table from 'cli-table3'
|
||||||
|
import boxen from 'boxen'
|
||||||
|
|
||||||
|
// Professional color scheme
|
||||||
|
const colors = {
|
||||||
|
primary: chalk.hex('#3A5F4A'), // Teal (from logo)
|
||||||
|
success: chalk.hex('#2D4A3A'), // Deep teal
|
||||||
|
info: chalk.hex('#4A6B5A'), // Medium teal
|
||||||
|
warning: chalk.hex('#D67441'), // Orange (from logo)
|
||||||
|
error: chalk.hex('#B85C35'), // Deep orange
|
||||||
|
brain: chalk.hex('#D67441'), // Brain orange
|
||||||
|
cream: chalk.hex('#F5E6A3'), // Cream background
|
||||||
|
dim: chalk.dim,
|
||||||
|
bold: chalk.bold,
|
||||||
|
cyan: chalk.cyan,
|
||||||
|
green: chalk.green,
|
||||||
|
yellow: chalk.yellow,
|
||||||
|
red: chalk.red
|
||||||
|
}
|
||||||
|
|
||||||
|
// Icons for consistent visual language
|
||||||
|
const icons = {
|
||||||
|
brain: '🧠',
|
||||||
|
search: '🔍',
|
||||||
|
add: '➕',
|
||||||
|
delete: '🗑️',
|
||||||
|
update: '🔄',
|
||||||
|
import: '📥',
|
||||||
|
export: '📤',
|
||||||
|
connect: '🔗',
|
||||||
|
question: '❓',
|
||||||
|
success: '✅',
|
||||||
|
error: '❌',
|
||||||
|
warning: '⚠️',
|
||||||
|
info: 'ℹ️',
|
||||||
|
sparkle: '✨',
|
||||||
|
rocket: '🚀',
|
||||||
|
thinking: '🤔',
|
||||||
|
chat: '💬',
|
||||||
|
stats: '📊',
|
||||||
|
config: '⚙️',
|
||||||
|
cloud: '☁️'
|
||||||
|
}
|
||||||
|
|
||||||
|
let brainyInstance = null
|
||||||
|
|
||||||
|
async function getBrainy() {
|
||||||
|
if (!brainyInstance) {
|
||||||
|
const spinner = ora('Initializing Brainy...').start()
|
||||||
|
try {
|
||||||
|
brainyInstance = new Brainy()
|
||||||
|
await brainyInstance.init()
|
||||||
|
spinner.succeed('Brainy initialized')
|
||||||
|
} catch (error) {
|
||||||
|
spinner.fail('Failed to initialize Brainy')
|
||||||
|
console.error(colors.error(error.message))
|
||||||
|
process.exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return brainyInstance
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Professional welcome screen
|
||||||
|
*/
|
||||||
|
function showWelcome() {
|
||||||
|
console.clear()
|
||||||
|
|
||||||
|
const welcomeBox = boxen(
|
||||||
|
colors.primary(`${icons.brain} BRAINY - Neural Intelligence System\n`) +
|
||||||
|
colors.dim('\nYour AI-Powered Second Brain\n') +
|
||||||
|
colors.info('Version 1.6.0'),
|
||||||
|
{
|
||||||
|
padding: 1,
|
||||||
|
margin: 1,
|
||||||
|
borderStyle: 'round',
|
||||||
|
borderColor: 'cyan',
|
||||||
|
textAlignment: 'center'
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
console.log(welcomeBox)
|
||||||
|
console.log()
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Main interactive menu
|
||||||
|
*/
|
||||||
|
async function mainMenu() {
|
||||||
|
const { action } = await inquirer.prompt([{
|
||||||
|
type: 'list',
|
||||||
|
name: 'action',
|
||||||
|
message: colors.cyan('What would you like to do?'),
|
||||||
|
choices: [
|
||||||
|
new inquirer.Separator(colors.dim('── Core Operations ──')),
|
||||||
|
{ name: `${icons.add} Add data to your brain`, value: 'add' },
|
||||||
|
{ name: `${icons.search} Search your knowledge`, value: 'search' },
|
||||||
|
{ name: `${icons.chat} Chat with your data`, value: 'chat' },
|
||||||
|
{ name: `${icons.update} Update existing data`, value: 'update' },
|
||||||
|
{ name: `${icons.delete} Delete data`, value: 'delete' },
|
||||||
|
|
||||||
|
new inquirer.Separator(colors.dim('── Advanced Features ──')),
|
||||||
|
{ name: `${icons.connect} Create relationships`, value: 'relate' },
|
||||||
|
{ name: `${icons.import} Import from file/URL`, value: 'import' },
|
||||||
|
{ name: `${icons.export} Export your brain`, value: 'export' },
|
||||||
|
{ name: `${icons.brain} Neural operations`, value: 'neural' },
|
||||||
|
|
||||||
|
new inquirer.Separator(colors.dim('── System ──')),
|
||||||
|
{ name: `${icons.stats} View statistics`, value: 'stats' },
|
||||||
|
{ name: `${icons.config} Configuration`, value: 'config' },
|
||||||
|
{ name: `${icons.cloud} Brain Cloud`, value: 'cloud' },
|
||||||
|
{ name: `${icons.info} Help & Documentation`, value: 'help' },
|
||||||
|
|
||||||
|
new inquirer.Separator(),
|
||||||
|
{ name: 'Exit', value: 'exit' }
|
||||||
|
],
|
||||||
|
pageSize: 20
|
||||||
|
}])
|
||||||
|
|
||||||
|
return action
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Neural operations submenu
|
||||||
|
*/
|
||||||
|
async function neuralMenu() {
|
||||||
|
const { operation } = await inquirer.prompt([{
|
||||||
|
type: 'list',
|
||||||
|
name: 'operation',
|
||||||
|
message: colors.cyan('Select neural operation:'),
|
||||||
|
choices: [
|
||||||
|
{ name: `${icons.brain} Calculate similarity`, value: 'similar' },
|
||||||
|
{ name: `${icons.search} Find clusters`, value: 'cluster' },
|
||||||
|
{ name: `${icons.connect} Find related items`, value: 'related' },
|
||||||
|
{ name: `${icons.thinking} Build hierarchy`, value: 'hierarchy' },
|
||||||
|
{ name: `${icons.rocket} Find semantic path`, value: 'path' },
|
||||||
|
{ name: `${icons.warning} Detect outliers`, value: 'outliers' },
|
||||||
|
{ name: `${icons.sparkle} Generate visualization`, value: 'visualize' },
|
||||||
|
new inquirer.Separator(),
|
||||||
|
{ name: '← Back to main menu', value: 'back' }
|
||||||
|
]
|
||||||
|
}])
|
||||||
|
|
||||||
|
return operation
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Execute commands with beautiful feedback
|
||||||
|
*/
|
||||||
|
async function executeCommand(command) {
|
||||||
|
const brain = await getBrainy()
|
||||||
|
|
||||||
|
switch (command) {
|
||||||
|
case 'add':
|
||||||
|
await interactiveAdd(brain)
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'search':
|
||||||
|
await interactiveSearch(brain)
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'chat':
|
||||||
|
await interactiveChat(brain)
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'update':
|
||||||
|
await interactiveUpdate(brain)
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'delete':
|
||||||
|
await interactiveDelete(brain)
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'relate':
|
||||||
|
await interactiveRelate(brain)
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'import':
|
||||||
|
await interactiveImport(brain)
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'export':
|
||||||
|
await interactiveExport(brain)
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'neural':
|
||||||
|
const neuralOp = await neuralMenu()
|
||||||
|
if (neuralOp !== 'back') {
|
||||||
|
await executeNeuralOperation(neuralOp, brain)
|
||||||
|
}
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'stats':
|
||||||
|
await showStatistics(brain)
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'config':
|
||||||
|
await interactiveConfig(brain)
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'cloud':
|
||||||
|
await showCloudInfo()
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'help':
|
||||||
|
await showHelp()
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Interactive add with rich prompts
|
||||||
|
*/
|
||||||
|
async function interactiveAdd(brain) {
|
||||||
|
console.log(colors.primary(`\n${icons.add} Add Data\n`))
|
||||||
|
|
||||||
|
const { inputType } = await inquirer.prompt([{
|
||||||
|
type: 'list',
|
||||||
|
name: 'inputType',
|
||||||
|
message: 'How would you like to add data?',
|
||||||
|
choices: [
|
||||||
|
{ name: 'Type or paste text', value: 'text' },
|
||||||
|
{ name: 'Multi-line editor', value: 'editor' },
|
||||||
|
{ name: 'JSON object', value: 'json' },
|
||||||
|
{ name: 'Import from clipboard', value: 'clipboard' }
|
||||||
|
]
|
||||||
|
}])
|
||||||
|
|
||||||
|
let data = ''
|
||||||
|
|
||||||
|
switch (inputType) {
|
||||||
|
case 'text':
|
||||||
|
const { text } = await inquirer.prompt([{
|
||||||
|
type: 'input',
|
||||||
|
name: 'text',
|
||||||
|
message: 'Enter your data:',
|
||||||
|
validate: input => input.trim() ? true : 'Please enter some data'
|
||||||
|
}])
|
||||||
|
data = text
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'editor':
|
||||||
|
const { editorText } = await inquirer.prompt([{
|
||||||
|
type: 'editor',
|
||||||
|
name: 'editorText',
|
||||||
|
message: 'Enter your data (opens editor):',
|
||||||
|
postfix: '.md'
|
||||||
|
}])
|
||||||
|
data = editorText
|
||||||
|
break
|
||||||
|
|
||||||
|
case 'json':
|
||||||
|
const { jsonText } = await inquirer.prompt([{
|
||||||
|
type: 'editor',
|
||||||
|
name: 'jsonText',
|
||||||
|
message: 'Enter JSON data:',
|
||||||
|
postfix: '.json',
|
||||||
|
default: '{\n \n}',
|
||||||
|
validate: input => {
|
||||||
|
try {
|
||||||
|
JSON.parse(input)
|
||||||
|
return true
|
||||||
|
} catch (e) {
|
||||||
|
return `Invalid JSON: ${e.message}`
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}])
|
||||||
|
data = jsonText
|
||||||
|
break
|
||||||
|
}
|
||||||
|
|
||||||
|
// Optional metadata
|
||||||
|
const { addMetadata } = await inquirer.prompt([{
|
||||||
|
type: 'confirm',
|
||||||
|
name: 'addMetadata',
|
||||||
|
message: 'Would you like to add metadata?',
|
||||||
|
default: false
|
||||||
|
}])
|
||||||
|
|
||||||
|
let metadata = {}
|
||||||
|
if (addMetadata) {
|
||||||
|
const { metadataJson } = await inquirer.prompt([{
|
||||||
|
type: 'editor',
|
||||||
|
name: 'metadataJson',
|
||||||
|
message: 'Enter metadata (JSON):',
|
||||||
|
postfix: '.json',
|
||||||
|
default: '{\n "type": "",\n "tags": [],\n "category": ""\n}',
|
||||||
|
validate: input => {
|
||||||
|
try {
|
||||||
|
JSON.parse(input)
|
||||||
|
return true
|
||||||
|
} catch (e) {
|
||||||
|
return `Invalid JSON: ${e.message}`
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}])
|
||||||
|
metadata = JSON.parse(metadataJson)
|
||||||
|
}
|
||||||
|
|
||||||
|
const spinner = ora('Adding data...').start()
|
||||||
|
try {
|
||||||
|
const id = await brain.add(data, metadata)
|
||||||
|
spinner.succeed(`Added successfully with ID: ${id}`)
|
||||||
|
|
||||||
|
// Show summary
|
||||||
|
console.log(boxen(
|
||||||
|
colors.success(`${icons.success} Data added successfully!\n\n`) +
|
||||||
|
colors.info(`ID: ${id}\n`) +
|
||||||
|
colors.dim(`Size: ${data.length} characters\n`) +
|
||||||
|
(Object.keys(metadata).length > 0 ? colors.dim(`Metadata: ${Object.keys(metadata).join(', ')}`) : ''),
|
||||||
|
{ padding: 1, borderColor: 'green', borderStyle: 'round' }
|
||||||
|
))
|
||||||
|
} catch (error) {
|
||||||
|
spinner.fail('Failed to add data')
|
||||||
|
console.error(colors.error(error.message))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Interactive search with filters
|
||||||
|
*/
|
||||||
|
async function interactiveSearch(brain) {
|
||||||
|
console.log(colors.primary(`\n${icons.search} Search\n`))
|
||||||
|
|
||||||
|
const { query } = await inquirer.prompt([{
|
||||||
|
type: 'input',
|
||||||
|
name: 'query',
|
||||||
|
message: 'Enter search query:',
|
||||||
|
validate: input => input.trim() ? true : 'Please enter a search query'
|
||||||
|
}])
|
||||||
|
|
||||||
|
// Advanced options
|
||||||
|
const { useFilters } = await inquirer.prompt([{
|
||||||
|
type: 'confirm',
|
||||||
|
name: 'useFilters',
|
||||||
|
message: 'Apply filters?',
|
||||||
|
default: false
|
||||||
|
}])
|
||||||
|
|
||||||
|
let searchOptions = { limit: 10 }
|
||||||
|
|
||||||
|
if (useFilters) {
|
||||||
|
const { limit, threshold } = await inquirer.prompt([
|
||||||
|
{
|
||||||
|
type: 'number',
|
||||||
|
name: 'limit',
|
||||||
|
message: 'Maximum results:',
|
||||||
|
default: 10
|
||||||
|
},
|
||||||
|
{
|
||||||
|
type: 'number',
|
||||||
|
name: 'threshold',
|
||||||
|
message: 'Similarity threshold (0-1):',
|
||||||
|
default: 0.5,
|
||||||
|
validate: input => input >= 0 && input <= 1 ? true : 'Must be between 0 and 1'
|
||||||
|
}
|
||||||
|
])
|
||||||
|
|
||||||
|
searchOptions.limit = limit
|
||||||
|
searchOptions.threshold = threshold
|
||||||
|
}
|
||||||
|
|
||||||
|
const spinner = ora('Searching...').start()
|
||||||
|
try {
|
||||||
|
const results = await brain.search(query, searchOptions.limit, searchOptions)
|
||||||
|
spinner.succeed(`Found ${results.length} results`)
|
||||||
|
|
||||||
|
if (results.length === 0) {
|
||||||
|
console.log(colors.warning('No results found'))
|
||||||
|
} else {
|
||||||
|
// Display results in a table
|
||||||
|
const table = new Table({
|
||||||
|
head: [colors.cyan('ID'), colors.cyan('Content'), colors.cyan('Score')],
|
||||||
|
style: { head: [], border: [] },
|
||||||
|
colWidths: [20, 50, 10]
|
||||||
|
})
|
||||||
|
|
||||||
|
results.forEach(result => {
|
||||||
|
const content = result.content || result.id
|
||||||
|
const truncated = content.length > 47 ? content.substring(0, 47) + '...' : content
|
||||||
|
const score = result.score ? `${(result.score * 100).toFixed(1)}%` : 'N/A'
|
||||||
|
|
||||||
|
table.push([
|
||||||
|
result.id.substring(0, 18),
|
||||||
|
truncated,
|
||||||
|
colors.green(score)
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
console.log(table.toString())
|
||||||
|
|
||||||
|
// Ask if user wants to see full details
|
||||||
|
const { viewDetails } = await inquirer.prompt([{
|
||||||
|
type: 'confirm',
|
||||||
|
name: 'viewDetails',
|
||||||
|
message: 'View full details of a result?',
|
||||||
|
default: false
|
||||||
|
}])
|
||||||
|
|
||||||
|
if (viewDetails) {
|
||||||
|
const { selectedId } = await inquirer.prompt([{
|
||||||
|
type: 'list',
|
||||||
|
name: 'selectedId',
|
||||||
|
message: 'Select result:',
|
||||||
|
choices: results.map(r => ({
|
||||||
|
name: `${r.id} - ${r.content?.substring(0, 50)}...`,
|
||||||
|
value: r.id
|
||||||
|
}))
|
||||||
|
}])
|
||||||
|
|
||||||
|
const selected = results.find(r => r.id === selectedId)
|
||||||
|
console.log(boxen(
|
||||||
|
colors.cyan('Full Details\n\n') +
|
||||||
|
colors.info(`ID: ${selected.id}\n\n`) +
|
||||||
|
`Content:\n${selected.content}\n\n` +
|
||||||
|
(selected.metadata ? `Metadata:\n${JSON.stringify(selected.metadata, null, 2)}` : ''),
|
||||||
|
{ padding: 1, borderColor: 'cyan', borderStyle: 'round' }
|
||||||
|
))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
spinner.fail('Search failed')
|
||||||
|
console.error(colors.error(error.message))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Show statistics with beautiful formatting
|
||||||
|
*/
|
||||||
|
async function showStatistics(brain) {
|
||||||
|
const spinner = ora('Gathering statistics...').start()
|
||||||
|
|
||||||
|
try {
|
||||||
|
const stats = brain.getStats()
|
||||||
|
spinner.succeed('Statistics loaded')
|
||||||
|
|
||||||
|
console.log(boxen(
|
||||||
|
colors.primary(`${icons.stats} Database Statistics\n\n`) +
|
||||||
|
colors.info(`Total Items: ${colors.bold(stats.total || 0)}\n`) +
|
||||||
|
colors.info(`Nouns: ${stats.nounCount || 0}\n`) +
|
||||||
|
colors.info(`Relationships: ${stats.verbCount || 0}\n`) +
|
||||||
|
colors.info(`Metadata Records: ${stats.metadataCount || 0}\n\n`) +
|
||||||
|
colors.dim(`Memory Usage: ${(process.memoryUsage().heapUsed / 1024 / 1024).toFixed(1)} MB`),
|
||||||
|
{
|
||||||
|
padding: 1,
|
||||||
|
borderColor: 'blue',
|
||||||
|
borderStyle: 'round',
|
||||||
|
textAlignment: 'left'
|
||||||
|
}
|
||||||
|
))
|
||||||
|
} catch (error) {
|
||||||
|
spinner.fail('Failed to get statistics')
|
||||||
|
console.error(colors.error(error.message))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Show help with examples
|
||||||
|
*/
|
||||||
|
async function showHelp() {
|
||||||
|
console.log(boxen(
|
||||||
|
colors.primary(`${icons.info} Brainy Help\n\n`) +
|
||||||
|
colors.cyan('Common Commands:\n') +
|
||||||
|
colors.dim(`
|
||||||
|
brainy add "text" Add data
|
||||||
|
brainy search "query" Search your brain
|
||||||
|
brainy chat Interactive AI chat
|
||||||
|
brainy status View statistics
|
||||||
|
brainy help This help menu
|
||||||
|
|
||||||
|
`) +
|
||||||
|
colors.cyan('Interactive Mode:\n') +
|
||||||
|
colors.dim(`
|
||||||
|
brainy Start interactive mode
|
||||||
|
brainy -i Alternative interactive mode
|
||||||
|
|
||||||
|
`) +
|
||||||
|
colors.cyan('Advanced Features:\n') +
|
||||||
|
colors.dim(`
|
||||||
|
brainy similar a b Calculate similarity
|
||||||
|
brainy cluster Find semantic clusters
|
||||||
|
brainy export Export your data
|
||||||
|
brainy cloud Brain Cloud features
|
||||||
|
`),
|
||||||
|
{ padding: 1, borderColor: 'yellow', borderStyle: 'round' }
|
||||||
|
))
|
||||||
|
|
||||||
|
const { learnMore } = await inquirer.prompt([{
|
||||||
|
type: 'confirm',
|
||||||
|
name: 'learnMore',
|
||||||
|
message: 'View detailed documentation?',
|
||||||
|
default: false
|
||||||
|
}])
|
||||||
|
|
||||||
|
if (learnMore) {
|
||||||
|
console.log(colors.info('\nDocumentation: https://github.com/TimeSoul/brainy'))
|
||||||
|
console.log(colors.info('Enterprise features: Coming in future releases'))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Main interactive loop
|
||||||
|
*/
|
||||||
|
async function main() {
|
||||||
|
showWelcome()
|
||||||
|
|
||||||
|
let running = true
|
||||||
|
while (running) {
|
||||||
|
const action = await mainMenu()
|
||||||
|
|
||||||
|
if (action === 'exit') {
|
||||||
|
console.log(colors.success(`\n${icons.success} Thank you for using Brainy!\n`))
|
||||||
|
running = false
|
||||||
|
} else {
|
||||||
|
await executeCommand(action)
|
||||||
|
|
||||||
|
// Pause before returning to menu
|
||||||
|
await inquirer.prompt([{
|
||||||
|
type: 'input',
|
||||||
|
name: 'continue',
|
||||||
|
message: colors.dim('\nPress Enter to continue...'),
|
||||||
|
prefix: ''
|
||||||
|
}])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
process.exit(0)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle errors gracefully
|
||||||
|
process.on('unhandledRejection', (error) => {
|
||||||
|
console.error(colors.error(`\n${icons.error} Unexpected error:`))
|
||||||
|
console.error(colors.red(error.message))
|
||||||
|
process.exit(1)
|
||||||
|
})
|
||||||
|
|
||||||
|
// Handle Ctrl+C gracefully
|
||||||
|
process.on('SIGINT', () => {
|
||||||
|
console.log(colors.info(`\n\n${icons.info} Exiting Brainy...`))
|
||||||
|
process.exit(0)
|
||||||
|
})
|
||||||
|
|
||||||
|
// Run if called directly
|
||||||
|
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||||
|
main().catch(error => {
|
||||||
|
console.error(colors.error('Fatal error:'), error)
|
||||||
|
process.exit(1)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
export { main as startInteractiveMode }
|
||||||
82
bin/brainy-minimal.js
Executable file
82
bin/brainy-minimal.js
Executable file
|
|
@ -0,0 +1,82 @@
|
||||||
|
#!/usr/bin/env node
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Brainy CLI - Minimal Version (Conversation Commands Only)
|
||||||
|
*
|
||||||
|
* This is a temporary minimal CLI that only includes working conversation commands
|
||||||
|
* Full CLI will be restored in version 3.20.0
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { Command } from 'commander'
|
||||||
|
import { readFileSync } from 'fs'
|
||||||
|
import { dirname, join } from 'path'
|
||||||
|
import { fileURLToPath } from 'url'
|
||||||
|
|
||||||
|
const __dirname = dirname(fileURLToPath(import.meta.url))
|
||||||
|
const packageJson = JSON.parse(readFileSync(join(__dirname, '..', 'package.json'), 'utf8'))
|
||||||
|
|
||||||
|
const program = new Command()
|
||||||
|
|
||||||
|
program
|
||||||
|
.name('brainy')
|
||||||
|
.description('🧠 Brainy - Infinite Agent Memory')
|
||||||
|
.version(packageJson.version)
|
||||||
|
|
||||||
|
// Dynamically load conversation command
|
||||||
|
const conversationCommand = await import('../dist/cli/commands/conversation.js').then(m => m.default)
|
||||||
|
|
||||||
|
program
|
||||||
|
.command('conversation')
|
||||||
|
.alias('conv')
|
||||||
|
.description('💬 Infinite agent memory and context management')
|
||||||
|
.addCommand(
|
||||||
|
new Command('setup')
|
||||||
|
.description('Set up MCP server for Claude Code integration')
|
||||||
|
.action(async () => {
|
||||||
|
await conversationCommand.handler({ action: 'setup', _: [] })
|
||||||
|
})
|
||||||
|
)
|
||||||
|
.addCommand(
|
||||||
|
new Command('remove')
|
||||||
|
.description('Remove MCP server and clean up')
|
||||||
|
.action(async () => {
|
||||||
|
await conversationCommand.handler({ action: 'remove', _: [] })
|
||||||
|
})
|
||||||
|
)
|
||||||
|
.addCommand(
|
||||||
|
new Command('search')
|
||||||
|
.description('Search messages across conversations')
|
||||||
|
.requiredOption('-q, --query <query>', 'Search query')
|
||||||
|
.option('-c, --conversation-id <id>', 'Filter by conversation')
|
||||||
|
.option('-r, --role <role>', 'Filter by role')
|
||||||
|
.option('-l, --limit <number>', 'Maximum results', '10')
|
||||||
|
.action(async (options) => {
|
||||||
|
await conversationCommand.handler({ action: 'search', ...options, _: [] })
|
||||||
|
})
|
||||||
|
)
|
||||||
|
.addCommand(
|
||||||
|
new Command('context')
|
||||||
|
.description('Get relevant context for a query')
|
||||||
|
.requiredOption('-q, --query <query>', 'Context query')
|
||||||
|
.option('-l, --limit <number>', 'Maximum messages', '10')
|
||||||
|
.action(async (options) => {
|
||||||
|
await conversationCommand.handler({ action: 'context', ...options, _: [] })
|
||||||
|
})
|
||||||
|
)
|
||||||
|
.addCommand(
|
||||||
|
new Command('thread')
|
||||||
|
.description('Get full conversation thread')
|
||||||
|
.requiredOption('-c, --conversation-id <id>', 'Conversation ID')
|
||||||
|
.action(async (options) => {
|
||||||
|
await conversationCommand.handler({ action: 'thread', ...options, _: [] })
|
||||||
|
})
|
||||||
|
)
|
||||||
|
.addCommand(
|
||||||
|
new Command('stats')
|
||||||
|
.description('Show conversation statistics')
|
||||||
|
.action(async () => {
|
||||||
|
await conversationCommand.handler({ action: 'stats', _: [] })
|
||||||
|
})
|
||||||
|
)
|
||||||
|
|
||||||
|
program.parse(process.argv)
|
||||||
18
bin/brainy-ts.js
Normal file
18
bin/brainy-ts.js
Normal file
|
|
@ -0,0 +1,18 @@
|
||||||
|
#!/usr/bin/env node
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Modern TypeScript CLI Runner
|
||||||
|
*
|
||||||
|
* This is the entry point after npm install @soulcraft/brainy
|
||||||
|
* It runs the compiled TypeScript CLI code
|
||||||
|
*/
|
||||||
|
|
||||||
|
// Use the compiled TypeScript CLI
|
||||||
|
import('../dist/cli/index.js').catch(err => {
|
||||||
|
// Fallback to legacy CLI if new one isn't built yet
|
||||||
|
import('./brainy.js').catch(() => {
|
||||||
|
console.error('Error: CLI not properly built. Please reinstall the package.')
|
||||||
|
console.error(err)
|
||||||
|
process.exit(1)
|
||||||
|
})
|
||||||
|
})
|
||||||
1470
bin/brainy.js
1470
bin/brainy.js
File diff suppressed because it is too large
Load diff
|
|
@ -1,110 +0,0 @@
|
||||||
<!DOCTYPE html>
|
|
||||||
<html>
|
|
||||||
<head>
|
|
||||||
<title>Brainy Models CDN</title>
|
|
||||||
<meta charset="utf-8">
|
|
||||||
<style>
|
|
||||||
body {
|
|
||||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
|
||||||
max-width: 900px;
|
|
||||||
margin: 50px auto;
|
|
||||||
padding: 20px;
|
|
||||||
background: #1a1a2e;
|
|
||||||
color: #eee;
|
|
||||||
}
|
|
||||||
.container {
|
|
||||||
background: rgba(255, 255, 255, 0.05);
|
|
||||||
border-radius: 10px;
|
|
||||||
padding: 30px;
|
|
||||||
border: 1px solid rgba(255, 255, 255, 0.1);
|
|
||||||
}
|
|
||||||
h1 {
|
|
||||||
color: #00ff88;
|
|
||||||
margin-top: 0;
|
|
||||||
}
|
|
||||||
.download-box {
|
|
||||||
background: rgba(0, 255, 136, 0.1);
|
|
||||||
border: 1px solid #00ff88;
|
|
||||||
padding: 20px;
|
|
||||||
border-radius: 5px;
|
|
||||||
margin: 20px 0;
|
|
||||||
}
|
|
||||||
.hash {
|
|
||||||
font-family: monospace;
|
|
||||||
background: rgba(0, 0, 0, 0.3);
|
|
||||||
padding: 5px 10px;
|
|
||||||
border-radius: 3px;
|
|
||||||
font-size: 12px;
|
|
||||||
word-break: break-all;
|
|
||||||
}
|
|
||||||
.critical {
|
|
||||||
background: rgba(255, 0, 0, 0.2);
|
|
||||||
border: 1px solid #ff4444;
|
|
||||||
padding: 15px;
|
|
||||||
border-radius: 5px;
|
|
||||||
margin: 20px 0;
|
|
||||||
}
|
|
||||||
code {
|
|
||||||
background: rgba(0, 0, 0, 0.3);
|
|
||||||
padding: 2px 6px;
|
|
||||||
border-radius: 3px;
|
|
||||||
}
|
|
||||||
a {
|
|
||||||
color: #00ff88;
|
|
||||||
text-decoration: none;
|
|
||||||
}
|
|
||||||
a:hover {
|
|
||||||
text-decoration: underline;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<div class="container">
|
|
||||||
<h1>🧠 Brainy Models CDN</h1>
|
|
||||||
|
|
||||||
<div class="critical">
|
|
||||||
<strong>⚠️ CRITICAL:</strong> These models MUST NEVER CHANGE<br>
|
|
||||||
They are the foundation of user data access. Changing them would break existing embeddings.
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h2>Model: all-MiniLM-L6-v2</h2>
|
|
||||||
<p>384-dimensional transformer model for embeddings</p>
|
|
||||||
|
|
||||||
<div class="download-box">
|
|
||||||
<h3>📦 Download Package</h3>
|
|
||||||
<p><a href="/models/all-MiniLM-L6-v2.tar.gz">all-MiniLM-L6-v2.tar.gz</a> (87MB)</p>
|
|
||||||
<p class="hash">SHA256: <span id="model-hash">Loading...</span></p>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h3>Individual Files:</h3>
|
|
||||||
<ul>
|
|
||||||
<li><a href="/models/Xenova/all-MiniLM-L6-v2/onnx/model.onnx">model.onnx</a> - ONNX model (87MB)</li>
|
|
||||||
<li><a href="/models/Xenova/all-MiniLM-L6-v2/tokenizer.json">tokenizer.json</a> - Tokenizer (695KB)</li>
|
|
||||||
<li><a href="/models/Xenova/all-MiniLM-L6-v2/config.json">config.json</a> - Configuration (650B)</li>
|
|
||||||
</ul>
|
|
||||||
|
|
||||||
<h3>Integration:</h3>
|
|
||||||
<pre><code># Direct download
|
|
||||||
curl -O https://models.soulcraft.com/models/all-MiniLM-L6-v2.tar.gz
|
|
||||||
|
|
||||||
# Verify integrity
|
|
||||||
echo "SHA256_HASH all-MiniLM-L6-v2.tar.gz" | sha256sum -c</code></pre>
|
|
||||||
|
|
||||||
<h3>Manifest:</h3>
|
|
||||||
<p><a href="/models/manifest.json">manifest.json</a> - Model manifest with all hashes</p>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<script>
|
|
||||||
// Load and display model hash
|
|
||||||
fetch('/models/manifest.json')
|
|
||||||
.then(r => r.json())
|
|
||||||
.then(data => {
|
|
||||||
document.getElementById('model-hash').textContent =
|
|
||||||
data.models['all-MiniLM-L6-v2'].sha256 || 'See manifest';
|
|
||||||
})
|
|
||||||
.catch(() => {
|
|
||||||
document.getElementById('model-hash').textContent = 'See manifest.json';
|
|
||||||
});
|
|
||||||
</script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -1,103 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
|
|
||||||
# Deploy Brainy Models to CDN
|
|
||||||
# This script uploads the CRITICAL transformer models to our CDN
|
|
||||||
|
|
||||||
set -e
|
|
||||||
|
|
||||||
echo "🧠 Brainy Models CDN Deployment"
|
|
||||||
echo "================================"
|
|
||||||
echo "⚠️ CRITICAL: These models MUST NEVER CHANGE"
|
|
||||||
echo ""
|
|
||||||
|
|
||||||
# Configuration
|
|
||||||
MODELS_DIR="../../models"
|
|
||||||
R2_BUCKET="brainy-models"
|
|
||||||
CDN_URL="https://models.soulcraft.com"
|
|
||||||
|
|
||||||
# Check if models exist locally
|
|
||||||
if [ ! -d "$MODELS_DIR/Xenova/all-MiniLM-L6-v2" ]; then
|
|
||||||
echo "❌ Models not found locally. Run 'npm run download-models' first."
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Calculate SHA256 hashes
|
|
||||||
echo "🔐 Calculating SHA256 hashes..."
|
|
||||||
MODEL_HASH=$(sha256sum $MODELS_DIR/Xenova/all-MiniLM-L6-v2/onnx/model.onnx | cut -d' ' -f1)
|
|
||||||
TOKENIZER_HASH=$(sha256sum $MODELS_DIR/Xenova/all-MiniLM-L6-v2/tokenizer.json | cut -d' ' -f1)
|
|
||||||
CONFIG_HASH=$(sha256sum $MODELS_DIR/Xenova/all-MiniLM-L6-v2/config.json | cut -d' ' -f1)
|
|
||||||
|
|
||||||
echo " model.onnx: $MODEL_HASH"
|
|
||||||
echo " tokenizer.json: $TOKENIZER_HASH"
|
|
||||||
echo " config.json: $CONFIG_HASH"
|
|
||||||
|
|
||||||
# Create tarball
|
|
||||||
echo "📦 Creating model tarball..."
|
|
||||||
cd $MODELS_DIR
|
|
||||||
tar -czf all-MiniLM-L6-v2.tar.gz Xenova/all-MiniLM-L6-v2/
|
|
||||||
TARBALL_HASH=$(sha256sum all-MiniLM-L6-v2.tar.gz | cut -d' ' -f1)
|
|
||||||
echo " Tarball SHA256: $TARBALL_HASH"
|
|
||||||
cd -
|
|
||||||
|
|
||||||
# Upload to R2
|
|
||||||
echo "☁️ Uploading to Cloudflare R2..."
|
|
||||||
|
|
||||||
# Upload individual files
|
|
||||||
wrangler r2 object put $R2_BUCKET/models/Xenova/all-MiniLM-L6-v2/onnx/model.onnx \
|
|
||||||
--file=$MODELS_DIR/Xenova/all-MiniLM-L6-v2/onnx/model.onnx \
|
|
||||||
--content-type="application/octet-stream"
|
|
||||||
|
|
||||||
wrangler r2 object put $R2_BUCKET/models/Xenova/all-MiniLM-L6-v2/tokenizer.json \
|
|
||||||
--file=$MODELS_DIR/Xenova/all-MiniLM-L6-v2/tokenizer.json \
|
|
||||||
--content-type="application/json"
|
|
||||||
|
|
||||||
wrangler r2 object put $R2_BUCKET/models/Xenova/all-MiniLM-L6-v2/config.json \
|
|
||||||
--file=$MODELS_DIR/Xenova/all-MiniLM-L6-v2/config.json \
|
|
||||||
--content-type="application/json"
|
|
||||||
|
|
||||||
wrangler r2 object put $R2_BUCKET/models/Xenova/all-MiniLM-L6-v2/tokenizer_config.json \
|
|
||||||
--file=$MODELS_DIR/Xenova/all-MiniLM-L6-v2/tokenizer_config.json \
|
|
||||||
--content-type="application/json"
|
|
||||||
|
|
||||||
# Upload tarball
|
|
||||||
wrangler r2 object put $R2_BUCKET/tarballs/all-MiniLM-L6-v2.tar.gz \
|
|
||||||
--file=$MODELS_DIR/all-MiniLM-L6-v2.tar.gz \
|
|
||||||
--content-type="application/gzip"
|
|
||||||
|
|
||||||
# Deploy Worker
|
|
||||||
echo "🚀 Deploying Cloudflare Worker..."
|
|
||||||
wrangler deploy
|
|
||||||
|
|
||||||
# Create immutable backup
|
|
||||||
echo "💾 Creating immutable backup..."
|
|
||||||
BACKUP_NAME="models-backup-$(date +%Y%m%d)-$MODEL_HASH.tar.gz"
|
|
||||||
cp $MODELS_DIR/all-MiniLM-L6-v2.tar.gz $BACKUP_NAME
|
|
||||||
|
|
||||||
# Save hashes for verification
|
|
||||||
cat > model-hashes.json <<EOF
|
|
||||||
{
|
|
||||||
"version": "1.0.0",
|
|
||||||
"created": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
|
|
||||||
"hashes": {
|
|
||||||
"model.onnx": "$MODEL_HASH",
|
|
||||||
"tokenizer.json": "$TOKENIZER_HASH",
|
|
||||||
"config.json": "$CONFIG_HASH",
|
|
||||||
"tarball": "$TARBALL_HASH"
|
|
||||||
},
|
|
||||||
"cdn": "$CDN_URL",
|
|
||||||
"backup": "$BACKUP_NAME"
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
echo "✅ Deployment complete!"
|
|
||||||
echo ""
|
|
||||||
echo "📍 CDN URL: $CDN_URL/brainy/v1/all-MiniLM-L6-v2.tar.gz"
|
|
||||||
echo "🔒 Hashes saved to: model-hashes.json"
|
|
||||||
echo "💾 Backup saved as: $BACKUP_NAME"
|
|
||||||
echo ""
|
|
||||||
echo "⚠️ CRITICAL REMINDER:"
|
|
||||||
echo " These models are now immutable and cached globally."
|
|
||||||
echo " They MUST NEVER be changed or users will lose access to their data."
|
|
||||||
echo ""
|
|
||||||
echo "To verify deployment:"
|
|
||||||
echo " curl -I $CDN_URL/brainy/v1/all-MiniLM-L6-v2.tar.gz"
|
|
||||||
|
|
@ -1,302 +0,0 @@
|
||||||
/**
|
|
||||||
* Brainy Models CDN - Cloudflare Workers
|
|
||||||
*
|
|
||||||
* CRITICAL: These models MUST NEVER CHANGE
|
|
||||||
* They are the foundation of user data access
|
|
||||||
*/
|
|
||||||
|
|
||||||
export interface Env {
|
|
||||||
MODEL_BUCKET: R2Bucket
|
|
||||||
MODELS: KVNamespace
|
|
||||||
DOWNLOAD_TRACKER: DurableObjectNamespace
|
|
||||||
}
|
|
||||||
|
|
||||||
// Model manifest with SHA256 hashes for verification
|
|
||||||
const MODEL_MANIFEST = {
|
|
||||||
'all-MiniLM-L6-v2': {
|
|
||||||
version: '1.0.0',
|
|
||||||
files: {
|
|
||||||
'model.onnx': {
|
|
||||||
path: 'Xenova/all-MiniLM-L6-v2/onnx/model.onnx',
|
|
||||||
size: 90555481,
|
|
||||||
sha256: 'TO_BE_COMPUTED', // Will compute from actual file
|
|
||||||
contentType: 'application/octet-stream'
|
|
||||||
},
|
|
||||||
'tokenizer.json': {
|
|
||||||
path: 'Xenova/all-MiniLM-L6-v2/tokenizer.json',
|
|
||||||
size: 711661,
|
|
||||||
sha256: 'TO_BE_COMPUTED',
|
|
||||||
contentType: 'application/json'
|
|
||||||
},
|
|
||||||
'config.json': {
|
|
||||||
path: 'Xenova/all-MiniLM-L6-v2/config.json',
|
|
||||||
size: 650,
|
|
||||||
sha256: 'TO_BE_COMPUTED',
|
|
||||||
contentType: 'application/json'
|
|
||||||
},
|
|
||||||
'tokenizer_config.json': {
|
|
||||||
path: 'Xenova/all-MiniLM-L6-v2/tokenizer_config.json',
|
|
||||||
size: 366,
|
|
||||||
sha256: 'TO_BE_COMPUTED',
|
|
||||||
contentType: 'application/json'
|
|
||||||
}
|
|
||||||
},
|
|
||||||
tarball: 'all-MiniLM-L6-v2.tar.gz'
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
export default {
|
|
||||||
async fetch(
|
|
||||||
request: Request,
|
|
||||||
env: Env,
|
|
||||||
ctx: ExecutionContext
|
|
||||||
): Promise<Response> {
|
|
||||||
const url = new URL(request.url)
|
|
||||||
|
|
||||||
// CORS headers for browser access
|
|
||||||
const corsHeaders = {
|
|
||||||
'Access-Control-Allow-Origin': '*',
|
|
||||||
'Access-Control-Allow-Methods': 'GET, HEAD, OPTIONS',
|
|
||||||
'Access-Control-Max-Age': '86400',
|
|
||||||
'Cache-Control': 'public, max-age=31536000, immutable' // 1 year cache
|
|
||||||
}
|
|
||||||
|
|
||||||
// Handle CORS preflight
|
|
||||||
if (request.method === 'OPTIONS') {
|
|
||||||
return new Response(null, { headers: corsHeaders })
|
|
||||||
}
|
|
||||||
|
|
||||||
// Parse the path
|
|
||||||
const path = url.pathname.slice(1) // Remove leading /
|
|
||||||
|
|
||||||
// Home page - show status
|
|
||||||
if (!path || path === '/') {
|
|
||||||
return handleStatus(env)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Download model tarball
|
|
||||||
if (path === 'brainy/v1/all-MiniLM-L6-v2.tar.gz') {
|
|
||||||
return handleTarballDownload(env, corsHeaders)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Download individual model file
|
|
||||||
if (path.startsWith('brainy/v1/')) {
|
|
||||||
return handleFileDownload(path.replace('brainy/v1/', ''), env, corsHeaders)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Model manifest
|
|
||||||
if (path === 'brainy/manifest.json') {
|
|
||||||
return new Response(JSON.stringify(MODEL_MANIFEST, null, 2), {
|
|
||||||
headers: {
|
|
||||||
...corsHeaders,
|
|
||||||
'Content-Type': 'application/json'
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// Health check
|
|
||||||
if (path === 'health') {
|
|
||||||
return handleHealthCheck(env)
|
|
||||||
}
|
|
||||||
|
|
||||||
return new Response('Not Found', { status: 404 })
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleStatus(env: Env): Promise<Response> {
|
|
||||||
const html = `
|
|
||||||
<!DOCTYPE html>
|
|
||||||
<html>
|
|
||||||
<head>
|
|
||||||
<title>Brainy Models CDN</title>
|
|
||||||
<style>
|
|
||||||
body {
|
|
||||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
|
||||||
max-width: 800px;
|
|
||||||
margin: 50px auto;
|
|
||||||
padding: 20px;
|
|
||||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
|
||||||
color: white;
|
|
||||||
}
|
|
||||||
.container {
|
|
||||||
background: rgba(255, 255, 255, 0.1);
|
|
||||||
border-radius: 10px;
|
|
||||||
padding: 30px;
|
|
||||||
backdrop-filter: blur(10px);
|
|
||||||
}
|
|
||||||
h1 { margin-top: 0; }
|
|
||||||
.status {
|
|
||||||
background: #10b981;
|
|
||||||
display: inline-block;
|
|
||||||
padding: 5px 10px;
|
|
||||||
border-radius: 5px;
|
|
||||||
font-weight: bold;
|
|
||||||
}
|
|
||||||
code {
|
|
||||||
background: rgba(0, 0, 0, 0.2);
|
|
||||||
padding: 2px 6px;
|
|
||||||
border-radius: 3px;
|
|
||||||
}
|
|
||||||
.endpoint {
|
|
||||||
background: rgba(0, 0, 0, 0.3);
|
|
||||||
padding: 15px;
|
|
||||||
border-radius: 5px;
|
|
||||||
margin: 10px 0;
|
|
||||||
font-family: monospace;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<div class="container">
|
|
||||||
<h1>🧠 Brainy Models CDN</h1>
|
|
||||||
<p class="status">✅ OPERATIONAL</p>
|
|
||||||
|
|
||||||
<h2>Critical Model Hosting</h2>
|
|
||||||
<p>This CDN hosts the transformer models required for Brainy operations.</p>
|
|
||||||
<p><strong>⚠️ These models MUST NEVER change</strong> - they are the foundation of user data access.</p>
|
|
||||||
|
|
||||||
<h3>Available Models:</h3>
|
|
||||||
<div class="endpoint">
|
|
||||||
<strong>all-MiniLM-L6-v2</strong> (v1.0.0)<br>
|
|
||||||
384-dimensional embeddings<br>
|
|
||||||
Size: ~87MB<br>
|
|
||||||
SHA256: Verified on every request
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h3>Endpoints:</h3>
|
|
||||||
<div class="endpoint">
|
|
||||||
GET /brainy/v1/all-MiniLM-L6-v2.tar.gz<br>
|
|
||||||
→ Complete model package (tar.gz)
|
|
||||||
</div>
|
|
||||||
<div class="endpoint">
|
|
||||||
GET /brainy/v1/Xenova/all-MiniLM-L6-v2/onnx/model.onnx<br>
|
|
||||||
→ Individual model file
|
|
||||||
</div>
|
|
||||||
<div class="endpoint">
|
|
||||||
GET /brainy/manifest.json<br>
|
|
||||||
→ Model manifest with hashes
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h3>Integration:</h3>
|
|
||||||
<code>https://models.soulcraft.com/brainy/v1/all-MiniLM-L6-v2.tar.gz</code>
|
|
||||||
|
|
||||||
<h3>Features:</h3>
|
|
||||||
<ul>
|
|
||||||
<li>🚀 Global edge deployment (Cloudflare)</li>
|
|
||||||
<li>🔒 SHA256 verification</li>
|
|
||||||
<li>📦 Immutable model versioning</li>
|
|
||||||
<li>⚡ 1-year browser cache</li>
|
|
||||||
<li>🌍 CORS enabled</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
`
|
|
||||||
|
|
||||||
return new Response(html, {
|
|
||||||
headers: {
|
|
||||||
'Content-Type': 'text/html'
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleTarballDownload(
|
|
||||||
env: Env,
|
|
||||||
headers: Record<string, string>
|
|
||||||
): Promise<Response> {
|
|
||||||
// Get from R2 bucket
|
|
||||||
const object = await env.MODEL_BUCKET.get('tarballs/all-MiniLM-L6-v2.tar.gz')
|
|
||||||
|
|
||||||
if (!object) {
|
|
||||||
return new Response('Model not found', { status: 404 })
|
|
||||||
}
|
|
||||||
|
|
||||||
// Return with proper headers
|
|
||||||
return new Response(object.body, {
|
|
||||||
headers: {
|
|
||||||
...headers,
|
|
||||||
'Content-Type': 'application/gzip',
|
|
||||||
'Content-Disposition': 'attachment; filename="all-MiniLM-L6-v2.tar.gz"',
|
|
||||||
'X-Model-Version': '1.0.0',
|
|
||||||
'X-Model-SHA256': MODEL_MANIFEST['all-MiniLM-L6-v2'].files['model.onnx'].sha256
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleFileDownload(
|
|
||||||
path: string,
|
|
||||||
env: Env,
|
|
||||||
headers: Record<string, string>
|
|
||||||
): Promise<Response> {
|
|
||||||
// Find the file in manifest
|
|
||||||
let fileInfo = null
|
|
||||||
let modelName = ''
|
|
||||||
|
|
||||||
for (const [model, config] of Object.entries(MODEL_MANIFEST)) {
|
|
||||||
for (const [_, file] of Object.entries(config.files)) {
|
|
||||||
if (file.path === path) {
|
|
||||||
fileInfo = file
|
|
||||||
modelName = model
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (fileInfo) break
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!fileInfo) {
|
|
||||||
return new Response('File not found', { status: 404 })
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get from R2
|
|
||||||
const object = await env.MODEL_BUCKET.get(`models/${path}`)
|
|
||||||
|
|
||||||
if (!object) {
|
|
||||||
return new Response('Model file not found', { status: 404 })
|
|
||||||
}
|
|
||||||
|
|
||||||
// Verify size
|
|
||||||
if (object.size !== fileInfo.size) {
|
|
||||||
console.error(`Size mismatch for ${path}: expected ${fileInfo.size}, got ${object.size}`)
|
|
||||||
return new Response('Model integrity check failed', { status: 500 })
|
|
||||||
}
|
|
||||||
|
|
||||||
return new Response(object.body, {
|
|
||||||
headers: {
|
|
||||||
...headers,
|
|
||||||
'Content-Type': fileInfo.contentType,
|
|
||||||
'Content-Length': fileInfo.size.toString(),
|
|
||||||
'X-Model-SHA256': fileInfo.sha256,
|
|
||||||
'X-Model-Name': modelName,
|
|
||||||
'ETag': `"${fileInfo.sha256}"`
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleHealthCheck(env: Env): Promise<Response> {
|
|
||||||
try {
|
|
||||||
// Check R2 bucket is accessible
|
|
||||||
const testFile = await env.MODEL_BUCKET.head('health.txt')
|
|
||||||
|
|
||||||
return new Response(JSON.stringify({
|
|
||||||
status: 'healthy',
|
|
||||||
timestamp: new Date().toISOString(),
|
|
||||||
models: Object.keys(MODEL_MANIFEST),
|
|
||||||
cdn: 'cloudflare',
|
|
||||||
region: 'global'
|
|
||||||
}), {
|
|
||||||
headers: {
|
|
||||||
'Content-Type': 'application/json'
|
|
||||||
}
|
|
||||||
})
|
|
||||||
} catch (error) {
|
|
||||||
return new Response(JSON.stringify({
|
|
||||||
status: 'unhealthy',
|
|
||||||
error: (error as Error).message
|
|
||||||
}), {
|
|
||||||
status: 503,
|
|
||||||
headers: {
|
|
||||||
'Content-Type': 'application/json'
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,34 +0,0 @@
|
||||||
name = "brainy-models-cdn"
|
|
||||||
main = "src/index.ts"
|
|
||||||
compatibility_date = "2024-01-01"
|
|
||||||
|
|
||||||
# Custom domain
|
|
||||||
routes = [
|
|
||||||
{ pattern = "models.soulcraft.com/*", zone_name = "soulcraft.com" }
|
|
||||||
]
|
|
||||||
|
|
||||||
# KV namespace for model storage
|
|
||||||
kv_namespaces = [
|
|
||||||
{ binding = "MODELS", id = "brainy_models_kv" }
|
|
||||||
]
|
|
||||||
|
|
||||||
# R2 bucket for large model files
|
|
||||||
r2_buckets = [
|
|
||||||
{ binding = "MODEL_BUCKET", bucket_name = "brainy-models" }
|
|
||||||
]
|
|
||||||
|
|
||||||
[env.production]
|
|
||||||
vars = { ENVIRONMENT = "production" }
|
|
||||||
|
|
||||||
# Cache everything for 1 year (models never change)
|
|
||||||
[site]
|
|
||||||
bucket = "./models"
|
|
||||||
|
|
||||||
[build]
|
|
||||||
command = "npm install && npm run build"
|
|
||||||
|
|
||||||
# Durable Objects for download tracking
|
|
||||||
[[durable_objects.bindings]]
|
|
||||||
name = "DOWNLOAD_TRACKER"
|
|
||||||
class_name = "DownloadTracker"
|
|
||||||
script_name = "download-tracker"
|
|
||||||
|
|
@ -1,334 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
|
|
||||||
# Setup Google Cloud Storage for Brainy Models CDN
|
|
||||||
# This creates an immutable model hosting solution at models.soulcraft.com
|
|
||||||
|
|
||||||
set -e
|
|
||||||
|
|
||||||
echo "🧠 Setting up Brainy Models CDN on Google Cloud Storage"
|
|
||||||
echo "========================================================"
|
|
||||||
|
|
||||||
# Configuration
|
|
||||||
PROJECT_ID="soulcraft-brain"
|
|
||||||
BUCKET_NAME="models.soulcraft.com"
|
|
||||||
MODELS_DIR="../../models"
|
|
||||||
DOMAIN="models.soulcraft.com"
|
|
||||||
|
|
||||||
# Check if models exist locally
|
|
||||||
if [ ! -d "$MODELS_DIR/Xenova/all-MiniLM-L6-v2" ]; then
|
|
||||||
echo "❌ Models not found. Run 'npm run download-models' first."
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Set the project
|
|
||||||
echo "📍 Setting project: $PROJECT_ID"
|
|
||||||
gcloud config set project $PROJECT_ID
|
|
||||||
|
|
||||||
# Create the bucket (if it doesn't exist)
|
|
||||||
echo "🪣 Creating GCS bucket: $BUCKET_NAME"
|
|
||||||
gsutil mb -p $PROJECT_ID -c STANDARD -l US -b on gs://$BUCKET_NAME 2>/dev/null || echo "Bucket already exists"
|
|
||||||
|
|
||||||
# Enable public access
|
|
||||||
echo "🌍 Enabling public access..."
|
|
||||||
gsutil iam ch allUsers:objectViewer gs://$BUCKET_NAME
|
|
||||||
|
|
||||||
# Set CORS policy for browser access
|
|
||||||
echo "🔧 Setting CORS policy..."
|
|
||||||
cat > cors.json <<EOF
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"origin": ["*"],
|
|
||||||
"method": ["GET", "HEAD"],
|
|
||||||
"responseHeader": ["Content-Type", "Content-Length", "X-Model-Version", "X-Model-SHA256"],
|
|
||||||
"maxAgeSeconds": 31536000
|
|
||||||
}
|
|
||||||
]
|
|
||||||
EOF
|
|
||||||
gsutil cors set cors.json gs://$BUCKET_NAME
|
|
||||||
|
|
||||||
# Calculate SHA256 hashes
|
|
||||||
echo "🔐 Calculating SHA256 hashes..."
|
|
||||||
MODEL_HASH=$(sha256sum $MODELS_DIR/Xenova/all-MiniLM-L6-v2/onnx/model.onnx | cut -d' ' -f1)
|
|
||||||
TOKENIZER_HASH=$(sha256sum $MODELS_DIR/Xenova/all-MiniLM-L6-v2/tokenizer.json | cut -d' ' -f1)
|
|
||||||
CONFIG_HASH=$(sha256sum $MODELS_DIR/Xenova/all-MiniLM-L6-v2/config.json | cut -d' ' -f1)
|
|
||||||
|
|
||||||
echo " model.onnx: $MODEL_HASH"
|
|
||||||
echo " tokenizer.json: $TOKENIZER_HASH"
|
|
||||||
echo " config.json: $CONFIG_HASH"
|
|
||||||
|
|
||||||
# Create tarball
|
|
||||||
echo "📦 Creating model package..."
|
|
||||||
cd $MODELS_DIR
|
|
||||||
tar -czf all-MiniLM-L6-v2.tar.gz Xenova/all-MiniLM-L6-v2/
|
|
||||||
TARBALL_HASH=$(sha256sum all-MiniLM-L6-v2.tar.gz | cut -d' ' -f1)
|
|
||||||
cd -
|
|
||||||
|
|
||||||
# Upload model files with cache headers
|
|
||||||
echo "☁️ Uploading models to GCS..."
|
|
||||||
|
|
||||||
# Upload individual files
|
|
||||||
gsutil -h "Cache-Control:public, max-age=31536000, immutable" \
|
|
||||||
-h "Content-Type:application/octet-stream" \
|
|
||||||
cp $MODELS_DIR/Xenova/all-MiniLM-L6-v2/onnx/model.onnx \
|
|
||||||
gs://$BUCKET_NAME/models/Xenova/all-MiniLM-L6-v2/onnx/model.onnx
|
|
||||||
|
|
||||||
gsutil -h "Cache-Control:public, max-age=31536000, immutable" \
|
|
||||||
-h "Content-Type:application/json" \
|
|
||||||
cp $MODELS_DIR/Xenova/all-MiniLM-L6-v2/tokenizer.json \
|
|
||||||
gs://$BUCKET_NAME/models/Xenova/all-MiniLM-L6-v2/tokenizer.json
|
|
||||||
|
|
||||||
gsutil -h "Cache-Control:public, max-age=31536000, immutable" \
|
|
||||||
-h "Content-Type:application/json" \
|
|
||||||
cp $MODELS_DIR/Xenova/all-MiniLM-L6-v2/config.json \
|
|
||||||
gs://$BUCKET_NAME/models/Xenova/all-MiniLM-L6-v2/config.json
|
|
||||||
|
|
||||||
gsutil -h "Cache-Control:public, max-age=31536000, immutable" \
|
|
||||||
-h "Content-Type:application/json" \
|
|
||||||
cp $MODELS_DIR/Xenova/all-MiniLM-L6-v2/tokenizer_config.json \
|
|
||||||
gs://$BUCKET_NAME/models/Xenova/all-MiniLM-L6-v2/tokenizer_config.json
|
|
||||||
|
|
||||||
# Upload tarball
|
|
||||||
gsutil -h "Cache-Control:public, max-age=31536000, immutable" \
|
|
||||||
-h "Content-Type:application/gzip" \
|
|
||||||
cp $MODELS_DIR/all-MiniLM-L6-v2.tar.gz \
|
|
||||||
gs://$BUCKET_NAME/models/all-MiniLM-L6-v2.tar.gz
|
|
||||||
|
|
||||||
# Create and upload manifest
|
|
||||||
echo "📝 Creating manifest..."
|
|
||||||
cat > manifest.json <<EOF
|
|
||||||
{
|
|
||||||
"version": "1.0.0",
|
|
||||||
"created": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
|
|
||||||
"models": {
|
|
||||||
"all-MiniLM-L6-v2": {
|
|
||||||
"version": "1.0.0",
|
|
||||||
"dimensions": 384,
|
|
||||||
"sha256": "$TARBALL_HASH",
|
|
||||||
"files": {
|
|
||||||
"model.onnx": {
|
|
||||||
"size": 90555481,
|
|
||||||
"sha256": "$MODEL_HASH",
|
|
||||||
"path": "models/Xenova/all-MiniLM-L6-v2/onnx/model.onnx"
|
|
||||||
},
|
|
||||||
"tokenizer.json": {
|
|
||||||
"size": 711661,
|
|
||||||
"sha256": "$TOKENIZER_HASH",
|
|
||||||
"path": "models/Xenova/all-MiniLM-L6-v2/tokenizer.json"
|
|
||||||
},
|
|
||||||
"config.json": {
|
|
||||||
"size": 650,
|
|
||||||
"sha256": "$CONFIG_HASH",
|
|
||||||
"path": "models/Xenova/all-MiniLM-L6-v2/config.json"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"download": {
|
|
||||||
"tarball": "https://$DOMAIN/models/all-MiniLM-L6-v2.tar.gz",
|
|
||||||
"github": "https://github.com/soulcraftlabs/brainy-models/releases/download/v1.0.0/all-MiniLM-L6-v2.tar.gz"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
gsutil -h "Cache-Control:public, max-age=3600" \
|
|
||||||
-h "Content-Type:application/json" \
|
|
||||||
cp manifest.json gs://$BUCKET_NAME/models/manifest.json
|
|
||||||
|
|
||||||
# Create and upload index.html
|
|
||||||
echo "🎨 Creating index page..."
|
|
||||||
cat > index.html <<EOF
|
|
||||||
<!DOCTYPE html>
|
|
||||||
<html>
|
|
||||||
<head>
|
|
||||||
<title>Brainy Models CDN - Soulcraft</title>
|
|
||||||
<meta charset="utf-8">
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
||||||
<style>
|
|
||||||
body {
|
|
||||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
|
||||||
max-width: 900px;
|
|
||||||
margin: 50px auto;
|
|
||||||
padding: 20px;
|
|
||||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
|
||||||
min-height: 100vh;
|
|
||||||
}
|
|
||||||
.container {
|
|
||||||
background: white;
|
|
||||||
border-radius: 10px;
|
|
||||||
padding: 40px;
|
|
||||||
box-shadow: 0 20px 60px rgba(0,0,0,0.3);
|
|
||||||
}
|
|
||||||
h1 {
|
|
||||||
color: #333;
|
|
||||||
margin-top: 0;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 10px;
|
|
||||||
}
|
|
||||||
.status {
|
|
||||||
background: #10b981;
|
|
||||||
color: white;
|
|
||||||
padding: 5px 15px;
|
|
||||||
border-radius: 20px;
|
|
||||||
font-size: 14px;
|
|
||||||
font-weight: bold;
|
|
||||||
}
|
|
||||||
.critical {
|
|
||||||
background: #fef2f2;
|
|
||||||
border-left: 4px solid #ef4444;
|
|
||||||
padding: 15px;
|
|
||||||
margin: 20px 0;
|
|
||||||
border-radius: 5px;
|
|
||||||
}
|
|
||||||
.download-box {
|
|
||||||
background: #f0fdf4;
|
|
||||||
border: 1px solid #10b981;
|
|
||||||
padding: 20px;
|
|
||||||
border-radius: 8px;
|
|
||||||
margin: 20px 0;
|
|
||||||
}
|
|
||||||
.hash {
|
|
||||||
font-family: monospace;
|
|
||||||
background: #f3f4f6;
|
|
||||||
padding: 8px;
|
|
||||||
border-radius: 4px;
|
|
||||||
font-size: 12px;
|
|
||||||
word-break: break-all;
|
|
||||||
display: inline-block;
|
|
||||||
margin-top: 5px;
|
|
||||||
}
|
|
||||||
code {
|
|
||||||
background: #f3f4f6;
|
|
||||||
padding: 2px 6px;
|
|
||||||
border-radius: 3px;
|
|
||||||
font-family: monospace;
|
|
||||||
}
|
|
||||||
pre {
|
|
||||||
background: #1f2937;
|
|
||||||
color: #e5e7eb;
|
|
||||||
padding: 15px;
|
|
||||||
border-radius: 8px;
|
|
||||||
overflow-x: auto;
|
|
||||||
}
|
|
||||||
a {
|
|
||||||
color: #667eea;
|
|
||||||
text-decoration: none;
|
|
||||||
}
|
|
||||||
a:hover {
|
|
||||||
text-decoration: underline;
|
|
||||||
}
|
|
||||||
.file-list {
|
|
||||||
background: #f9fafb;
|
|
||||||
padding: 15px;
|
|
||||||
border-radius: 8px;
|
|
||||||
margin: 10px 0;
|
|
||||||
}
|
|
||||||
.file-list li {
|
|
||||||
margin: 8px 0;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<div class="container">
|
|
||||||
<h1>🧠 Brainy Models CDN <span class="status">ACTIVE</span></h1>
|
|
||||||
|
|
||||||
<div class="critical">
|
|
||||||
<strong>⚠️ CRITICAL:</strong> These models MUST NEVER CHANGE<br>
|
|
||||||
<small>They are the foundation of user data access. Any change would break existing embeddings and make user data inaccessible.</small>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h2>Transformer Model: all-MiniLM-L6-v2</h2>
|
|
||||||
<p>This 384-dimensional transformer model is essential for Brainy's vector operations.</p>
|
|
||||||
|
|
||||||
<div class="download-box">
|
|
||||||
<h3>📦 Complete Package</h3>
|
|
||||||
<p>
|
|
||||||
<strong><a href="/models/all-MiniLM-L6-v2.tar.gz">all-MiniLM-L6-v2.tar.gz</a></strong> (87MB)<br>
|
|
||||||
<span class="hash">SHA256: $TARBALL_HASH</span>
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h3>Individual Files:</h3>
|
|
||||||
<div class="file-list">
|
|
||||||
<ul>
|
|
||||||
<li>
|
|
||||||
<a href="/models/Xenova/all-MiniLM-L6-v2/onnx/model.onnx">model.onnx</a> - ONNX Runtime model (87MB)<br>
|
|
||||||
<span class="hash">$MODEL_HASH</span>
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
<a href="/models/Xenova/all-MiniLM-L6-v2/tokenizer.json">tokenizer.json</a> - Tokenizer configuration (695KB)<br>
|
|
||||||
<span class="hash">$TOKENIZER_HASH</span>
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
<a href="/models/Xenova/all-MiniLM-L6-v2/config.json">config.json</a> - Model configuration (650B)<br>
|
|
||||||
<span class="hash">$CONFIG_HASH</span>
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h3>Integration:</h3>
|
|
||||||
<pre><code># Download and verify
|
|
||||||
curl -O https://models.soulcraft.com/models/all-MiniLM-L6-v2.tar.gz
|
|
||||||
echo "$TARBALL_HASH all-MiniLM-L6-v2.tar.gz" | sha256sum -c
|
|
||||||
|
|
||||||
# Extract
|
|
||||||
tar -xzf all-MiniLM-L6-v2.tar.gz</code></pre>
|
|
||||||
|
|
||||||
<h3>API Endpoints:</h3>
|
|
||||||
<ul>
|
|
||||||
<li><code>GET /models/manifest.json</code> - Model manifest with all hashes</li>
|
|
||||||
<li><code>GET /models/all-MiniLM-L6-v2.tar.gz</code> - Complete model package</li>
|
|
||||||
<li><code>GET /models/Xenova/all-MiniLM-L6-v2/*</code> - Individual model files</li>
|
|
||||||
</ul>
|
|
||||||
|
|
||||||
<h3>Features:</h3>
|
|
||||||
<ul>
|
|
||||||
<li>✅ Immutable content (1-year cache headers)</li>
|
|
||||||
<li>✅ SHA256 verification for integrity</li>
|
|
||||||
<li>✅ Global CDN via Google Cloud</li>
|
|
||||||
<li>✅ CORS enabled for browser access</li>
|
|
||||||
<li>✅ 99.95% uptime SLA</li>
|
|
||||||
</ul>
|
|
||||||
|
|
||||||
<hr style="margin: 30px 0; border: none; border-top: 1px solid #e5e7eb;">
|
|
||||||
|
|
||||||
<p style="text-align: center; color: #6b7280; font-size: 14px;">
|
|
||||||
Powered by Google Cloud Storage |
|
|
||||||
<a href="/models/manifest.json">View Manifest</a> |
|
|
||||||
<a href="https://github.com/soulcraftlabs/brainy">GitHub</a>
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
EOF
|
|
||||||
|
|
||||||
gsutil -h "Cache-Control:public, max-age=3600" \
|
|
||||||
-h "Content-Type:text/html" \
|
|
||||||
cp index.html gs://$BUCKET_NAME/index.html
|
|
||||||
|
|
||||||
# Set up website configuration
|
|
||||||
echo "🌐 Configuring website settings..."
|
|
||||||
gsutil web set -m index.html -e 404.html gs://$BUCKET_NAME
|
|
||||||
|
|
||||||
# Create a load balancer (optional - for custom domain)
|
|
||||||
echo ""
|
|
||||||
echo "✅ GCS CDN setup complete!"
|
|
||||||
echo ""
|
|
||||||
echo "📍 Access your models at:"
|
|
||||||
echo " https://storage.googleapis.com/$BUCKET_NAME/index.html"
|
|
||||||
echo " https://storage.googleapis.com/$BUCKET_NAME/models/all-MiniLM-L6-v2.tar.gz"
|
|
||||||
echo ""
|
|
||||||
echo "To point models.soulcraft.com to this bucket:"
|
|
||||||
echo "1. Add a CNAME record: models.soulcraft.com -> c.storage.googleapis.com"
|
|
||||||
echo "2. Verify domain ownership in Google Cloud Console"
|
|
||||||
echo "3. The bucket name must match the domain (models.soulcraft.com)"
|
|
||||||
echo ""
|
|
||||||
echo "Model hashes:"
|
|
||||||
echo " Tarball: $TARBALL_HASH"
|
|
||||||
echo " Model: $MODEL_HASH"
|
|
||||||
echo ""
|
|
||||||
echo "⚠️ Remember: These models must NEVER change!"
|
|
||||||
|
|
||||||
# Cleanup
|
|
||||||
rm cors.json manifest.json index.html
|
|
||||||
rm $MODELS_DIR/all-MiniLM-L6-v2.tar.gz
|
|
||||||
86
dist/augmentationFactory.d.ts
vendored
86
dist/augmentationFactory.d.ts
vendored
|
|
@ -1,86 +0,0 @@
|
||||||
/**
|
|
||||||
* Augmentation Factory
|
|
||||||
*
|
|
||||||
* This module provides a simplified factory for creating augmentations with minimal boilerplate.
|
|
||||||
* It reduces the complexity of creating and using augmentations by providing a fluent API
|
|
||||||
* and handling common patterns automatically.
|
|
||||||
*/
|
|
||||||
import { IAugmentation, AugmentationResponse, ISenseAugmentation, IConduitAugmentation, IMemoryAugmentation, IWebSocketSupport, WebSocketConnection } from './types/augmentations.js';
|
|
||||||
/**
|
|
||||||
* Options for creating an augmentation
|
|
||||||
*/
|
|
||||||
export interface AugmentationOptions {
|
|
||||||
name: string;
|
|
||||||
description?: string;
|
|
||||||
enabled?: boolean;
|
|
||||||
autoRegister?: boolean;
|
|
||||||
autoInitialize?: boolean;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Factory for creating sense augmentations
|
|
||||||
*/
|
|
||||||
export declare function createSenseAugmentation(options: AugmentationOptions & {
|
|
||||||
processRawData?: (rawData: Buffer | string, dataType: string) => Promise<AugmentationResponse<{
|
|
||||||
nouns: string[];
|
|
||||||
verbs: string[];
|
|
||||||
}>> | AugmentationResponse<{
|
|
||||||
nouns: string[];
|
|
||||||
verbs: string[];
|
|
||||||
}>;
|
|
||||||
listenToFeed?: (feedUrl: string, callback: (data: {
|
|
||||||
nouns: string[];
|
|
||||||
verbs: string[];
|
|
||||||
}) => void) => Promise<void>;
|
|
||||||
}): ISenseAugmentation;
|
|
||||||
/**
|
|
||||||
* Factory for creating conduit augmentations
|
|
||||||
*/
|
|
||||||
export declare function createConduitAugmentation(options: AugmentationOptions & {
|
|
||||||
establishConnection?: (targetSystemId: string, config: Record<string, unknown>) => Promise<AugmentationResponse<WebSocketConnection>> | AugmentationResponse<WebSocketConnection>;
|
|
||||||
readData?: (query: Record<string, unknown>, options?: Record<string, unknown>) => Promise<AugmentationResponse<unknown>> | AugmentationResponse<unknown>;
|
|
||||||
writeData?: (data: Record<string, unknown>, options?: Record<string, unknown>) => Promise<AugmentationResponse<unknown>> | AugmentationResponse<unknown>;
|
|
||||||
monitorStream?: (streamId: string, callback: (data: unknown) => void) => Promise<void>;
|
|
||||||
}): IConduitAugmentation;
|
|
||||||
/**
|
|
||||||
* Factory for creating memory augmentations
|
|
||||||
*/
|
|
||||||
export declare function createMemoryAugmentation(options: AugmentationOptions & {
|
|
||||||
storeData?: (key: string, data: unknown, options?: Record<string, unknown>) => Promise<AugmentationResponse<boolean>> | AugmentationResponse<boolean>;
|
|
||||||
retrieveData?: (key: string, options?: Record<string, unknown>) => Promise<AugmentationResponse<unknown>> | AugmentationResponse<unknown>;
|
|
||||||
updateData?: (key: string, data: unknown, options?: Record<string, unknown>) => Promise<AugmentationResponse<boolean>> | AugmentationResponse<boolean>;
|
|
||||||
deleteData?: (key: string, options?: Record<string, unknown>) => Promise<AugmentationResponse<boolean>> | AugmentationResponse<boolean>;
|
|
||||||
listDataKeys?: (pattern?: string, options?: Record<string, unknown>) => Promise<AugmentationResponse<string[]>> | AugmentationResponse<string[]>;
|
|
||||||
search?: (query: unknown, k?: number, options?: Record<string, unknown>) => Promise<AugmentationResponse<Array<{
|
|
||||||
id: string;
|
|
||||||
score: number;
|
|
||||||
data: unknown;
|
|
||||||
}>>> | AugmentationResponse<Array<{
|
|
||||||
id: string;
|
|
||||||
score: number;
|
|
||||||
data: unknown;
|
|
||||||
}>>;
|
|
||||||
}): IMemoryAugmentation;
|
|
||||||
/**
|
|
||||||
* Factory for creating WebSocket-enabled augmentations
|
|
||||||
* This can be combined with other augmentation factories to create WebSocket-enabled versions
|
|
||||||
*/
|
|
||||||
export declare function addWebSocketSupport<T extends IAugmentation>(augmentation: T, options: {
|
|
||||||
connectWebSocket?: (url: string, protocols?: string | string[]) => Promise<WebSocketConnection>;
|
|
||||||
sendWebSocketMessage?: (connectionId: string, data: unknown) => Promise<void>;
|
|
||||||
onWebSocketMessage?: (connectionId: string, callback: (data: unknown) => void) => Promise<void>;
|
|
||||||
offWebSocketMessage?: (connectionId: string, callback: (data: unknown) => void) => Promise<void>;
|
|
||||||
closeWebSocket?: (connectionId: string, code?: number, reason?: string) => Promise<void>;
|
|
||||||
}): T & IWebSocketSupport;
|
|
||||||
/**
|
|
||||||
* Simplified function to execute an augmentation method with automatic error handling
|
|
||||||
* This provides a more concise way to execute augmentation methods compared to the full pipeline
|
|
||||||
*/
|
|
||||||
export declare function executeAugmentation<T, R>(augmentation: IAugmentation, method: string, ...args: any[]): Promise<AugmentationResponse<R>>;
|
|
||||||
/**
|
|
||||||
* Dynamically load augmentations from a module at runtime
|
|
||||||
* This allows for lazy-loading augmentations when needed instead of at build time
|
|
||||||
*/
|
|
||||||
export declare function loadAugmentationModule(modulePromise: Promise<any>, options?: {
|
|
||||||
autoRegister?: boolean;
|
|
||||||
autoInitialize?: boolean;
|
|
||||||
}): Promise<IAugmentation[]>;
|
|
||||||
342
dist/augmentationFactory.js
vendored
342
dist/augmentationFactory.js
vendored
|
|
@ -1,342 +0,0 @@
|
||||||
/**
|
|
||||||
* Augmentation Factory
|
|
||||||
*
|
|
||||||
* This module provides a simplified factory for creating augmentations with minimal boilerplate.
|
|
||||||
* It reduces the complexity of creating and using augmentations by providing a fluent API
|
|
||||||
* and handling common patterns automatically.
|
|
||||||
*/
|
|
||||||
import { registerAugmentation } from './augmentationRegistry.js';
|
|
||||||
/**
|
|
||||||
* Base class for all augmentations created with the factory
|
|
||||||
* Handles common functionality like initialization, shutdown, and status
|
|
||||||
*/
|
|
||||||
class BaseAugmentation {
|
|
||||||
constructor(options) {
|
|
||||||
this.enabled = true;
|
|
||||||
this.isInitialized = false;
|
|
||||||
this.name = options.name;
|
|
||||||
this.description = options.description || `${options.name} augmentation`;
|
|
||||||
this.enabled = options.enabled !== false;
|
|
||||||
}
|
|
||||||
async initialize() {
|
|
||||||
if (this.isInitialized)
|
|
||||||
return;
|
|
||||||
this.isInitialized = true;
|
|
||||||
}
|
|
||||||
async shutDown() {
|
|
||||||
this.isInitialized = false;
|
|
||||||
}
|
|
||||||
async getStatus() {
|
|
||||||
return this.isInitialized ? 'active' : 'inactive';
|
|
||||||
}
|
|
||||||
async ensureInitialized() {
|
|
||||||
if (!this.isInitialized) {
|
|
||||||
await this.initialize();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Factory for creating sense augmentations
|
|
||||||
*/
|
|
||||||
export function createSenseAugmentation(options) {
|
|
||||||
const augmentation = new BaseAugmentation(options);
|
|
||||||
// Implement the sense augmentation methods
|
|
||||||
augmentation.processRawData = async (rawData, dataType) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.processRawData) {
|
|
||||||
const result = options.processRawData(rawData, dataType);
|
|
||||||
return result instanceof Promise ? await result : result;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: { nouns: [], verbs: [] },
|
|
||||||
error: 'processRawData not implemented'
|
|
||||||
};
|
|
||||||
};
|
|
||||||
augmentation.listenToFeed = async (feedUrl, callback) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.listenToFeed) {
|
|
||||||
return options.listenToFeed(feedUrl, callback);
|
|
||||||
}
|
|
||||||
throw new Error('listenToFeed not implemented');
|
|
||||||
};
|
|
||||||
// Auto-register if requested
|
|
||||||
if (options.autoRegister) {
|
|
||||||
registerAugmentation(augmentation);
|
|
||||||
// Auto-initialize if requested
|
|
||||||
if (options.autoInitialize) {
|
|
||||||
augmentation.initialize().catch((error) => {
|
|
||||||
console.error(`Failed to initialize augmentation ${augmentation.name}:`, error);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return augmentation;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Factory for creating conduit augmentations
|
|
||||||
*/
|
|
||||||
export function createConduitAugmentation(options) {
|
|
||||||
const augmentation = new BaseAugmentation(options);
|
|
||||||
// Implement the conduit augmentation methods
|
|
||||||
augmentation.establishConnection = async (targetSystemId, config) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.establishConnection) {
|
|
||||||
const result = options.establishConnection(targetSystemId, config);
|
|
||||||
return result instanceof Promise ? await result : result;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'establishConnection not implemented'
|
|
||||||
};
|
|
||||||
};
|
|
||||||
augmentation.readData = async (query, opts) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.readData) {
|
|
||||||
const result = options.readData(query, opts);
|
|
||||||
return result instanceof Promise ? await result : result;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'readData not implemented'
|
|
||||||
};
|
|
||||||
};
|
|
||||||
augmentation.writeData = async (data, opts) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.writeData) {
|
|
||||||
const result = options.writeData(data, opts);
|
|
||||||
return result instanceof Promise ? await result : result;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'writeData not implemented'
|
|
||||||
};
|
|
||||||
};
|
|
||||||
augmentation.monitorStream = async (streamId, callback) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.monitorStream) {
|
|
||||||
return options.monitorStream(streamId, callback);
|
|
||||||
}
|
|
||||||
throw new Error('monitorStream not implemented');
|
|
||||||
};
|
|
||||||
// Auto-register if requested
|
|
||||||
if (options.autoRegister) {
|
|
||||||
registerAugmentation(augmentation);
|
|
||||||
// Auto-initialize if requested
|
|
||||||
if (options.autoInitialize) {
|
|
||||||
augmentation.initialize().catch((error) => {
|
|
||||||
console.error(`Failed to initialize augmentation ${augmentation.name}:`, error);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return augmentation;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Factory for creating memory augmentations
|
|
||||||
*/
|
|
||||||
export function createMemoryAugmentation(options) {
|
|
||||||
const augmentation = new BaseAugmentation(options);
|
|
||||||
// Implement the memory augmentation methods
|
|
||||||
augmentation.storeData = async (key, data, opts) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.storeData) {
|
|
||||||
const result = options.storeData(key, data, opts);
|
|
||||||
return result instanceof Promise ? await result : result;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: false,
|
|
||||||
error: 'storeData not implemented'
|
|
||||||
};
|
|
||||||
};
|
|
||||||
augmentation.retrieveData = async (key, opts) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.retrieveData) {
|
|
||||||
const result = options.retrieveData(key, opts);
|
|
||||||
return result instanceof Promise ? await result : result;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'retrieveData not implemented'
|
|
||||||
};
|
|
||||||
};
|
|
||||||
augmentation.updateData = async (key, data, opts) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.updateData) {
|
|
||||||
const result = options.updateData(key, data, opts);
|
|
||||||
return result instanceof Promise ? await result : result;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: false,
|
|
||||||
error: 'updateData not implemented'
|
|
||||||
};
|
|
||||||
};
|
|
||||||
augmentation.deleteData = async (key, opts) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.deleteData) {
|
|
||||||
const result = options.deleteData(key, opts);
|
|
||||||
return result instanceof Promise ? await result : result;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: false,
|
|
||||||
error: 'deleteData not implemented'
|
|
||||||
};
|
|
||||||
};
|
|
||||||
augmentation.listDataKeys = async (pattern, opts) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.listDataKeys) {
|
|
||||||
const result = options.listDataKeys(pattern, opts);
|
|
||||||
return result instanceof Promise ? await result : result;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: [],
|
|
||||||
error: 'listDataKeys not implemented'
|
|
||||||
};
|
|
||||||
};
|
|
||||||
augmentation.search = async (query, k, opts) => {
|
|
||||||
await augmentation.ensureInitialized();
|
|
||||||
if (options.search) {
|
|
||||||
const result = options.search(query, k, opts);
|
|
||||||
return result instanceof Promise ? await result : result;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: [],
|
|
||||||
error: 'search not implemented'
|
|
||||||
};
|
|
||||||
};
|
|
||||||
// Auto-register if requested
|
|
||||||
if (options.autoRegister) {
|
|
||||||
registerAugmentation(augmentation);
|
|
||||||
// Auto-initialize if requested
|
|
||||||
if (options.autoInitialize) {
|
|
||||||
augmentation.initialize().catch((error) => {
|
|
||||||
console.error(`Failed to initialize augmentation ${augmentation.name}:`, error);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return augmentation;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Factory for creating WebSocket-enabled augmentations
|
|
||||||
* This can be combined with other augmentation factories to create WebSocket-enabled versions
|
|
||||||
*/
|
|
||||||
export function addWebSocketSupport(augmentation, options) {
|
|
||||||
const wsAugmentation = augmentation;
|
|
||||||
// Add WebSocket methods
|
|
||||||
wsAugmentation.connectWebSocket = async (url, protocols) => {
|
|
||||||
await augmentation.ensureInitialized?.();
|
|
||||||
if (options.connectWebSocket) {
|
|
||||||
return options.connectWebSocket(url, protocols);
|
|
||||||
}
|
|
||||||
throw new Error('connectWebSocket not implemented');
|
|
||||||
};
|
|
||||||
wsAugmentation.sendWebSocketMessage = async (connectionId, data) => {
|
|
||||||
await augmentation.ensureInitialized?.();
|
|
||||||
if (options.sendWebSocketMessage) {
|
|
||||||
return options.sendWebSocketMessage(connectionId, data);
|
|
||||||
}
|
|
||||||
throw new Error('sendWebSocketMessage not implemented');
|
|
||||||
};
|
|
||||||
wsAugmentation.onWebSocketMessage = async (connectionId, callback) => {
|
|
||||||
await augmentation.ensureInitialized?.();
|
|
||||||
if (options.onWebSocketMessage) {
|
|
||||||
return options.onWebSocketMessage(connectionId, callback);
|
|
||||||
}
|
|
||||||
throw new Error('onWebSocketMessage not implemented');
|
|
||||||
};
|
|
||||||
wsAugmentation.offWebSocketMessage = async (connectionId, callback) => {
|
|
||||||
await augmentation.ensureInitialized?.();
|
|
||||||
if (options.offWebSocketMessage) {
|
|
||||||
return options.offWebSocketMessage(connectionId, callback);
|
|
||||||
}
|
|
||||||
throw new Error('offWebSocketMessage not implemented');
|
|
||||||
};
|
|
||||||
wsAugmentation.closeWebSocket = async (connectionId, code, reason) => {
|
|
||||||
await augmentation.ensureInitialized?.();
|
|
||||||
if (options.closeWebSocket) {
|
|
||||||
return options.closeWebSocket(connectionId, code, reason);
|
|
||||||
}
|
|
||||||
throw new Error('closeWebSocket not implemented');
|
|
||||||
};
|
|
||||||
return wsAugmentation;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Simplified function to execute an augmentation method with automatic error handling
|
|
||||||
* This provides a more concise way to execute augmentation methods compared to the full pipeline
|
|
||||||
*/
|
|
||||||
export async function executeAugmentation(augmentation, method, ...args) {
|
|
||||||
try {
|
|
||||||
if (!augmentation.enabled) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: `Augmentation ${augmentation.name} is disabled`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
if (typeof augmentation[method] !== 'function') {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: `Method ${method} not found on augmentation ${augmentation.name}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
const result = await augmentation[method](...args);
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error(`Error executing ${method} on ${augmentation.name}:`, error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: error instanceof Error ? error.message : String(error)
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Dynamically load augmentations from a module at runtime
|
|
||||||
* This allows for lazy-loading augmentations when needed instead of at build time
|
|
||||||
*/
|
|
||||||
export async function loadAugmentationModule(modulePromise, options = {}) {
|
|
||||||
try {
|
|
||||||
const module = await modulePromise;
|
|
||||||
const augmentations = [];
|
|
||||||
// Extract augmentations from the module
|
|
||||||
for (const key in module) {
|
|
||||||
const exported = module[key];
|
|
||||||
// Skip non-objects and null
|
|
||||||
if (!exported || typeof exported !== 'object') {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
// Check if it's an augmentation
|
|
||||||
if (typeof exported.name === 'string' &&
|
|
||||||
typeof exported.initialize === 'function' &&
|
|
||||||
typeof exported.shutDown === 'function' &&
|
|
||||||
typeof exported.getStatus === 'function') {
|
|
||||||
augmentations.push(exported);
|
|
||||||
// Auto-register if requested
|
|
||||||
if (options.autoRegister) {
|
|
||||||
registerAugmentation(exported);
|
|
||||||
// Auto-initialize if requested
|
|
||||||
if (options.autoInitialize) {
|
|
||||||
exported.initialize().catch((error) => {
|
|
||||||
console.error(`Failed to initialize augmentation ${exported.name}:`, error);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return augmentations;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error loading augmentation module:', error);
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=augmentationFactory.js.map
|
|
||||||
1
dist/augmentationFactory.js.map
vendored
1
dist/augmentationFactory.js.map
vendored
File diff suppressed because one or more lines are too long
87
dist/augmentationManager.d.ts
vendored
87
dist/augmentationManager.d.ts
vendored
|
|
@ -1,87 +0,0 @@
|
||||||
/**
|
|
||||||
* Type-safe augmentation management system for Brainy
|
|
||||||
* Provides a clean API for managing augmentations without string literals
|
|
||||||
*/
|
|
||||||
import { IAugmentation, AugmentationType } from './types/augmentations.js';
|
|
||||||
export interface AugmentationInfo {
|
|
||||||
name: string;
|
|
||||||
type: string;
|
|
||||||
enabled: boolean;
|
|
||||||
description: string;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Type-safe augmentation manager
|
|
||||||
* Accessed via brain.augmentations for all management operations
|
|
||||||
*/
|
|
||||||
export declare class AugmentationManager {
|
|
||||||
private pipeline;
|
|
||||||
/**
|
|
||||||
* List all registered augmentations with their status
|
|
||||||
* @returns Array of augmentation information
|
|
||||||
*/
|
|
||||||
list(): AugmentationInfo[];
|
|
||||||
/**
|
|
||||||
* Get information about a specific augmentation
|
|
||||||
* @param name The augmentation name
|
|
||||||
* @returns Augmentation info or undefined if not found
|
|
||||||
*/
|
|
||||||
get(name: string): AugmentationInfo | undefined;
|
|
||||||
/**
|
|
||||||
* Check if an augmentation is enabled
|
|
||||||
* @param name The augmentation name
|
|
||||||
* @returns True if enabled, false otherwise
|
|
||||||
*/
|
|
||||||
isEnabled(name: string): boolean;
|
|
||||||
/**
|
|
||||||
* Enable a specific augmentation
|
|
||||||
* @param name The augmentation name
|
|
||||||
* @returns True if successfully enabled
|
|
||||||
*/
|
|
||||||
enable(name: string): boolean;
|
|
||||||
/**
|
|
||||||
* Disable a specific augmentation
|
|
||||||
* @param name The augmentation name
|
|
||||||
* @returns True if successfully disabled
|
|
||||||
*/
|
|
||||||
disable(name: string): boolean;
|
|
||||||
/**
|
|
||||||
* Remove an augmentation from the pipeline
|
|
||||||
* @param name The augmentation name
|
|
||||||
* @returns True if successfully removed
|
|
||||||
*/
|
|
||||||
remove(name: string): boolean;
|
|
||||||
/**
|
|
||||||
* Enable all augmentations of a specific type
|
|
||||||
* @param type The augmentation type
|
|
||||||
* @returns Number of augmentations enabled
|
|
||||||
*/
|
|
||||||
enableType(type: AugmentationType): number;
|
|
||||||
/**
|
|
||||||
* Disable all augmentations of a specific type
|
|
||||||
* @param type The augmentation type
|
|
||||||
* @returns Number of augmentations disabled
|
|
||||||
*/
|
|
||||||
disableType(type: AugmentationType): number;
|
|
||||||
/**
|
|
||||||
* Get all augmentations of a specific type
|
|
||||||
* @param type The augmentation type
|
|
||||||
* @returns Array of augmentations of that type
|
|
||||||
*/
|
|
||||||
listByType(type: AugmentationType): AugmentationInfo[];
|
|
||||||
/**
|
|
||||||
* Get all enabled augmentations
|
|
||||||
* @returns Array of enabled augmentations
|
|
||||||
*/
|
|
||||||
listEnabled(): AugmentationInfo[];
|
|
||||||
/**
|
|
||||||
* Get all disabled augmentations
|
|
||||||
* @returns Array of disabled augmentations
|
|
||||||
*/
|
|
||||||
listDisabled(): AugmentationInfo[];
|
|
||||||
/**
|
|
||||||
* Register a new augmentation (internal use)
|
|
||||||
* @param augmentation The augmentation to register
|
|
||||||
*/
|
|
||||||
register(augmentation: IAugmentation): void;
|
|
||||||
}
|
|
||||||
export { AugmentationType } from './types/augmentations.js';
|
|
||||||
112
dist/augmentationManager.js
vendored
112
dist/augmentationManager.js
vendored
|
|
@ -1,112 +0,0 @@
|
||||||
/**
|
|
||||||
* Type-safe augmentation management system for Brainy
|
|
||||||
* Provides a clean API for managing augmentations without string literals
|
|
||||||
*/
|
|
||||||
import { augmentationPipeline } from './augmentationPipeline.js';
|
|
||||||
/**
|
|
||||||
* Type-safe augmentation manager
|
|
||||||
* Accessed via brain.augmentations for all management operations
|
|
||||||
*/
|
|
||||||
export class AugmentationManager {
|
|
||||||
constructor() {
|
|
||||||
this.pipeline = augmentationPipeline;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* List all registered augmentations with their status
|
|
||||||
* @returns Array of augmentation information
|
|
||||||
*/
|
|
||||||
list() {
|
|
||||||
return this.pipeline.listAugmentationsWithStatus();
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get information about a specific augmentation
|
|
||||||
* @param name The augmentation name
|
|
||||||
* @returns Augmentation info or undefined if not found
|
|
||||||
*/
|
|
||||||
get(name) {
|
|
||||||
const all = this.list();
|
|
||||||
return all.find(a => a.name === name);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Check if an augmentation is enabled
|
|
||||||
* @param name The augmentation name
|
|
||||||
* @returns True if enabled, false otherwise
|
|
||||||
*/
|
|
||||||
isEnabled(name) {
|
|
||||||
const aug = this.get(name);
|
|
||||||
return aug?.enabled ?? false;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Enable a specific augmentation
|
|
||||||
* @param name The augmentation name
|
|
||||||
* @returns True if successfully enabled
|
|
||||||
*/
|
|
||||||
enable(name) {
|
|
||||||
return this.pipeline.enableAugmentation(name);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Disable a specific augmentation
|
|
||||||
* @param name The augmentation name
|
|
||||||
* @returns True if successfully disabled
|
|
||||||
*/
|
|
||||||
disable(name) {
|
|
||||||
return this.pipeline.disableAugmentation(name);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Remove an augmentation from the pipeline
|
|
||||||
* @param name The augmentation name
|
|
||||||
* @returns True if successfully removed
|
|
||||||
*/
|
|
||||||
remove(name) {
|
|
||||||
this.pipeline.unregister(name);
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Enable all augmentations of a specific type
|
|
||||||
* @param type The augmentation type
|
|
||||||
* @returns Number of augmentations enabled
|
|
||||||
*/
|
|
||||||
enableType(type) {
|
|
||||||
return this.pipeline.enableAugmentationType(type);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Disable all augmentations of a specific type
|
|
||||||
* @param type The augmentation type
|
|
||||||
* @returns Number of augmentations disabled
|
|
||||||
*/
|
|
||||||
disableType(type) {
|
|
||||||
return this.pipeline.disableAugmentationType(type);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get all augmentations of a specific type
|
|
||||||
* @param type The augmentation type
|
|
||||||
* @returns Array of augmentations of that type
|
|
||||||
*/
|
|
||||||
listByType(type) {
|
|
||||||
return this.list().filter(a => a.type === type);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get all enabled augmentations
|
|
||||||
* @returns Array of enabled augmentations
|
|
||||||
*/
|
|
||||||
listEnabled() {
|
|
||||||
return this.list().filter(a => a.enabled);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get all disabled augmentations
|
|
||||||
* @returns Array of disabled augmentations
|
|
||||||
*/
|
|
||||||
listDisabled() {
|
|
||||||
return this.list().filter(a => !a.enabled);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Register a new augmentation (internal use)
|
|
||||||
* @param augmentation The augmentation to register
|
|
||||||
*/
|
|
||||||
register(augmentation) {
|
|
||||||
this.pipeline.register(augmentation);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Export types for external use
|
|
||||||
export { AugmentationType } from './types/augmentations.js';
|
|
||||||
//# sourceMappingURL=augmentationManager.js.map
|
|
||||||
1
dist/augmentationManager.js.map
vendored
1
dist/augmentationManager.js.map
vendored
|
|
@ -1 +0,0 @@
|
||||||
{"version":3,"file":"augmentationManager.js","sourceRoot":"","sources":["../src/augmentationManager.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAGH,OAAO,EAAE,oBAAoB,EAAE,MAAM,2BAA2B,CAAA;AAShE;;;GAGG;AACH,MAAM,OAAO,mBAAmB;IAAhC;QACU,aAAQ,GAAG,oBAAoB,CAAA;IA4GzC,CAAC;IA1GC;;;OAGG;IACH,IAAI;QACF,OAAO,IAAI,CAAC,QAAQ,CAAC,2BAA2B,EAAE,CAAA;IACpD,CAAC;IAED;;;;OAIG;IACH,GAAG,CAAC,IAAY;QACd,MAAM,GAAG,GAAG,IAAI,CAAC,IAAI,EAAE,CAAA;QACvB,OAAO,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,IAAI,CAAC,CAAA;IACvC,CAAC;IAED;;;;OAIG;IACH,SAAS,CAAC,IAAY;QACpB,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,CAAA;QAC1B,OAAO,GAAG,EAAE,OAAO,IAAI,KAAK,CAAA;IAC9B,CAAC;IAED;;;;OAIG;IACH,MAAM,CAAC,IAAY;QACjB,OAAO,IAAI,CAAC,QAAQ,CAAC,kBAAkB,CAAC,IAAI,CAAC,CAAA;IAC/C,CAAC;IAED;;;;OAIG;IACH,OAAO,CAAC,IAAY;QAClB,OAAO,IAAI,CAAC,QAAQ,CAAC,mBAAmB,CAAC,IAAI,CAAC,CAAA;IAChD,CAAC;IAED;;;;OAIG;IACH,MAAM,CAAC,IAAY;QACjB,IAAI,CAAC,QAAQ,CAAC,UAAU,CAAC,IAAI,CAAC,CAAA;QAC9B,OAAO,IAAI,CAAA;IACb,CAAC;IAED;;;;OAIG;IACH,UAAU,CAAC,IAAsB;QAC/B,OAAO,IAAI,CAAC,QAAQ,CAAC,sBAAsB,CAAC,IAAW,CAAC,CAAA;IAC1D,CAAC;IAED;;;;OAIG;IACH,WAAW,CAAC,IAAsB;QAChC,OAAO,IAAI,CAAC,QAAQ,CAAC,uBAAuB,CAAC,IAAW,CAAC,CAAA;IAC3D,CAAC;IAED;;;;OAIG;IACH,UAAU,CAAC,IAAsB;QAC/B,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,IAAI,CAAC,CAAA;IACjD,CAAC;IAED;;;OAGG;IACH,WAAW;QACT,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,CAAA;IAC3C,CAAC;IAED;;;OAGG;IACH,YAAY;QACV,OAAO,IAAI,CAAC,IAAI,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAA;IAC5C,CAAC;IAED;;;OAGG;IACH,QAAQ,CAAC,YAA2B;QAClC,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,YAAY,CAAC,CAAA;IACtC,CAAC;CACF;AAED,gCAAgC;AAChC,OAAO,EAAE,gBAAgB,EAAE,MAAM,0BAA0B,CAAA"}
|
|
||||||
271
dist/augmentationPipeline.d.ts
vendored
271
dist/augmentationPipeline.d.ts
vendored
|
|
@ -1,271 +0,0 @@
|
||||||
/**
|
|
||||||
* Cortex - The Brain's Orchestration System
|
|
||||||
*
|
|
||||||
* 🧠⚛️ The cerebral cortex that coordinates all augmentations
|
|
||||||
*
|
|
||||||
* This module provides the central coordination system for managing and executing
|
|
||||||
* augmentations across all categories. Like the brain's cortex, it orchestrates
|
|
||||||
* different capabilities (augmentations) in sequence or parallel.
|
|
||||||
*
|
|
||||||
* @deprecated AugmentationPipeline - Use Cortex instead
|
|
||||||
*/
|
|
||||||
import { BrainyAugmentations, IAugmentation, IWebSocketSupport, AugmentationResponse, AugmentationType } from './types/augmentations.js';
|
|
||||||
/**
|
|
||||||
* Type definitions for the augmentation registry
|
|
||||||
*/
|
|
||||||
type AugmentationRegistry = {
|
|
||||||
sense: BrainyAugmentations.ISenseAugmentation[];
|
|
||||||
conduit: BrainyAugmentations.IConduitAugmentation[];
|
|
||||||
cognition: BrainyAugmentations.ICognitionAugmentation[];
|
|
||||||
memory: BrainyAugmentations.IMemoryAugmentation[];
|
|
||||||
perception: BrainyAugmentations.IPerceptionAugmentation[];
|
|
||||||
dialog: BrainyAugmentations.IDialogAugmentation[];
|
|
||||||
activation: BrainyAugmentations.IActivationAugmentation[];
|
|
||||||
webSocket: IWebSocketSupport[];
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Execution mode for the pipeline
|
|
||||||
*/
|
|
||||||
export declare enum ExecutionMode {
|
|
||||||
SEQUENTIAL = "sequential",
|
|
||||||
PARALLEL = "parallel",
|
|
||||||
FIRST_SUCCESS = "firstSuccess",
|
|
||||||
FIRST_RESULT = "firstResult",
|
|
||||||
THREADED = "threaded"
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Options for pipeline execution
|
|
||||||
*/
|
|
||||||
export interface PipelineOptions {
|
|
||||||
mode?: ExecutionMode;
|
|
||||||
timeout?: number;
|
|
||||||
stopOnError?: boolean;
|
|
||||||
forceThreading?: boolean;
|
|
||||||
disableThreading?: boolean;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Cortex class - The Brain's Orchestration Center
|
|
||||||
*
|
|
||||||
* Manages all augmentations like the cerebral cortex coordinates different brain regions.
|
|
||||||
* This is the central pipeline that orchestrates all augmentation execution.
|
|
||||||
*/
|
|
||||||
export declare class Cortex {
|
|
||||||
private registry;
|
|
||||||
/**
|
|
||||||
* Register an augmentation with the cortex
|
|
||||||
*
|
|
||||||
* @param augmentation The augmentation to register
|
|
||||||
* @returns The cortex instance for chaining
|
|
||||||
*/
|
|
||||||
register<T extends IAugmentation>(augmentation: T): Cortex;
|
|
||||||
/**
|
|
||||||
* Unregister an augmentation from the pipeline
|
|
||||||
*
|
|
||||||
* @param augmentationName The name of the augmentation to unregister
|
|
||||||
* @returns The pipeline instance for chaining
|
|
||||||
*/
|
|
||||||
unregister(augmentationName: string): Cortex;
|
|
||||||
/**
|
|
||||||
* Initialize all registered augmentations
|
|
||||||
*
|
|
||||||
* @returns A promise that resolves when all augmentations are initialized
|
|
||||||
*/
|
|
||||||
initialize(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Shut down all registered augmentations
|
|
||||||
*
|
|
||||||
* @returns A promise that resolves when all augmentations are shut down
|
|
||||||
*/
|
|
||||||
shutDown(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Execute a sense pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each sense augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
executeSensePipeline<M extends keyof BrainyAugmentations.ISenseAugmentation & string, R extends BrainyAugmentations.ISenseAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never>(method: M & (BrainyAugmentations.ISenseAugmentation[M] extends (...args: any[]) => any ? M : never), args: Parameters<Extract<BrainyAugmentations.ISenseAugmentation[M], (...args: any[]) => any>>, options?: PipelineOptions): Promise<Promise<{
|
|
||||||
success: boolean;
|
|
||||||
data: R;
|
|
||||||
error?: string;
|
|
||||||
}>[]>;
|
|
||||||
/**
|
|
||||||
* Execute a conduit pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each conduit augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
executeConduitPipeline<M extends keyof BrainyAugmentations.IConduitAugmentation & string, R extends BrainyAugmentations.IConduitAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never>(method: M & (BrainyAugmentations.IConduitAugmentation[M] extends (...args: any[]) => any ? M : never), args: Parameters<Extract<BrainyAugmentations.IConduitAugmentation[M], (...args: any[]) => any>>, options?: PipelineOptions): Promise<Promise<{
|
|
||||||
success: boolean;
|
|
||||||
data: R;
|
|
||||||
error?: string;
|
|
||||||
}>[]>;
|
|
||||||
/**
|
|
||||||
* Execute a cognition pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each cognition augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
executeCognitionPipeline<M extends keyof BrainyAugmentations.ICognitionAugmentation & string, R extends BrainyAugmentations.ICognitionAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never>(method: M & (BrainyAugmentations.ICognitionAugmentation[M] extends (...args: any[]) => any ? M : never), args: Parameters<Extract<BrainyAugmentations.ICognitionAugmentation[M], (...args: any[]) => any>>, options?: PipelineOptions): Promise<Promise<{
|
|
||||||
success: boolean;
|
|
||||||
data: R;
|
|
||||||
error?: string;
|
|
||||||
}>[]>;
|
|
||||||
/**
|
|
||||||
* Execute a memory pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each memory augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
executeMemoryPipeline<M extends keyof BrainyAugmentations.IMemoryAugmentation & string, R extends BrainyAugmentations.IMemoryAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never>(method: M & (BrainyAugmentations.IMemoryAugmentation[M] extends (...args: any[]) => any ? M : never), args: Parameters<Extract<BrainyAugmentations.IMemoryAugmentation[M], (...args: any[]) => any>>, options?: PipelineOptions): Promise<Promise<{
|
|
||||||
success: boolean;
|
|
||||||
data: R;
|
|
||||||
error?: string;
|
|
||||||
}>[]>;
|
|
||||||
/**
|
|
||||||
* Execute a perception pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each perception augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
executePerceptionPipeline<M extends keyof BrainyAugmentations.IPerceptionAugmentation & string, R extends BrainyAugmentations.IPerceptionAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never>(method: M & (BrainyAugmentations.IPerceptionAugmentation[M] extends (...args: any[]) => any ? M : never), args: Parameters<Extract<BrainyAugmentations.IPerceptionAugmentation[M], (...args: any[]) => any>>, options?: PipelineOptions): Promise<Promise<{
|
|
||||||
success: boolean;
|
|
||||||
data: R;
|
|
||||||
error?: string;
|
|
||||||
}>[]>;
|
|
||||||
/**
|
|
||||||
* Execute a dialog pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each dialog augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
executeDialogPipeline<M extends keyof BrainyAugmentations.IDialogAugmentation & string, R extends BrainyAugmentations.IDialogAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never>(method: M & (BrainyAugmentations.IDialogAugmentation[M] extends (...args: any[]) => any ? M : never), args: Parameters<Extract<BrainyAugmentations.IDialogAugmentation[M], (...args: any[]) => any>>, options?: PipelineOptions): Promise<Promise<{
|
|
||||||
success: boolean;
|
|
||||||
data: R;
|
|
||||||
error?: string;
|
|
||||||
}>[]>;
|
|
||||||
/**
|
|
||||||
* Execute an activation pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each activation augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
executeActivationPipeline<M extends keyof BrainyAugmentations.IActivationAugmentation & string, R extends BrainyAugmentations.IActivationAugmentation[M] extends (...args: any[]) => AugmentationResponse<infer U> ? U : never>(method: M & (BrainyAugmentations.IActivationAugmentation[M] extends (...args: any[]) => any ? M : never), args: Parameters<Extract<BrainyAugmentations.IActivationAugmentation[M], (...args: any[]) => any>>, options?: PipelineOptions): Promise<Promise<{
|
|
||||||
success: boolean;
|
|
||||||
data: R;
|
|
||||||
error?: string;
|
|
||||||
}>[]>;
|
|
||||||
/**
|
|
||||||
* Get all registered augmentations
|
|
||||||
*
|
|
||||||
* @returns An array of all registered augmentations
|
|
||||||
*/
|
|
||||||
getAllAugmentations(): IAugmentation[];
|
|
||||||
/**
|
|
||||||
* Get all augmentations of a specific type
|
|
||||||
*
|
|
||||||
* @param type The type of augmentation to get
|
|
||||||
* @returns An array of all augmentations of the specified type
|
|
||||||
*/
|
|
||||||
getAugmentationsByType(type: AugmentationType): IAugmentation[];
|
|
||||||
/**
|
|
||||||
* Get all available augmentation types
|
|
||||||
*
|
|
||||||
* @returns An array of all augmentation types that have at least one registered augmentation
|
|
||||||
*/
|
|
||||||
getAvailableAugmentationTypes(): AugmentationType[];
|
|
||||||
/**
|
|
||||||
* Get all WebSocket-supporting augmentations
|
|
||||||
*
|
|
||||||
* @returns An array of all augmentations that support WebSocket connections
|
|
||||||
*/
|
|
||||||
getWebSocketAugmentations(): IWebSocketSupport[];
|
|
||||||
/**
|
|
||||||
* Check if an augmentation is of a specific type
|
|
||||||
*
|
|
||||||
* @param augmentation The augmentation to check
|
|
||||||
* @param methods The methods that should be present on the augmentation
|
|
||||||
* @returns True if the augmentation is of the specified type
|
|
||||||
*/
|
|
||||||
private isAugmentationType;
|
|
||||||
/**
|
|
||||||
* Determines if threading should be used based on options and environment
|
|
||||||
*
|
|
||||||
* @param options The pipeline options
|
|
||||||
* @returns True if threading should be used, false otherwise
|
|
||||||
*/
|
|
||||||
private shouldUseThreading;
|
|
||||||
/**
|
|
||||||
* Execute a pipeline for a specific augmentation type
|
|
||||||
*
|
|
||||||
* @param augmentations The augmentations to execute
|
|
||||||
* @param method The method to execute on each augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
private executeTypedPipeline;
|
|
||||||
/**
|
|
||||||
* Enable an augmentation by name
|
|
||||||
*
|
|
||||||
* @param name The name of the augmentation to enable
|
|
||||||
* @returns True if augmentation was found and enabled
|
|
||||||
*/
|
|
||||||
enableAugmentation(name: string): boolean;
|
|
||||||
/**
|
|
||||||
* Disable an augmentation by name
|
|
||||||
*
|
|
||||||
* @param name The name of the augmentation to disable
|
|
||||||
* @returns True if augmentation was found and disabled
|
|
||||||
*/
|
|
||||||
disableAugmentation(name: string): boolean;
|
|
||||||
/**
|
|
||||||
* Check if an augmentation is enabled
|
|
||||||
*
|
|
||||||
* @param name The name of the augmentation to check
|
|
||||||
* @returns True if augmentation is found and enabled, false otherwise
|
|
||||||
*/
|
|
||||||
isAugmentationEnabled(name: string): boolean;
|
|
||||||
/**
|
|
||||||
* Get all augmentations with their enabled status
|
|
||||||
*
|
|
||||||
* @returns Array of augmentations with name, type, and enabled status
|
|
||||||
*/
|
|
||||||
listAugmentationsWithStatus(): Array<{
|
|
||||||
name: string;
|
|
||||||
type: keyof AugmentationRegistry;
|
|
||||||
enabled: boolean;
|
|
||||||
description: string;
|
|
||||||
}>;
|
|
||||||
/**
|
|
||||||
* Enable all augmentations of a specific type
|
|
||||||
*
|
|
||||||
* @param type The type of augmentations to enable
|
|
||||||
* @returns Number of augmentations enabled
|
|
||||||
*/
|
|
||||||
enableAugmentationType(type: keyof AugmentationRegistry): number;
|
|
||||||
/**
|
|
||||||
* Disable all augmentations of a specific type
|
|
||||||
*
|
|
||||||
* @param type The type of augmentations to disable
|
|
||||||
* @returns Number of augmentations disabled
|
|
||||||
*/
|
|
||||||
disableAugmentationType(type: keyof AugmentationRegistry): number;
|
|
||||||
}
|
|
||||||
export declare const cortex: Cortex;
|
|
||||||
export declare const AugmentationPipeline: typeof Cortex;
|
|
||||||
export declare const augmentationPipeline: Cortex;
|
|
||||||
export {};
|
|
||||||
574
dist/augmentationPipeline.js
vendored
574
dist/augmentationPipeline.js
vendored
|
|
@ -1,574 +0,0 @@
|
||||||
/**
|
|
||||||
* Cortex - The Brain's Orchestration System
|
|
||||||
*
|
|
||||||
* 🧠⚛️ The cerebral cortex that coordinates all augmentations
|
|
||||||
*
|
|
||||||
* This module provides the central coordination system for managing and executing
|
|
||||||
* augmentations across all categories. Like the brain's cortex, it orchestrates
|
|
||||||
* different capabilities (augmentations) in sequence or parallel.
|
|
||||||
*
|
|
||||||
* @deprecated AugmentationPipeline - Use Cortex instead
|
|
||||||
*/
|
|
||||||
import { AugmentationType } from './types/augmentations.js';
|
|
||||||
import { isThreadingAvailable } from './utils/environment.js';
|
|
||||||
import { executeInThread } from './utils/workerUtils.js';
|
|
||||||
/**
|
|
||||||
* Execution mode for the pipeline
|
|
||||||
*/
|
|
||||||
export var ExecutionMode;
|
|
||||||
(function (ExecutionMode) {
|
|
||||||
ExecutionMode["SEQUENTIAL"] = "sequential";
|
|
||||||
ExecutionMode["PARALLEL"] = "parallel";
|
|
||||||
ExecutionMode["FIRST_SUCCESS"] = "firstSuccess";
|
|
||||||
ExecutionMode["FIRST_RESULT"] = "firstResult";
|
|
||||||
ExecutionMode["THREADED"] = "threaded"; // Execute in separate threads when available
|
|
||||||
})(ExecutionMode || (ExecutionMode = {}));
|
|
||||||
/**
|
|
||||||
* Default pipeline options
|
|
||||||
*/
|
|
||||||
const DEFAULT_PIPELINE_OPTIONS = {
|
|
||||||
mode: ExecutionMode.SEQUENTIAL,
|
|
||||||
timeout: 30000,
|
|
||||||
stopOnError: false,
|
|
||||||
forceThreading: false,
|
|
||||||
disableThreading: false
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Cortex class - The Brain's Orchestration Center
|
|
||||||
*
|
|
||||||
* Manages all augmentations like the cerebral cortex coordinates different brain regions.
|
|
||||||
* This is the central pipeline that orchestrates all augmentation execution.
|
|
||||||
*/
|
|
||||||
export class Cortex {
|
|
||||||
constructor() {
|
|
||||||
this.registry = {
|
|
||||||
sense: [],
|
|
||||||
conduit: [],
|
|
||||||
cognition: [],
|
|
||||||
memory: [],
|
|
||||||
perception: [],
|
|
||||||
dialog: [],
|
|
||||||
activation: [],
|
|
||||||
webSocket: []
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Register an augmentation with the cortex
|
|
||||||
*
|
|
||||||
* @param augmentation The augmentation to register
|
|
||||||
* @returns The cortex instance for chaining
|
|
||||||
*/
|
|
||||||
register(augmentation) {
|
|
||||||
let registered = false;
|
|
||||||
// Check for specific augmentation types
|
|
||||||
if (this.isAugmentationType(augmentation, 'processRawData', 'listenToFeed')) {
|
|
||||||
this.registry.sense.push(augmentation);
|
|
||||||
registered = true;
|
|
||||||
}
|
|
||||||
else if (this.isAugmentationType(augmentation, 'establishConnection', 'readData', 'writeData', 'monitorStream')) {
|
|
||||||
this.registry.conduit.push(augmentation);
|
|
||||||
registered = true;
|
|
||||||
}
|
|
||||||
else if (this.isAugmentationType(augmentation, 'reason', 'infer', 'executeLogic')) {
|
|
||||||
this.registry.cognition.push(augmentation);
|
|
||||||
registered = true;
|
|
||||||
}
|
|
||||||
else if (this.isAugmentationType(augmentation, 'storeData', 'retrieveData', 'updateData', 'deleteData', 'listDataKeys')) {
|
|
||||||
this.registry.memory.push(augmentation);
|
|
||||||
registered = true;
|
|
||||||
}
|
|
||||||
else if (this.isAugmentationType(augmentation, 'interpret', 'organize', 'generateVisualization')) {
|
|
||||||
this.registry.perception.push(augmentation);
|
|
||||||
registered = true;
|
|
||||||
}
|
|
||||||
else if (this.isAugmentationType(augmentation, 'processUserInput', 'generateResponse', 'manageContext')) {
|
|
||||||
this.registry.dialog.push(augmentation);
|
|
||||||
registered = true;
|
|
||||||
}
|
|
||||||
else if (this.isAugmentationType(augmentation, 'triggerAction', 'generateOutput', 'interactExternal')) {
|
|
||||||
this.registry.activation.push(augmentation);
|
|
||||||
registered = true;
|
|
||||||
}
|
|
||||||
// Check if the augmentation supports WebSocket
|
|
||||||
if (this.isAugmentationType(augmentation, 'connectWebSocket', 'sendWebSocketMessage', 'onWebSocketMessage', 'closeWebSocket')) {
|
|
||||||
this.registry.webSocket.push(augmentation);
|
|
||||||
registered = true;
|
|
||||||
}
|
|
||||||
// If the augmentation wasn't registered as any known type, throw an error
|
|
||||||
if (!registered) {
|
|
||||||
throw new Error(`Unknown augmentation type: ${augmentation.name}`);
|
|
||||||
}
|
|
||||||
return this;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Unregister an augmentation from the pipeline
|
|
||||||
*
|
|
||||||
* @param augmentationName The name of the augmentation to unregister
|
|
||||||
* @returns The pipeline instance for chaining
|
|
||||||
*/
|
|
||||||
unregister(augmentationName) {
|
|
||||||
let found = false;
|
|
||||||
// Remove from all registries
|
|
||||||
for (const type in this.registry) {
|
|
||||||
const typedRegistry = this.registry[type];
|
|
||||||
const index = typedRegistry.findIndex((aug) => aug.name === augmentationName);
|
|
||||||
if (index !== -1) {
|
|
||||||
typedRegistry.splice(index, 1);
|
|
||||||
found = true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return this;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Initialize all registered augmentations
|
|
||||||
*
|
|
||||||
* @returns A promise that resolves when all augmentations are initialized
|
|
||||||
*/
|
|
||||||
async initialize() {
|
|
||||||
const allAugmentations = this.getAllAugmentations();
|
|
||||||
await Promise.all(allAugmentations.map((augmentation) => augmentation.initialize().catch((error) => {
|
|
||||||
console.error(`Failed to initialize augmentation ${augmentation.name}:`, error);
|
|
||||||
})));
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Shut down all registered augmentations
|
|
||||||
*
|
|
||||||
* @returns A promise that resolves when all augmentations are shut down
|
|
||||||
*/
|
|
||||||
async shutDown() {
|
|
||||||
const allAugmentations = this.getAllAugmentations();
|
|
||||||
await Promise.all(allAugmentations.map((augmentation) => augmentation.shutDown().catch((error) => {
|
|
||||||
console.error(`Failed to shut down augmentation ${augmentation.name}:`, error);
|
|
||||||
})));
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Execute a sense pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each sense augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
async executeSensePipeline(method, args, options = {}) {
|
|
||||||
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options };
|
|
||||||
return this.executeTypedPipeline(this.registry.sense, method, args, opts);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Execute a conduit pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each conduit augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
async executeConduitPipeline(method, args, options = {}) {
|
|
||||||
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options };
|
|
||||||
return this.executeTypedPipeline(this.registry.conduit, method, args, opts);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Execute a cognition pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each cognition augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
async executeCognitionPipeline(method, args, options = {}) {
|
|
||||||
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options };
|
|
||||||
return this.executeTypedPipeline(this.registry.cognition, method, args, opts);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Execute a memory pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each memory augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
async executeMemoryPipeline(method, args, options = {}) {
|
|
||||||
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options };
|
|
||||||
return this.executeTypedPipeline(this.registry.memory, method, args, opts);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Execute a perception pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each perception augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
async executePerceptionPipeline(method, args, options = {}) {
|
|
||||||
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options };
|
|
||||||
return this.executeTypedPipeline(this.registry.perception, method, args, opts);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Execute a dialog pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each dialog augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
async executeDialogPipeline(method, args, options = {}) {
|
|
||||||
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options };
|
|
||||||
return this.executeTypedPipeline(this.registry.dialog, method, args, opts);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Execute an activation pipeline
|
|
||||||
*
|
|
||||||
* @param method The method to execute on each activation augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
async executeActivationPipeline(method, args, options = {}) {
|
|
||||||
const opts = { ...DEFAULT_PIPELINE_OPTIONS, ...options };
|
|
||||||
return this.executeTypedPipeline(this.registry.activation, method, args, opts);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get all registered augmentations
|
|
||||||
*
|
|
||||||
* @returns An array of all registered augmentations
|
|
||||||
*/
|
|
||||||
getAllAugmentations() {
|
|
||||||
// Create a Set to avoid duplicates (an augmentation might be in multiple registries)
|
|
||||||
const allAugmentations = new Set([
|
|
||||||
...this.registry.sense,
|
|
||||||
...this.registry.conduit,
|
|
||||||
...this.registry.cognition,
|
|
||||||
...this.registry.memory,
|
|
||||||
...this.registry.perception,
|
|
||||||
...this.registry.dialog,
|
|
||||||
...this.registry.activation,
|
|
||||||
...this.registry.webSocket
|
|
||||||
]);
|
|
||||||
// Convert back to array
|
|
||||||
return Array.from(allAugmentations);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get all augmentations of a specific type
|
|
||||||
*
|
|
||||||
* @param type The type of augmentation to get
|
|
||||||
* @returns An array of all augmentations of the specified type
|
|
||||||
*/
|
|
||||||
getAugmentationsByType(type) {
|
|
||||||
switch (type) {
|
|
||||||
case AugmentationType.SENSE:
|
|
||||||
return [...this.registry.sense];
|
|
||||||
case AugmentationType.CONDUIT:
|
|
||||||
return [...this.registry.conduit];
|
|
||||||
case AugmentationType.COGNITION:
|
|
||||||
return [...this.registry.cognition];
|
|
||||||
case AugmentationType.MEMORY:
|
|
||||||
return [...this.registry.memory];
|
|
||||||
case AugmentationType.PERCEPTION:
|
|
||||||
return [...this.registry.perception];
|
|
||||||
case AugmentationType.DIALOG:
|
|
||||||
return [...this.registry.dialog];
|
|
||||||
case AugmentationType.ACTIVATION:
|
|
||||||
return [...this.registry.activation];
|
|
||||||
case AugmentationType.WEBSOCKET:
|
|
||||||
return [...this.registry.webSocket];
|
|
||||||
default:
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get all available augmentation types
|
|
||||||
*
|
|
||||||
* @returns An array of all augmentation types that have at least one registered augmentation
|
|
||||||
*/
|
|
||||||
getAvailableAugmentationTypes() {
|
|
||||||
const availableTypes = [];
|
|
||||||
if (this.registry.sense.length > 0)
|
|
||||||
availableTypes.push(AugmentationType.SENSE);
|
|
||||||
if (this.registry.conduit.length > 0)
|
|
||||||
availableTypes.push(AugmentationType.CONDUIT);
|
|
||||||
if (this.registry.cognition.length > 0)
|
|
||||||
availableTypes.push(AugmentationType.COGNITION);
|
|
||||||
if (this.registry.memory.length > 0)
|
|
||||||
availableTypes.push(AugmentationType.MEMORY);
|
|
||||||
if (this.registry.perception.length > 0)
|
|
||||||
availableTypes.push(AugmentationType.PERCEPTION);
|
|
||||||
if (this.registry.dialog.length > 0)
|
|
||||||
availableTypes.push(AugmentationType.DIALOG);
|
|
||||||
if (this.registry.activation.length > 0)
|
|
||||||
availableTypes.push(AugmentationType.ACTIVATION);
|
|
||||||
if (this.registry.webSocket.length > 0)
|
|
||||||
availableTypes.push(AugmentationType.WEBSOCKET);
|
|
||||||
return availableTypes;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get all WebSocket-supporting augmentations
|
|
||||||
*
|
|
||||||
* @returns An array of all augmentations that support WebSocket connections
|
|
||||||
*/
|
|
||||||
getWebSocketAugmentations() {
|
|
||||||
return [...this.registry.webSocket];
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Check if an augmentation is of a specific type
|
|
||||||
*
|
|
||||||
* @param augmentation The augmentation to check
|
|
||||||
* @param methods The methods that should be present on the augmentation
|
|
||||||
* @returns True if the augmentation is of the specified type
|
|
||||||
*/
|
|
||||||
isAugmentationType(augmentation, ...methods) {
|
|
||||||
// First check that the augmentation has all the required base methods
|
|
||||||
const baseMethodsExist = ['initialize', 'shutDown', 'getStatus'].every((method) => typeof augmentation[method] === 'function');
|
|
||||||
if (!baseMethodsExist) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
// Then check that it has all the specific methods for this type
|
|
||||||
return methods.every((method) => typeof augmentation[method] === 'function');
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Determines if threading should be used based on options and environment
|
|
||||||
*
|
|
||||||
* @param options The pipeline options
|
|
||||||
* @returns True if threading should be used, false otherwise
|
|
||||||
*/
|
|
||||||
shouldUseThreading(options) {
|
|
||||||
// If threading is explicitly disabled, don't use it
|
|
||||||
if (options.disableThreading) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
// If threading is explicitly forced, use it if available
|
|
||||||
if (options.forceThreading) {
|
|
||||||
return isThreadingAvailable();
|
|
||||||
}
|
|
||||||
// If in THREADED mode, use threading if available
|
|
||||||
if (options.mode === ExecutionMode.THREADED) {
|
|
||||||
return isThreadingAvailable();
|
|
||||||
}
|
|
||||||
// Otherwise, don't use threading
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Execute a pipeline for a specific augmentation type
|
|
||||||
*
|
|
||||||
* @param augmentations The augmentations to execute
|
|
||||||
* @param method The method to execute on each augmentation
|
|
||||||
* @param args The arguments to pass to the method
|
|
||||||
* @param options The pipeline execution options
|
|
||||||
* @returns A promise that resolves with the results from all augmentations
|
|
||||||
*/
|
|
||||||
async executeTypedPipeline(augmentations, method, args, options) {
|
|
||||||
// Filter out disabled augmentations
|
|
||||||
const enabledAugmentations = augmentations.filter((aug) => aug.enabled !== false);
|
|
||||||
if (enabledAugmentations.length === 0) {
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
// Create a function to execute the method on an augmentation
|
|
||||||
const executeMethod = async (augmentation) => {
|
|
||||||
try {
|
|
||||||
// Create a timeout promise if a timeout is specified
|
|
||||||
const timeoutPromise = options.timeout
|
|
||||||
? new Promise((_, reject) => {
|
|
||||||
setTimeout(() => {
|
|
||||||
reject(new Error(`Timeout executing ${String(method)} on ${augmentation.name}`));
|
|
||||||
}, options.timeout);
|
|
||||||
})
|
|
||||||
: null;
|
|
||||||
// Check if threading should be used
|
|
||||||
const useThreading = this.shouldUseThreading(options);
|
|
||||||
// Execute the method on the augmentation, using threading if appropriate
|
|
||||||
let methodPromise;
|
|
||||||
if (useThreading) {
|
|
||||||
// Execute in a separate thread
|
|
||||||
try {
|
|
||||||
// Create a function that can be serialized and executed in a worker
|
|
||||||
const workerFn = (...workerArgs) => {
|
|
||||||
// This function will be stringified and executed in the worker
|
|
||||||
// It needs to be self-contained
|
|
||||||
const augFn = augmentation[method];
|
|
||||||
return augFn.apply(augmentation, workerArgs);
|
|
||||||
};
|
|
||||||
methodPromise = executeInThread(workerFn.toString(), args);
|
|
||||||
}
|
|
||||||
catch (threadError) {
|
|
||||||
console.warn(`Failed to execute in thread, falling back to main thread: ${threadError}`);
|
|
||||||
// Fall back to executing in the main thread
|
|
||||||
methodPromise = Promise.resolve(augmentation[method](...args));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
// Execute in the main thread
|
|
||||||
methodPromise = Promise.resolve(augmentation[method](...args));
|
|
||||||
}
|
|
||||||
// Race the method promise against the timeout promise if a timeout is specified
|
|
||||||
const result = timeoutPromise
|
|
||||||
? await Promise.race([methodPromise, timeoutPromise])
|
|
||||||
: await methodPromise;
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error(`Error executing ${String(method)} on ${augmentation.name}:`, error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: error instanceof Error ? error.message : String(error)
|
|
||||||
};
|
|
||||||
}
|
|
||||||
};
|
|
||||||
// Execute the pipeline based on the specified mode
|
|
||||||
switch (options.mode) {
|
|
||||||
case ExecutionMode.PARALLEL:
|
|
||||||
// Execute all augmentations in parallel
|
|
||||||
return enabledAugmentations.map(executeMethod);
|
|
||||||
case ExecutionMode.THREADED:
|
|
||||||
// Execute all augmentations in parallel with threading enabled
|
|
||||||
// Force threading for this mode
|
|
||||||
const threadedOptions = { ...options, forceThreading: true };
|
|
||||||
// Create a new executeMethod function that uses the threaded options
|
|
||||||
const executeMethodThreaded = async (augmentation) => {
|
|
||||||
// Save the original options
|
|
||||||
const originalOptions = options;
|
|
||||||
// Set the options to the threaded options
|
|
||||||
options = threadedOptions;
|
|
||||||
// Execute the method
|
|
||||||
const result = await executeMethod(augmentation);
|
|
||||||
// Restore the original options
|
|
||||||
options = originalOptions;
|
|
||||||
return result;
|
|
||||||
};
|
|
||||||
return enabledAugmentations.map(executeMethodThreaded);
|
|
||||||
case ExecutionMode.FIRST_SUCCESS:
|
|
||||||
// Execute augmentations sequentially until one succeeds
|
|
||||||
for (const augmentation of enabledAugmentations) {
|
|
||||||
const resultPromise = executeMethod(augmentation);
|
|
||||||
const result = await resultPromise;
|
|
||||||
if (result.success) {
|
|
||||||
return [resultPromise];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return [];
|
|
||||||
case ExecutionMode.FIRST_RESULT:
|
|
||||||
// Execute augmentations sequentially until one returns a result
|
|
||||||
for (const augmentation of enabledAugmentations) {
|
|
||||||
const resultPromise = executeMethod(augmentation);
|
|
||||||
const result = await resultPromise;
|
|
||||||
if (result.success && result.data) {
|
|
||||||
return [resultPromise];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return [];
|
|
||||||
case ExecutionMode.SEQUENTIAL:
|
|
||||||
default:
|
|
||||||
// Execute augmentations sequentially
|
|
||||||
const results = [];
|
|
||||||
for (const augmentation of enabledAugmentations) {
|
|
||||||
const resultPromise = executeMethod(augmentation);
|
|
||||||
results.push(resultPromise);
|
|
||||||
// Check if we need to stop on error
|
|
||||||
if (options.stopOnError) {
|
|
||||||
const result = await resultPromise;
|
|
||||||
if (!result.success) {
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return results;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Enable an augmentation by name
|
|
||||||
*
|
|
||||||
* @param name The name of the augmentation to enable
|
|
||||||
* @returns True if augmentation was found and enabled
|
|
||||||
*/
|
|
||||||
enableAugmentation(name) {
|
|
||||||
for (const type of Object.keys(this.registry)) {
|
|
||||||
const augmentation = this.registry[type].find(aug => aug.name === name);
|
|
||||||
if (augmentation) {
|
|
||||||
augmentation.enabled = true;
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Disable an augmentation by name
|
|
||||||
*
|
|
||||||
* @param name The name of the augmentation to disable
|
|
||||||
* @returns True if augmentation was found and disabled
|
|
||||||
*/
|
|
||||||
disableAugmentation(name) {
|
|
||||||
for (const type of Object.keys(this.registry)) {
|
|
||||||
const augmentation = this.registry[type].find(aug => aug.name === name);
|
|
||||||
if (augmentation) {
|
|
||||||
augmentation.enabled = false;
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Check if an augmentation is enabled
|
|
||||||
*
|
|
||||||
* @param name The name of the augmentation to check
|
|
||||||
* @returns True if augmentation is found and enabled, false otherwise
|
|
||||||
*/
|
|
||||||
isAugmentationEnabled(name) {
|
|
||||||
for (const type of Object.keys(this.registry)) {
|
|
||||||
const augmentation = this.registry[type].find(aug => aug.name === name);
|
|
||||||
if (augmentation) {
|
|
||||||
return augmentation.enabled;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get all augmentations with their enabled status
|
|
||||||
*
|
|
||||||
* @returns Array of augmentations with name, type, and enabled status
|
|
||||||
*/
|
|
||||||
listAugmentationsWithStatus() {
|
|
||||||
const result = [];
|
|
||||||
for (const [type, augmentations] of Object.entries(this.registry)) {
|
|
||||||
for (const aug of augmentations) {
|
|
||||||
result.push({
|
|
||||||
name: aug.name,
|
|
||||||
type: type,
|
|
||||||
enabled: aug.enabled,
|
|
||||||
description: aug.description
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Enable all augmentations of a specific type
|
|
||||||
*
|
|
||||||
* @param type The type of augmentations to enable
|
|
||||||
* @returns Number of augmentations enabled
|
|
||||||
*/
|
|
||||||
enableAugmentationType(type) {
|
|
||||||
let count = 0;
|
|
||||||
for (const aug of this.registry[type]) {
|
|
||||||
aug.enabled = true;
|
|
||||||
count++;
|
|
||||||
}
|
|
||||||
return count;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Disable all augmentations of a specific type
|
|
||||||
*
|
|
||||||
* @param type The type of augmentations to disable
|
|
||||||
* @returns Number of augmentations disabled
|
|
||||||
*/
|
|
||||||
disableAugmentationType(type) {
|
|
||||||
let count = 0;
|
|
||||||
for (const aug of this.registry[type]) {
|
|
||||||
aug.enabled = false;
|
|
||||||
count++;
|
|
||||||
}
|
|
||||||
return count;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Create and export a default instance of the cortex
|
|
||||||
export const cortex = new Cortex();
|
|
||||||
// Backward compatibility exports
|
|
||||||
export const AugmentationPipeline = Cortex;
|
|
||||||
export const augmentationPipeline = cortex;
|
|
||||||
//# sourceMappingURL=augmentationPipeline.js.map
|
|
||||||
1
dist/augmentationPipeline.js.map
vendored
1
dist/augmentationPipeline.js.map
vendored
File diff suppressed because one or more lines are too long
47
dist/augmentationRegistry.d.ts
vendored
47
dist/augmentationRegistry.d.ts
vendored
|
|
@ -1,47 +0,0 @@
|
||||||
/**
|
|
||||||
* Augmentation Registry
|
|
||||||
*
|
|
||||||
* This module provides a registry for augmentations that are loaded at build time.
|
|
||||||
* It replaces the dynamic loading mechanism in pluginLoader.ts.
|
|
||||||
*/
|
|
||||||
import { IPipeline } from './types/pipelineTypes.js';
|
|
||||||
import { AugmentationType, IAugmentation } from './types/augmentations.js';
|
|
||||||
/**
|
|
||||||
* Sets the default pipeline instance
|
|
||||||
* This function should be called from pipeline.ts after the pipeline is created
|
|
||||||
*/
|
|
||||||
export declare function setDefaultPipeline(pipeline: IPipeline): void;
|
|
||||||
/**
|
|
||||||
* Registry of all available augmentations
|
|
||||||
*/
|
|
||||||
export declare const availableAugmentations: IAugmentation[];
|
|
||||||
/**
|
|
||||||
* Registers an augmentation with the registry
|
|
||||||
*
|
|
||||||
* @param augmentation The augmentation to register
|
|
||||||
* @returns The augmentation that was registered
|
|
||||||
*/
|
|
||||||
export declare function registerAugmentation<T extends IAugmentation>(augmentation: T): T;
|
|
||||||
/**
|
|
||||||
* Initializes the augmentation pipeline with all registered augmentations
|
|
||||||
*
|
|
||||||
* @param pipeline Optional custom pipeline to use instead of the default
|
|
||||||
* @returns The pipeline that was initialized
|
|
||||||
* @throws Error if no pipeline is provided and the default pipeline hasn't been set
|
|
||||||
*/
|
|
||||||
export declare function initializeAugmentationPipeline(pipelineInstance?: IPipeline): IPipeline;
|
|
||||||
/**
|
|
||||||
* Enables or disables an augmentation by name
|
|
||||||
*
|
|
||||||
* @param name The name of the augmentation to enable/disable
|
|
||||||
* @param enabled Whether to enable or disable the augmentation
|
|
||||||
* @returns True if the augmentation was found and updated, false otherwise
|
|
||||||
*/
|
|
||||||
export declare function setAugmentationEnabled(name: string, enabled: boolean): boolean;
|
|
||||||
/**
|
|
||||||
* Gets all augmentations of a specific type
|
|
||||||
*
|
|
||||||
* @param type The type of augmentation to get
|
|
||||||
* @returns An array of all augmentations of the specified type
|
|
||||||
*/
|
|
||||||
export declare function getAugmentationsByType(type: AugmentationType): IAugmentation[];
|
|
||||||
105
dist/augmentationRegistry.js
vendored
105
dist/augmentationRegistry.js
vendored
|
|
@ -1,105 +0,0 @@
|
||||||
/**
|
|
||||||
* Augmentation Registry
|
|
||||||
*
|
|
||||||
* This module provides a registry for augmentations that are loaded at build time.
|
|
||||||
* It replaces the dynamic loading mechanism in pluginLoader.ts.
|
|
||||||
*/
|
|
||||||
import { AugmentationType } from './types/augmentations.js';
|
|
||||||
// Forward declaration of the pipeline instance to avoid circular dependency
|
|
||||||
// The actual pipeline will be provided when initializeAugmentationPipeline is called
|
|
||||||
let defaultPipeline = null;
|
|
||||||
/**
|
|
||||||
* Sets the default pipeline instance
|
|
||||||
* This function should be called from pipeline.ts after the pipeline is created
|
|
||||||
*/
|
|
||||||
export function setDefaultPipeline(pipeline) {
|
|
||||||
defaultPipeline = pipeline;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Registry of all available augmentations
|
|
||||||
*/
|
|
||||||
export const availableAugmentations = [];
|
|
||||||
/**
|
|
||||||
* Registers an augmentation with the registry
|
|
||||||
*
|
|
||||||
* @param augmentation The augmentation to register
|
|
||||||
* @returns The augmentation that was registered
|
|
||||||
*/
|
|
||||||
export function registerAugmentation(augmentation) {
|
|
||||||
// Set enabled to true by default if not specified
|
|
||||||
if (augmentation.enabled === undefined) {
|
|
||||||
augmentation.enabled = true;
|
|
||||||
}
|
|
||||||
// Add to the registry
|
|
||||||
availableAugmentations.push(augmentation);
|
|
||||||
return augmentation;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Initializes the augmentation pipeline with all registered augmentations
|
|
||||||
*
|
|
||||||
* @param pipeline Optional custom pipeline to use instead of the default
|
|
||||||
* @returns The pipeline that was initialized
|
|
||||||
* @throws Error if no pipeline is provided and the default pipeline hasn't been set
|
|
||||||
*/
|
|
||||||
export function initializeAugmentationPipeline(pipelineInstance) {
|
|
||||||
// Use the provided pipeline or fall back to the default
|
|
||||||
const pipeline = pipelineInstance || defaultPipeline;
|
|
||||||
if (!pipeline) {
|
|
||||||
throw new Error('No pipeline provided and default pipeline not set. Call setDefaultPipeline first.');
|
|
||||||
}
|
|
||||||
// Register all augmentations with the pipeline
|
|
||||||
for (const augmentation of availableAugmentations) {
|
|
||||||
if (augmentation.enabled) {
|
|
||||||
pipeline.register(augmentation);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return pipeline;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Enables or disables an augmentation by name
|
|
||||||
*
|
|
||||||
* @param name The name of the augmentation to enable/disable
|
|
||||||
* @param enabled Whether to enable or disable the augmentation
|
|
||||||
* @returns True if the augmentation was found and updated, false otherwise
|
|
||||||
*/
|
|
||||||
export function setAugmentationEnabled(name, enabled) {
|
|
||||||
const augmentation = availableAugmentations.find(aug => aug.name === name);
|
|
||||||
if (augmentation) {
|
|
||||||
augmentation.enabled = enabled;
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Gets all augmentations of a specific type
|
|
||||||
*
|
|
||||||
* @param type The type of augmentation to get
|
|
||||||
* @returns An array of all augmentations of the specified type
|
|
||||||
*/
|
|
||||||
export function getAugmentationsByType(type) {
|
|
||||||
return availableAugmentations.filter(aug => {
|
|
||||||
// Check if the augmentation is of the specified type
|
|
||||||
// This is a simplified check and may need to be updated based on how types are determined
|
|
||||||
switch (type) {
|
|
||||||
case AugmentationType.SENSE:
|
|
||||||
return 'processRawData' in aug && 'listenToFeed' in aug;
|
|
||||||
case AugmentationType.CONDUIT:
|
|
||||||
return 'establishConnection' in aug && 'readData' in aug && 'writeData' in aug;
|
|
||||||
case AugmentationType.COGNITION:
|
|
||||||
return 'reason' in aug && 'infer' in aug && 'executeLogic' in aug;
|
|
||||||
case AugmentationType.MEMORY:
|
|
||||||
return 'storeData' in aug && 'retrieveData' in aug && 'updateData' in aug;
|
|
||||||
case AugmentationType.PERCEPTION:
|
|
||||||
return 'interpret' in aug && 'organize' in aug && 'generateVisualization' in aug;
|
|
||||||
case AugmentationType.DIALOG:
|
|
||||||
return 'processUserInput' in aug && 'generateResponse' in aug && 'manageContext' in aug;
|
|
||||||
case AugmentationType.ACTIVATION:
|
|
||||||
return 'triggerAction' in aug && 'generateOutput' in aug && 'interactExternal' in aug;
|
|
||||||
case AugmentationType.WEBSOCKET:
|
|
||||||
return 'connectWebSocket' in aug && 'sendWebSocketMessage' in aug && 'onWebSocketMessage' in aug;
|
|
||||||
default:
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=augmentationRegistry.js.map
|
|
||||||
1
dist/augmentationRegistry.js.map
vendored
1
dist/augmentationRegistry.js.map
vendored
|
|
@ -1 +0,0 @@
|
||||||
{"version":3,"file":"augmentationRegistry.js","sourceRoot":"","sources":["../src/augmentationRegistry.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAGH,OAAO,EAAE,gBAAgB,EAAiB,MAAM,0BAA0B,CAAA;AAE1E,4EAA4E;AAC5E,qFAAqF;AACrF,IAAI,eAAe,GAAqB,IAAI,CAAA;AAE5C;;;GAGG;AACH,MAAM,UAAU,kBAAkB,CAAC,QAAmB;IACpD,eAAe,GAAG,QAAQ,CAAA;AAC5B,CAAC;AAED;;GAEG;AACH,MAAM,CAAC,MAAM,sBAAsB,GAAoB,EAAE,CAAA;AAEzD;;;;;GAKG;AACH,MAAM,UAAU,oBAAoB,CAA0B,YAAe;IAC3E,kDAAkD;IAClD,IAAI,YAAY,CAAC,OAAO,KAAK,SAAS,EAAE,CAAC;QACvC,YAAY,CAAC,OAAO,GAAG,IAAI,CAAA;IAC7B,CAAC;IAED,sBAAsB;IACtB,sBAAsB,CAAC,IAAI,CAAC,YAAY,CAAC,CAAA;IAEzC,OAAO,YAAY,CAAA;AACrB,CAAC;AAED;;;;;;GAMG;AACH,MAAM,UAAU,8BAA8B,CAC5C,gBAA4B;IAE5B,wDAAwD;IACxD,MAAM,QAAQ,GAAG,gBAAgB,IAAI,eAAe,CAAA;IAEpD,IAAI,CAAC,QAAQ,EAAE,CAAC;QACd,MAAM,IAAI,KAAK,CAAC,mFAAmF,CAAC,CAAA;IACtG,CAAC;IAED,+CAA+C;IAC/C,KAAK,MAAM,YAAY,IAAI,sBAAsB,EAAE,CAAC;QAClD,IAAI,YAAY,CAAC,OAAO,EAAE,CAAC;YACzB,QAAQ,CAAC,QAAQ,CAAC,YAAY,CAAC,CAAA;QACjC,CAAC;IACH,CAAC;IAED,OAAO,QAAQ,CAAA;AACjB,CAAC;AAED;;;;;;GAMG;AACH,MAAM,UAAU,sBAAsB,CAAC,IAAY,EAAE,OAAgB;IACnE,MAAM,YAAY,GAAG,sBAAsB,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,IAAI,KAAK,IAAI,CAAC,CAAA;IAE1E,IAAI,YAAY,EAAE,CAAC;QACjB,YAAY,CAAC,OAAO,GAAG,OAAO,CAAA;QAC9B,OAAO,IAAI,CAAA;IACb,CAAC;IAED,OAAO,KAAK,CAAA;AACd,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,sBAAsB,CAAC,IAAsB;IAC3D,OAAO,sBAAsB,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE;QACzC,qDAAqD;QACrD,0FAA0F;QAC1F,QAAQ,IAAI,EAAE,CAAC;YACb,KAAK,gBAAgB,CAAC,KAAK;gBACzB,OAAO,gBAAgB,IAAI,GAAG,IAAI,cAAc,IAAI,GAAG,CAAA;YACzD,KAAK,gBAAgB,CAAC,OAAO;gBAC3B,OAAO,qBAAqB,IAAI,GAAG,IAAI,UAAU,IAAI,GAAG,IAAI,WAAW,IAAI,GAAG,CAAA;YAChF,KAAK,gBAAgB,CAAC,SAAS;gBAC7B,OAAO,QAAQ,IAAI,GAAG,IAAI,OAAO,IAAI,GAAG,IAAI,cAAc,IAAI,GAAG,CAAA;YACnE,KAAK,gBAAgB,CAAC,MAAM;gBAC1B,OAAO,WAAW,IAAI,GAAG,IAAI,cAAc,IAAI,GAAG,IAAI,YAAY,IAAI,GAAG,CAAA;YAC3E,KAAK,gBAAgB,CAAC,UAAU;gBAC9B,OAAO,WAAW,IAAI,GAAG,IAAI,UAAU,IAAI,GAAG,IAAI,uBAAuB,IAAI,GAAG,CAAA;YAClF,KAAK,gBAAgB,CAAC,MAAM;gBAC1B,OAAO,kBAAkB,IAAI,GAAG,IAAI,kBAAkB,IAAI,GAAG,IAAI,eAAe,IAAI,GAAG,CAAA;YACzF,KAAK,gBAAgB,CAAC,UAAU;gBAC9B,OAAO,eAAe,IAAI,GAAG,IAAI,gBAAgB,IAAI,GAAG,IAAI,kBAAkB,IAAI,GAAG,CAAA;YACvF,KAAK,gBAAgB,CAAC,SAAS;gBAC7B,OAAO,kBAAkB,IAAI,GAAG,IAAI,sBAAsB,IAAI,GAAG,IAAI,oBAAoB,IAAI,GAAG,CAAA;YAClG;gBACE,OAAO,KAAK,CAAA;QAChB,CAAC;IACH,CAAC,CAAC,CAAA;AACJ,CAAC"}
|
|
||||||
146
dist/augmentationRegistryLoader.d.ts
vendored
146
dist/augmentationRegistryLoader.d.ts
vendored
|
|
@ -1,146 +0,0 @@
|
||||||
/**
|
|
||||||
* Augmentation Registry Loader
|
|
||||||
*
|
|
||||||
* This module provides functionality for loading augmentation registrations
|
|
||||||
* at build time. It's designed to be used with build tools like webpack or rollup
|
|
||||||
* to automatically discover and register augmentations.
|
|
||||||
*/
|
|
||||||
import { IAugmentation } from './types/augmentations.js';
|
|
||||||
/**
|
|
||||||
* Options for the augmentation registry loader
|
|
||||||
*/
|
|
||||||
export interface AugmentationRegistryLoaderOptions {
|
|
||||||
/**
|
|
||||||
* Whether to automatically initialize the augmentations after loading
|
|
||||||
* @default false
|
|
||||||
*/
|
|
||||||
autoInitialize?: boolean;
|
|
||||||
/**
|
|
||||||
* Whether to log debug information during loading
|
|
||||||
* @default false
|
|
||||||
*/
|
|
||||||
debug?: boolean;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Result of loading augmentations
|
|
||||||
*/
|
|
||||||
export interface AugmentationLoadResult {
|
|
||||||
/**
|
|
||||||
* The augmentations that were loaded
|
|
||||||
*/
|
|
||||||
augmentations: IAugmentation[];
|
|
||||||
/**
|
|
||||||
* Any errors that occurred during loading
|
|
||||||
*/
|
|
||||||
errors: Error[];
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Loads augmentations from the specified modules
|
|
||||||
*
|
|
||||||
* This function is designed to be used with build tools like webpack or rollup
|
|
||||||
* to automatically discover and register augmentations.
|
|
||||||
*
|
|
||||||
* @param modules An object containing modules with augmentations to register
|
|
||||||
* @param options Options for the loader
|
|
||||||
* @returns A promise that resolves with the result of loading the augmentations
|
|
||||||
*
|
|
||||||
* @example
|
|
||||||
* ```typescript
|
|
||||||
* // webpack.config.js
|
|
||||||
* const { AugmentationRegistryPlugin } = require('brainy/dist/webpack');
|
|
||||||
*
|
|
||||||
* module.exports = {
|
|
||||||
* // ... other webpack config
|
|
||||||
* plugins: [
|
|
||||||
* new AugmentationRegistryPlugin({
|
|
||||||
* // Pattern to match files containing augmentations
|
|
||||||
* pattern: /augmentation\.js$/,
|
|
||||||
* // Options for the loader
|
|
||||||
* options: {
|
|
||||||
* autoInitialize: true,
|
|
||||||
* debug: true
|
|
||||||
* }
|
|
||||||
* })
|
|
||||||
* ]
|
|
||||||
* };
|
|
||||||
* ```
|
|
||||||
*/
|
|
||||||
export declare function loadAugmentationsFromModules(modules: Record<string, any>, options?: AugmentationRegistryLoaderOptions): Promise<AugmentationLoadResult>;
|
|
||||||
/**
|
|
||||||
* Creates a webpack plugin for automatically loading augmentations
|
|
||||||
*
|
|
||||||
* @param options Options for the plugin
|
|
||||||
* @returns A webpack plugin
|
|
||||||
*
|
|
||||||
* @example
|
|
||||||
* ```typescript
|
|
||||||
* // webpack.config.js
|
|
||||||
* const { createAugmentationRegistryPlugin } = require('brainy/dist/webpack');
|
|
||||||
*
|
|
||||||
* module.exports = {
|
|
||||||
* // ... other webpack config
|
|
||||||
* plugins: [
|
|
||||||
* createAugmentationRegistryPlugin({
|
|
||||||
* pattern: /augmentation\.js$/,
|
|
||||||
* options: {
|
|
||||||
* autoInitialize: true,
|
|
||||||
* debug: true
|
|
||||||
* }
|
|
||||||
* })
|
|
||||||
* ]
|
|
||||||
* };
|
|
||||||
* ```
|
|
||||||
*/
|
|
||||||
export declare function createAugmentationRegistryPlugin(options: {
|
|
||||||
/**
|
|
||||||
* Pattern to match files containing augmentations
|
|
||||||
*/
|
|
||||||
pattern: RegExp;
|
|
||||||
/**
|
|
||||||
* Options for the loader
|
|
||||||
*/
|
|
||||||
options?: AugmentationRegistryLoaderOptions;
|
|
||||||
}): {
|
|
||||||
name: string;
|
|
||||||
pattern: RegExp;
|
|
||||||
options: AugmentationRegistryLoaderOptions;
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Creates a rollup plugin for automatically loading augmentations
|
|
||||||
*
|
|
||||||
* @param options Options for the plugin
|
|
||||||
* @returns A rollup plugin
|
|
||||||
*
|
|
||||||
* @example
|
|
||||||
* ```typescript
|
|
||||||
* // rollup.config.js
|
|
||||||
* import { createAugmentationRegistryRollupPlugin } from 'brainy/dist/rollup';
|
|
||||||
*
|
|
||||||
* export default {
|
|
||||||
* // ... other rollup config
|
|
||||||
* plugins: [
|
|
||||||
* createAugmentationRegistryRollupPlugin({
|
|
||||||
* pattern: /augmentation\.js$/,
|
|
||||||
* options: {
|
|
||||||
* autoInitialize: true,
|
|
||||||
* debug: true
|
|
||||||
* }
|
|
||||||
* })
|
|
||||||
* ]
|
|
||||||
* };
|
|
||||||
* ```
|
|
||||||
*/
|
|
||||||
export declare function createAugmentationRegistryRollupPlugin(options: {
|
|
||||||
/**
|
|
||||||
* Pattern to match files containing augmentations
|
|
||||||
*/
|
|
||||||
pattern: RegExp;
|
|
||||||
/**
|
|
||||||
* Options for the loader
|
|
||||||
*/
|
|
||||||
options?: AugmentationRegistryLoaderOptions;
|
|
||||||
}): {
|
|
||||||
name: string;
|
|
||||||
pattern: RegExp;
|
|
||||||
options: AugmentationRegistryLoaderOptions;
|
|
||||||
};
|
|
||||||
213
dist/augmentationRegistryLoader.js
vendored
213
dist/augmentationRegistryLoader.js
vendored
|
|
@ -1,213 +0,0 @@
|
||||||
/**
|
|
||||||
* Augmentation Registry Loader
|
|
||||||
*
|
|
||||||
* This module provides functionality for loading augmentation registrations
|
|
||||||
* at build time. It's designed to be used with build tools like webpack or rollup
|
|
||||||
* to automatically discover and register augmentations.
|
|
||||||
*/
|
|
||||||
import { registerAugmentation } from './augmentationRegistry.js';
|
|
||||||
/**
|
|
||||||
* Default options for the augmentation registry loader
|
|
||||||
*/
|
|
||||||
const DEFAULT_OPTIONS = {
|
|
||||||
autoInitialize: false,
|
|
||||||
debug: false
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Loads augmentations from the specified modules
|
|
||||||
*
|
|
||||||
* This function is designed to be used with build tools like webpack or rollup
|
|
||||||
* to automatically discover and register augmentations.
|
|
||||||
*
|
|
||||||
* @param modules An object containing modules with augmentations to register
|
|
||||||
* @param options Options for the loader
|
|
||||||
* @returns A promise that resolves with the result of loading the augmentations
|
|
||||||
*
|
|
||||||
* @example
|
|
||||||
* ```typescript
|
|
||||||
* // webpack.config.js
|
|
||||||
* const { AugmentationRegistryPlugin } = require('brainy/dist/webpack');
|
|
||||||
*
|
|
||||||
* module.exports = {
|
|
||||||
* // ... other webpack config
|
|
||||||
* plugins: [
|
|
||||||
* new AugmentationRegistryPlugin({
|
|
||||||
* // Pattern to match files containing augmentations
|
|
||||||
* pattern: /augmentation\.js$/,
|
|
||||||
* // Options for the loader
|
|
||||||
* options: {
|
|
||||||
* autoInitialize: true,
|
|
||||||
* debug: true
|
|
||||||
* }
|
|
||||||
* })
|
|
||||||
* ]
|
|
||||||
* };
|
|
||||||
* ```
|
|
||||||
*/
|
|
||||||
export async function loadAugmentationsFromModules(modules, options = {}) {
|
|
||||||
const opts = { ...DEFAULT_OPTIONS, ...options };
|
|
||||||
const result = {
|
|
||||||
augmentations: [],
|
|
||||||
errors: []
|
|
||||||
};
|
|
||||||
if (opts.debug) {
|
|
||||||
console.log(`[AugmentationRegistryLoader] Loading augmentations from ${Object.keys(modules).length} modules`);
|
|
||||||
}
|
|
||||||
// Process each module
|
|
||||||
for (const [modulePath, module] of Object.entries(modules)) {
|
|
||||||
try {
|
|
||||||
if (opts.debug) {
|
|
||||||
console.log(`[AugmentationRegistryLoader] Processing module: ${modulePath}`);
|
|
||||||
}
|
|
||||||
// Extract augmentations from the module
|
|
||||||
const augmentations = extractAugmentationsFromModule(module);
|
|
||||||
if (augmentations.length === 0) {
|
|
||||||
if (opts.debug) {
|
|
||||||
console.log(`[AugmentationRegistryLoader] No augmentations found in module: ${modulePath}`);
|
|
||||||
}
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
// Register each augmentation
|
|
||||||
for (const augmentation of augmentations) {
|
|
||||||
try {
|
|
||||||
const registered = registerAugmentation(augmentation);
|
|
||||||
result.augmentations.push(registered);
|
|
||||||
if (opts.debug) {
|
|
||||||
console.log(`[AugmentationRegistryLoader] Registered augmentation: ${registered.name}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
const err = error instanceof Error ? error : new Error(String(error));
|
|
||||||
result.errors.push(err);
|
|
||||||
if (opts.debug) {
|
|
||||||
console.error(`[AugmentationRegistryLoader] Failed to register augmentation: ${err.message}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
const err = error instanceof Error ? error : new Error(String(error));
|
|
||||||
result.errors.push(err);
|
|
||||||
if (opts.debug) {
|
|
||||||
console.error(`[AugmentationRegistryLoader] Error processing module ${modulePath}: ${err.message}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (opts.debug) {
|
|
||||||
console.log(`[AugmentationRegistryLoader] Loaded ${result.augmentations.length} augmentations with ${result.errors.length} errors`);
|
|
||||||
}
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Extracts augmentations from a module
|
|
||||||
*
|
|
||||||
* @param module The module to extract augmentations from
|
|
||||||
* @returns An array of augmentations found in the module
|
|
||||||
*/
|
|
||||||
function extractAugmentationsFromModule(module) {
|
|
||||||
const augmentations = [];
|
|
||||||
// If the module itself is an augmentation, add it
|
|
||||||
if (isAugmentation(module)) {
|
|
||||||
augmentations.push(module);
|
|
||||||
}
|
|
||||||
// Check for exported augmentations
|
|
||||||
if (module && typeof module === 'object') {
|
|
||||||
for (const key of Object.keys(module)) {
|
|
||||||
const exported = module[key];
|
|
||||||
// Skip non-objects and null
|
|
||||||
if (!exported || typeof exported !== 'object') {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
// If the exported value is an augmentation, add it
|
|
||||||
if (isAugmentation(exported)) {
|
|
||||||
augmentations.push(exported);
|
|
||||||
}
|
|
||||||
// If the exported value is an array of augmentations, add them
|
|
||||||
if (Array.isArray(exported) && exported.every(isAugmentation)) {
|
|
||||||
augmentations.push(...exported);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return augmentations;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Checks if an object is an augmentation
|
|
||||||
*
|
|
||||||
* @param obj The object to check
|
|
||||||
* @returns True if the object is an augmentation
|
|
||||||
*/
|
|
||||||
function isAugmentation(obj) {
|
|
||||||
return (obj &&
|
|
||||||
typeof obj === 'object' &&
|
|
||||||
typeof obj.name === 'string' &&
|
|
||||||
typeof obj.initialize === 'function' &&
|
|
||||||
typeof obj.shutDown === 'function' &&
|
|
||||||
typeof obj.getStatus === 'function');
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Creates a webpack plugin for automatically loading augmentations
|
|
||||||
*
|
|
||||||
* @param options Options for the plugin
|
|
||||||
* @returns A webpack plugin
|
|
||||||
*
|
|
||||||
* @example
|
|
||||||
* ```typescript
|
|
||||||
* // webpack.config.js
|
|
||||||
* const { createAugmentationRegistryPlugin } = require('brainy/dist/webpack');
|
|
||||||
*
|
|
||||||
* module.exports = {
|
|
||||||
* // ... other webpack config
|
|
||||||
* plugins: [
|
|
||||||
* createAugmentationRegistryPlugin({
|
|
||||||
* pattern: /augmentation\.js$/,
|
|
||||||
* options: {
|
|
||||||
* autoInitialize: true,
|
|
||||||
* debug: true
|
|
||||||
* }
|
|
||||||
* })
|
|
||||||
* ]
|
|
||||||
* };
|
|
||||||
* ```
|
|
||||||
*/
|
|
||||||
export function createAugmentationRegistryPlugin(options) {
|
|
||||||
// This is just a placeholder - the actual implementation would depend on the build tool
|
|
||||||
return {
|
|
||||||
name: 'AugmentationRegistryPlugin',
|
|
||||||
pattern: options.pattern,
|
|
||||||
options: options.options || {}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Creates a rollup plugin for automatically loading augmentations
|
|
||||||
*
|
|
||||||
* @param options Options for the plugin
|
|
||||||
* @returns A rollup plugin
|
|
||||||
*
|
|
||||||
* @example
|
|
||||||
* ```typescript
|
|
||||||
* // rollup.config.js
|
|
||||||
* import { createAugmentationRegistryRollupPlugin } from 'brainy/dist/rollup';
|
|
||||||
*
|
|
||||||
* export default {
|
|
||||||
* // ... other rollup config
|
|
||||||
* plugins: [
|
|
||||||
* createAugmentationRegistryRollupPlugin({
|
|
||||||
* pattern: /augmentation\.js$/,
|
|
||||||
* options: {
|
|
||||||
* autoInitialize: true,
|
|
||||||
* debug: true
|
|
||||||
* }
|
|
||||||
* })
|
|
||||||
* ]
|
|
||||||
* };
|
|
||||||
* ```
|
|
||||||
*/
|
|
||||||
export function createAugmentationRegistryRollupPlugin(options) {
|
|
||||||
// This is just a placeholder - the actual implementation would depend on the build tool
|
|
||||||
return {
|
|
||||||
name: 'augmentation-registry-rollup-plugin',
|
|
||||||
pattern: options.pattern,
|
|
||||||
options: options.options || {}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=augmentationRegistryLoader.js.map
|
|
||||||
1
dist/augmentationRegistryLoader.js.map
vendored
1
dist/augmentationRegistryLoader.js.map
vendored
|
|
@ -1 +0,0 @@
|
||||||
{"version":3,"file":"augmentationRegistryLoader.js","sourceRoot":"","sources":["../src/augmentationRegistryLoader.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAGH,OAAO,EAAE,oBAAoB,EAAE,MAAM,2BAA2B,CAAA;AAmBhE;;GAEG;AACH,MAAM,eAAe,GAAsC;IACzD,cAAc,EAAE,KAAK;IACrB,KAAK,EAAE,KAAK;CACb,CAAA;AAiBD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA8BG;AACH,MAAM,CAAC,KAAK,UAAU,4BAA4B,CAChD,OAA4B,EAC5B,UAA6C,EAAE;IAE/C,MAAM,IAAI,GAAG,EAAE,GAAG,eAAe,EAAE,GAAG,OAAO,EAAE,CAAA;IAC/C,MAAM,MAAM,GAA2B;QACrC,aAAa,EAAE,EAAE;QACjB,MAAM,EAAE,EAAE;KACX,CAAA;IAED,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;QACf,OAAO,CAAC,GAAG,CAAC,2DAA2D,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,MAAM,UAAU,CAAC,CAAA;IAC/G,CAAC;IAED,sBAAsB;IACtB,KAAK,MAAM,CAAC,UAAU,EAAE,MAAM,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC;QAC3D,IAAI,CAAC;YACH,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;gBACf,OAAO,CAAC,GAAG,CAAC,mDAAmD,UAAU,EAAE,CAAC,CAAA;YAC9E,CAAC;YAED,wCAAwC;YACxC,MAAM,aAAa,GAAG,8BAA8B,CAAC,MAAM,CAAC,CAAA;YAE5D,IAAI,aAAa,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBAC/B,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;oBACf,OAAO,CAAC,GAAG,CAAC,kEAAkE,UAAU,EAAE,CAAC,CAAA;gBAC7F,CAAC;gBACD,SAAQ;YACV,CAAC;YAED,6BAA6B;YAC7B,KAAK,MAAM,YAAY,IAAI,aAAa,EAAE,CAAC;gBACzC,IAAI,CAAC;oBACH,MAAM,UAAU,GAAG,oBAAoB,CAAC,YAAY,CAAC,CAAA;oBACrD,MAAM,CAAC,aAAa,CAAC,IAAI,CAAC,UAAU,CAAC,CAAA;oBAErC,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;wBACf,OAAO,CAAC,GAAG,CAAC,yDAAyD,UAAU,CAAC,IAAI,EAAE,CAAC,CAAA;oBACzF,CAAC;gBACH,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBACf,MAAM,GAAG,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAA;oBACrE,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAA;oBAEvB,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;wBACf,OAAO,CAAC,KAAK,CAAC,iEAAiE,GAAG,CAAC,OAAO,EAAE,CAAC,CAAA;oBAC/F,CAAC;gBACH,CAAC;YACH,CAAC;QACH,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,GAAG,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAA;YACrE,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAA;YAEvB,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;gBACf,OAAO,CAAC,KAAK,CAAC,wDAAwD,UAAU,KAAK,GAAG,CAAC,OAAO,EAAE,CAAC,CAAA;YACrG,CAAC;QACH,CAAC;IACH,CAAC;IAED,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;QACf,OAAO,CAAC,GAAG,CAAC,uCAAuC,MAAM,CAAC,aAAa,CAAC,MAAM,uBAAuB,MAAM,CAAC,MAAM,CAAC,MAAM,SAAS,CAAC,CAAA;IACrI,CAAC;IAED,OAAO,MAAM,CAAA;AACf,CAAC;AAED;;;;;GAKG;AACH,SAAS,8BAA8B,CAAC,MAAW;IACjD,MAAM,aAAa,GAAoB,EAAE,CAAA;IAEzC,kDAAkD;IAClD,IAAI,cAAc,CAAC,MAAM,CAAC,EAAE,CAAC;QAC3B,aAAa,CAAC,IAAI,CAAC,MAAM,CAAC,CAAA;IAC5B,CAAC;IAED,mCAAmC;IACnC,IAAI,MAAM,IAAI,OAAO,MAAM,KAAK,QAAQ,EAAE,CAAC;QACzC,KAAK,MAAM,GAAG,IAAI,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;YACtC,MAAM,QAAQ,GAAG,MAAM,CAAC,GAAG,CAAC,CAAA;YAE5B,4BAA4B;YAC5B,IAAI,CAAC,QAAQ,IAAI,OAAO,QAAQ,KAAK,QAAQ,EAAE,CAAC;gBAC9C,SAAQ;YACV,CAAC;YAED,mDAAmD;YACnD,IAAI,cAAc,CAAC,QAAQ,CAAC,EAAE,CAAC;gBAC7B,aAAa,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAA;YAC9B,CAAC;YAED,+DAA+D;YAC/D,IAAI,KAAK,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,QAAQ,CAAC,KAAK,CAAC,cAAc,CAAC,EAAE,CAAC;gBAC9D,aAAa,CAAC,IAAI,CAAC,GAAG,QAAQ,CAAC,CAAA;YACjC,CAAC;QACH,CAAC;IACH,CAAC;IAED,OAAO,aAAa,CAAA;AACtB,CAAC;AAED;;;;;GAKG;AACH,SAAS,cAAc,CAAC,GAAQ;IAC9B,OAAO,CACL,GAAG;QACH,OAAO,GAAG,KAAK,QAAQ;QACvB,OAAO,GAAG,CAAC,IAAI,KAAK,QAAQ;QAC5B,OAAO,GAAG,CAAC,UAAU,KAAK,UAAU;QACpC,OAAO,GAAG,CAAC,QAAQ,KAAK,UAAU;QAClC,OAAO,GAAG,CAAC,SAAS,KAAK,UAAU,CACpC,CAAA;AACH,CAAC;AAED;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AACH,MAAM,UAAU,gCAAgC,CAAC,OAUhD;IACC,wFAAwF;IACxF,OAAO;QACL,IAAI,EAAE,4BAA4B;QAClC,OAAO,EAAE,OAAO,CAAC,OAAO;QACxB,OAAO,EAAE,OAAO,CAAC,OAAO,IAAI,EAAE;KAC/B,CAAA;AACH,CAAC;AAED;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AACH,MAAM,UAAU,sCAAsC,CAAC,OAUtD;IACC,wFAAwF;IACxF,OAAO;QACL,IAAI,EAAE,qCAAqC;QAC3C,OAAO,EAAE,OAAO,CAAC,OAAO;QACxB,OAAO,EAAE,OAAO,CAAC,OAAO,IAAI,EAAE;KAC/B,CAAA;AACH,CAAC"}
|
|
||||||
172
dist/augmentations/conduitAugmentations.d.ts
vendored
172
dist/augmentations/conduitAugmentations.d.ts
vendored
|
|
@ -1,172 +0,0 @@
|
||||||
import { AugmentationType, IConduitAugmentation, IWebSocketSupport, AugmentationResponse, WebSocketConnection } from '../types/augmentations.js';
|
|
||||||
/**
|
|
||||||
* Base class for conduit augmentations that provide data synchronization between Brainy instances
|
|
||||||
*/
|
|
||||||
declare abstract class BaseConduitAugmentation implements IConduitAugmentation {
|
|
||||||
readonly name: string;
|
|
||||||
readonly description: string;
|
|
||||||
enabled: boolean;
|
|
||||||
protected isInitialized: boolean;
|
|
||||||
protected connections: Map<string, any>;
|
|
||||||
constructor(name: string);
|
|
||||||
initialize(): Promise<void>;
|
|
||||||
shutDown(): Promise<void>;
|
|
||||||
getStatus(): Promise<'active' | 'inactive' | 'error'>;
|
|
||||||
abstract establishConnection(targetSystemId: string, config: Record<string, unknown>): Promise<AugmentationResponse<WebSocketConnection>>;
|
|
||||||
abstract readData(query: Record<string, unknown>, options?: Record<string, unknown>): Promise<AugmentationResponse<unknown>>;
|
|
||||||
abstract writeData(data: Record<string, unknown>, options?: Record<string, unknown>): Promise<AugmentationResponse<unknown>>;
|
|
||||||
abstract monitorStream(streamId: string, callback: (data: unknown) => void): Promise<void>;
|
|
||||||
protected ensureInitialized(): Promise<void>;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* WebSocket conduit augmentation for syncing Brainy instances using WebSockets
|
|
||||||
*
|
|
||||||
* This conduit is for syncing between browsers and servers, or between servers.
|
|
||||||
* WebSockets cannot be used for direct browser-to-browser communication without a server in the middle.
|
|
||||||
*/
|
|
||||||
export declare class WebSocketConduitAugmentation extends BaseConduitAugmentation implements IWebSocketSupport {
|
|
||||||
readonly description = "Conduit augmentation that syncs Brainy instances using WebSockets";
|
|
||||||
private webSocketConnections;
|
|
||||||
private messageCallbacks;
|
|
||||||
constructor(name?: string);
|
|
||||||
getType(): AugmentationType;
|
|
||||||
/**
|
|
||||||
* Establishes a connection to another Brainy instance
|
|
||||||
* @param targetSystemId The URL or identifier of the target system
|
|
||||||
* @param config Configuration options for the connection
|
|
||||||
*/
|
|
||||||
establishConnection(targetSystemId: string, config: Record<string, unknown>): Promise<AugmentationResponse<WebSocketConnection>>;
|
|
||||||
/**
|
|
||||||
* Reads data from a connected Brainy instance
|
|
||||||
* @param query Query parameters for reading data
|
|
||||||
* @param options Additional options
|
|
||||||
*/
|
|
||||||
readData(query: Record<string, unknown>, options?: Record<string, unknown>): Promise<AugmentationResponse<unknown>>;
|
|
||||||
/**
|
|
||||||
* Writes data to a connected Brainy instance
|
|
||||||
* @param data The data to write
|
|
||||||
* @param options Additional options
|
|
||||||
*/
|
|
||||||
writeData(data: Record<string, unknown>, options?: Record<string, unknown>): Promise<AugmentationResponse<unknown>>;
|
|
||||||
/**
|
|
||||||
* Monitors a data stream from a connected Brainy instance
|
|
||||||
* @param streamId The ID of the stream to monitor (usually a connection ID)
|
|
||||||
* @param callback Function to call when new data is received
|
|
||||||
*/
|
|
||||||
monitorStream(streamId: string, callback: (data: unknown) => void): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Establishes a WebSocket connection
|
|
||||||
* @param url The WebSocket server URL to connect to
|
|
||||||
* @param protocols Optional subprotocols
|
|
||||||
*/
|
|
||||||
connectWebSocket(url: string, protocols?: string | string[]): Promise<WebSocketConnection>;
|
|
||||||
/**
|
|
||||||
* Sends data through an established WebSocket connection
|
|
||||||
* @param connectionId The identifier of the established connection
|
|
||||||
* @param data The data to send (will be serialized if not a string)
|
|
||||||
*/
|
|
||||||
sendWebSocketMessage(connectionId: string, data: unknown): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Registers a callback for incoming WebSocket messages
|
|
||||||
* @param connectionId The identifier of the established connection
|
|
||||||
* @param callback The function to call when a message is received
|
|
||||||
*/
|
|
||||||
onWebSocketMessage(connectionId: string, callback: (data: unknown) => void): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Removes a callback for incoming WebSocket messages
|
|
||||||
* @param connectionId The identifier of the established connection
|
|
||||||
* @param callback The function to remove from the callbacks
|
|
||||||
*/
|
|
||||||
offWebSocketMessage(connectionId: string, callback: (data: unknown) => void): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Closes an established WebSocket connection
|
|
||||||
* @param connectionId The identifier of the established connection
|
|
||||||
* @param code Optional close code
|
|
||||||
* @param reason Optional close reason
|
|
||||||
*/
|
|
||||||
closeWebSocket(connectionId: string, code?: number, reason?: string): Promise<void>;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* WebRTC conduit augmentation for syncing Brainy instances using WebRTC
|
|
||||||
*
|
|
||||||
* This conduit is for direct peer-to-peer syncing between browsers.
|
|
||||||
* It is the recommended approach for browser-to-browser communication.
|
|
||||||
*/
|
|
||||||
export declare class WebRTCConduitAugmentation extends BaseConduitAugmentation implements IWebSocketSupport {
|
|
||||||
readonly description = "Conduit augmentation that syncs Brainy instances using WebRTC";
|
|
||||||
private peerConnections;
|
|
||||||
private dataChannels;
|
|
||||||
private webSocketConnections;
|
|
||||||
private messageCallbacks;
|
|
||||||
private signalServer;
|
|
||||||
constructor(name?: string);
|
|
||||||
getType(): AugmentationType;
|
|
||||||
initialize(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Establishes a connection to another Brainy instance using WebRTC
|
|
||||||
* @param targetSystemId The peer ID or signal server URL
|
|
||||||
* @param config Configuration options for the connection
|
|
||||||
*/
|
|
||||||
establishConnection(targetSystemId: string, config: Record<string, unknown>): Promise<AugmentationResponse<WebSocketConnection>>;
|
|
||||||
/**
|
|
||||||
* Handles an incoming WebRTC offer
|
|
||||||
* @param peerId The ID of the peer sending the offer
|
|
||||||
* @param offer The SDP offer
|
|
||||||
* @param config Configuration options
|
|
||||||
*/
|
|
||||||
private handleOffer;
|
|
||||||
/**
|
|
||||||
* Reads data from a connected Brainy instance
|
|
||||||
* @param query Query parameters for reading data
|
|
||||||
* @param options Additional options
|
|
||||||
*/
|
|
||||||
readData(query: Record<string, unknown>, options?: Record<string, unknown>): Promise<AugmentationResponse<unknown>>;
|
|
||||||
/**
|
|
||||||
* Writes data to a connected Brainy instance
|
|
||||||
* @param data The data to write
|
|
||||||
* @param options Additional options
|
|
||||||
*/
|
|
||||||
writeData(data: Record<string, unknown>, options?: Record<string, unknown>): Promise<AugmentationResponse<unknown>>;
|
|
||||||
/**
|
|
||||||
* Monitors a data stream from a connected Brainy instance
|
|
||||||
* @param streamId The ID of the stream to monitor (usually a connection ID)
|
|
||||||
* @param callback Function to call when new data is received
|
|
||||||
*/
|
|
||||||
monitorStream(streamId: string, callback: (data: unknown) => void): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Establishes a WebSocket connection (used for signaling in WebRTC)
|
|
||||||
* @param url The WebSocket server URL to connect to
|
|
||||||
* @param protocols Optional subprotocols
|
|
||||||
*/
|
|
||||||
connectWebSocket(url: string, protocols?: string | string[]): Promise<WebSocketConnection>;
|
|
||||||
/**
|
|
||||||
* Sends data through an established WebSocket or WebRTC connection
|
|
||||||
* @param connectionId The identifier of the established connection
|
|
||||||
* @param data The data to send (will be serialized if not a string)
|
|
||||||
*/
|
|
||||||
sendWebSocketMessage(connectionId: string, data: unknown): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Registers a callback for incoming WebSocket or WebRTC messages
|
|
||||||
* @param connectionId The identifier of the established connection
|
|
||||||
* @param callback The function to call when a message is received
|
|
||||||
*/
|
|
||||||
onWebSocketMessage(connectionId: string, callback: (data: unknown) => void): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Removes a callback for incoming WebSocket or WebRTC messages
|
|
||||||
* @param connectionId The identifier of the established connection
|
|
||||||
* @param callback The function to remove from the callbacks
|
|
||||||
*/
|
|
||||||
offWebSocketMessage(connectionId: string, callback: (data: unknown) => void): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Closes an established WebSocket or WebRTC connection
|
|
||||||
* @param connectionId The identifier of the established connection
|
|
||||||
* @param code Optional close code
|
|
||||||
* @param reason Optional close reason
|
|
||||||
*/
|
|
||||||
closeWebSocket(connectionId: string, code?: number, reason?: string): Promise<void>;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Factory function to create the appropriate conduit augmentation based on the type
|
|
||||||
*/
|
|
||||||
export declare function createConduitAugmentation(type: 'websocket' | 'webrtc', name?: string, options?: Record<string, unknown>): Promise<IConduitAugmentation & IWebSocketSupport>;
|
|
||||||
export {};
|
|
||||||
1158
dist/augmentations/conduitAugmentations.js
vendored
1158
dist/augmentations/conduitAugmentations.js
vendored
File diff suppressed because it is too large
Load diff
File diff suppressed because one or more lines are too long
158
dist/augmentations/intelligentVerbScoring.d.ts
vendored
158
dist/augmentations/intelligentVerbScoring.d.ts
vendored
|
|
@ -1,158 +0,0 @@
|
||||||
import { ICognitionAugmentation, AugmentationResponse } from '../types/augmentations.js';
|
|
||||||
/**
|
|
||||||
* Configuration options for the Intelligent Verb Scoring augmentation
|
|
||||||
*/
|
|
||||||
export interface IVerbScoringConfig {
|
|
||||||
/** Enable semantic proximity scoring based on entity embeddings */
|
|
||||||
enableSemanticScoring: boolean;
|
|
||||||
/** Enable frequency-based weight amplification */
|
|
||||||
enableFrequencyAmplification: boolean;
|
|
||||||
/** Enable temporal decay for weights */
|
|
||||||
enableTemporalDecay: boolean;
|
|
||||||
/** Decay rate per day for temporal scoring (0-1) */
|
|
||||||
temporalDecayRate: number;
|
|
||||||
/** Minimum weight threshold */
|
|
||||||
minWeight: number;
|
|
||||||
/** Maximum weight threshold */
|
|
||||||
maxWeight: number;
|
|
||||||
/** Base confidence score for new relationships */
|
|
||||||
baseConfidence: number;
|
|
||||||
/** Learning rate for adaptive scoring (0-1) */
|
|
||||||
learningRate: number;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Default configuration for the Intelligent Verb Scoring augmentation
|
|
||||||
*/
|
|
||||||
export declare const DEFAULT_VERB_SCORING_CONFIG: IVerbScoringConfig;
|
|
||||||
/**
|
|
||||||
* Relationship statistics for learning and adaptation
|
|
||||||
*/
|
|
||||||
interface RelationshipStats {
|
|
||||||
count: number;
|
|
||||||
totalWeight: number;
|
|
||||||
averageWeight: number;
|
|
||||||
lastSeen: Date;
|
|
||||||
firstSeen: Date;
|
|
||||||
semanticSimilarity?: number;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Intelligent Verb Scoring Cognition Augmentation
|
|
||||||
*
|
|
||||||
* Automatically generates intelligent weight and confidence scores for verb relationships
|
|
||||||
* using semantic analysis, frequency patterns, and temporal factors.
|
|
||||||
*/
|
|
||||||
export declare class IntelligentVerbScoring implements ICognitionAugmentation {
|
|
||||||
readonly name = "intelligent-verb-scoring";
|
|
||||||
readonly description = "Automatically generates intelligent weight and confidence scores for verb relationships";
|
|
||||||
enabled: boolean;
|
|
||||||
private config;
|
|
||||||
private relationshipStats;
|
|
||||||
private brainyInstance;
|
|
||||||
private isInitialized;
|
|
||||||
constructor(config?: Partial<IVerbScoringConfig>);
|
|
||||||
initialize(): Promise<void>;
|
|
||||||
shutDown(): Promise<void>;
|
|
||||||
getStatus(): Promise<'active' | 'inactive' | 'error'>;
|
|
||||||
/**
|
|
||||||
* Set reference to the BrainyData instance for accessing graph data
|
|
||||||
*/
|
|
||||||
setBrainyInstance(instance: any): void;
|
|
||||||
/**
|
|
||||||
* Main reasoning method for generating intelligent verb scores
|
|
||||||
*/
|
|
||||||
reason(query: string, context?: Record<string, unknown>): AugmentationResponse<{
|
|
||||||
inference: string;
|
|
||||||
confidence: number;
|
|
||||||
}>;
|
|
||||||
infer(dataSubset: Record<string, unknown>): AugmentationResponse<Record<string, unknown>>;
|
|
||||||
executeLogic(ruleId: string, input: Record<string, unknown>): AugmentationResponse<boolean>;
|
|
||||||
/**
|
|
||||||
* Generate intelligent weight and confidence scores for a verb relationship
|
|
||||||
*
|
|
||||||
* @param sourceId - ID of the source entity
|
|
||||||
* @param targetId - ID of the target entity
|
|
||||||
* @param verbType - Type of the relationship
|
|
||||||
* @param existingWeight - Existing weight if any
|
|
||||||
* @param metadata - Additional metadata about the relationship
|
|
||||||
* @returns Computed weight and confidence scores
|
|
||||||
*/
|
|
||||||
computeVerbScores(sourceId: string, targetId: string, verbType: string, existingWeight?: number, metadata?: any): Promise<{
|
|
||||||
weight: number;
|
|
||||||
confidence: number;
|
|
||||||
reasoning: string[];
|
|
||||||
}>;
|
|
||||||
/**
|
|
||||||
* Calculate semantic similarity between two entities using their embeddings
|
|
||||||
*/
|
|
||||||
private calculateSemanticScore;
|
|
||||||
/**
|
|
||||||
* Calculate frequency-based boost for repeated relationships
|
|
||||||
*/
|
|
||||||
private calculateFrequencyBoost;
|
|
||||||
/**
|
|
||||||
* Calculate temporal decay factor based on recency
|
|
||||||
*/
|
|
||||||
private calculateTemporalFactor;
|
|
||||||
/**
|
|
||||||
* Calculate learning-based adjustment using historical patterns
|
|
||||||
*/
|
|
||||||
private calculateLearningAdjustment;
|
|
||||||
/**
|
|
||||||
* Update relationship statistics for learning
|
|
||||||
*/
|
|
||||||
private updateRelationshipStats;
|
|
||||||
/**
|
|
||||||
* Blend two scores using a weighted average
|
|
||||||
*/
|
|
||||||
private blendScores;
|
|
||||||
/**
|
|
||||||
* Get current configuration
|
|
||||||
*/
|
|
||||||
getConfig(): IVerbScoringConfig;
|
|
||||||
/**
|
|
||||||
* Update configuration
|
|
||||||
*/
|
|
||||||
updateConfig(newConfig: Partial<IVerbScoringConfig>): void;
|
|
||||||
/**
|
|
||||||
* Get relationship statistics (for debugging/monitoring)
|
|
||||||
*/
|
|
||||||
getRelationshipStats(): Map<string, RelationshipStats>;
|
|
||||||
/**
|
|
||||||
* Clear relationship statistics
|
|
||||||
*/
|
|
||||||
clearStats(): void;
|
|
||||||
/**
|
|
||||||
* Provide feedback to improve future scoring
|
|
||||||
* This allows the system to learn from user corrections or validation
|
|
||||||
*
|
|
||||||
* @param sourceId - Source entity ID
|
|
||||||
* @param targetId - Target entity ID
|
|
||||||
* @param verbType - Relationship type
|
|
||||||
* @param feedbackWeight - The corrected/validated weight (0-1)
|
|
||||||
* @param feedbackConfidence - The corrected/validated confidence (0-1)
|
|
||||||
* @param feedbackType - Type of feedback ('correction', 'validation', 'enhancement')
|
|
||||||
*/
|
|
||||||
provideFeedback(sourceId: string, targetId: string, verbType: string, feedbackWeight: number, feedbackConfidence?: number, feedbackType?: 'correction' | 'validation' | 'enhancement'): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Get learning statistics for monitoring and debugging
|
|
||||||
*/
|
|
||||||
getLearningStats(): {
|
|
||||||
totalRelationships: number;
|
|
||||||
averageConfidence: number;
|
|
||||||
feedbackCount: number;
|
|
||||||
topRelationships: Array<{
|
|
||||||
relationship: string;
|
|
||||||
count: number;
|
|
||||||
averageWeight: number;
|
|
||||||
}>;
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Export learning data for backup or analysis
|
|
||||||
*/
|
|
||||||
exportLearningData(): string;
|
|
||||||
/**
|
|
||||||
* Import learning data from backup
|
|
||||||
*/
|
|
||||||
importLearningData(jsonData: string): void;
|
|
||||||
}
|
|
||||||
export {};
|
|
||||||
377
dist/augmentations/intelligentVerbScoring.js
vendored
377
dist/augmentations/intelligentVerbScoring.js
vendored
|
|
@ -1,377 +0,0 @@
|
||||||
import { cosineDistance } from '../utils/distance.js';
|
|
||||||
/**
|
|
||||||
* Default configuration for the Intelligent Verb Scoring augmentation
|
|
||||||
*/
|
|
||||||
export const DEFAULT_VERB_SCORING_CONFIG = {
|
|
||||||
enableSemanticScoring: true,
|
|
||||||
enableFrequencyAmplification: true,
|
|
||||||
enableTemporalDecay: true,
|
|
||||||
temporalDecayRate: 0.01, // 1% decay per day
|
|
||||||
minWeight: 0.1,
|
|
||||||
maxWeight: 1.0,
|
|
||||||
baseConfidence: 0.5,
|
|
||||||
learningRate: 0.1
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Intelligent Verb Scoring Cognition Augmentation
|
|
||||||
*
|
|
||||||
* Automatically generates intelligent weight and confidence scores for verb relationships
|
|
||||||
* using semantic analysis, frequency patterns, and temporal factors.
|
|
||||||
*/
|
|
||||||
export class IntelligentVerbScoring {
|
|
||||||
constructor(config = {}) {
|
|
||||||
this.name = 'intelligent-verb-scoring';
|
|
||||||
this.description = 'Automatically generates intelligent weight and confidence scores for verb relationships';
|
|
||||||
this.enabled = false; // Off by default as requested
|
|
||||||
this.relationshipStats = new Map();
|
|
||||||
this.isInitialized = false;
|
|
||||||
this.config = { ...DEFAULT_VERB_SCORING_CONFIG, ...config };
|
|
||||||
}
|
|
||||||
async initialize() {
|
|
||||||
if (this.isInitialized)
|
|
||||||
return;
|
|
||||||
this.isInitialized = true;
|
|
||||||
}
|
|
||||||
async shutDown() {
|
|
||||||
this.relationshipStats.clear();
|
|
||||||
this.isInitialized = false;
|
|
||||||
}
|
|
||||||
async getStatus() {
|
|
||||||
return this.enabled && this.isInitialized ? 'active' : 'inactive';
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Set reference to the BrainyData instance for accessing graph data
|
|
||||||
*/
|
|
||||||
setBrainyInstance(instance) {
|
|
||||||
this.brainyInstance = instance;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Main reasoning method for generating intelligent verb scores
|
|
||||||
*/
|
|
||||||
reason(query, context) {
|
|
||||||
if (!this.enabled) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: { inference: 'Augmentation is disabled', confidence: 0 },
|
|
||||||
error: 'Intelligent verb scoring is disabled'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: {
|
|
||||||
inference: 'Intelligent verb scoring active',
|
|
||||||
confidence: 1.0
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
infer(dataSubset) {
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: dataSubset
|
|
||||||
};
|
|
||||||
}
|
|
||||||
executeLogic(ruleId, input) {
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: true
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Generate intelligent weight and confidence scores for a verb relationship
|
|
||||||
*
|
|
||||||
* @param sourceId - ID of the source entity
|
|
||||||
* @param targetId - ID of the target entity
|
|
||||||
* @param verbType - Type of the relationship
|
|
||||||
* @param existingWeight - Existing weight if any
|
|
||||||
* @param metadata - Additional metadata about the relationship
|
|
||||||
* @returns Computed weight and confidence scores
|
|
||||||
*/
|
|
||||||
async computeVerbScores(sourceId, targetId, verbType, existingWeight, metadata) {
|
|
||||||
if (!this.enabled || !this.brainyInstance) {
|
|
||||||
return {
|
|
||||||
weight: existingWeight ?? 0.5,
|
|
||||||
confidence: this.config.baseConfidence,
|
|
||||||
reasoning: ['Intelligent scoring disabled']
|
|
||||||
};
|
|
||||||
}
|
|
||||||
const reasoning = [];
|
|
||||||
let weight = existingWeight ?? 0.5;
|
|
||||||
let confidence = this.config.baseConfidence;
|
|
||||||
try {
|
|
||||||
// Get relationship key for statistics
|
|
||||||
const relationKey = `${sourceId}-${verbType}-${targetId}`;
|
|
||||||
// Update relationship statistics
|
|
||||||
this.updateRelationshipStats(relationKey, weight, metadata);
|
|
||||||
// Apply semantic scoring if enabled
|
|
||||||
if (this.config.enableSemanticScoring) {
|
|
||||||
const semanticScore = await this.calculateSemanticScore(sourceId, targetId);
|
|
||||||
if (semanticScore !== null) {
|
|
||||||
weight = this.blendScores(weight, semanticScore, 0.3);
|
|
||||||
confidence = Math.min(confidence + semanticScore * 0.2, 1.0);
|
|
||||||
reasoning.push(`Semantic similarity: ${semanticScore.toFixed(3)}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Apply frequency amplification if enabled
|
|
||||||
if (this.config.enableFrequencyAmplification) {
|
|
||||||
const frequencyBoost = this.calculateFrequencyBoost(relationKey);
|
|
||||||
weight = this.blendScores(weight, frequencyBoost, 0.2);
|
|
||||||
if (frequencyBoost > 0.5) {
|
|
||||||
confidence = Math.min(confidence + 0.1, 1.0);
|
|
||||||
reasoning.push(`Frequency boost: ${frequencyBoost.toFixed(3)}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Apply temporal decay if enabled
|
|
||||||
if (this.config.enableTemporalDecay) {
|
|
||||||
const temporalFactor = this.calculateTemporalFactor(relationKey);
|
|
||||||
weight *= temporalFactor;
|
|
||||||
reasoning.push(`Temporal factor: ${temporalFactor.toFixed(3)}`);
|
|
||||||
}
|
|
||||||
// Apply learning adjustments
|
|
||||||
const learningAdjustment = this.calculateLearningAdjustment(relationKey);
|
|
||||||
weight = this.blendScores(weight, learningAdjustment, this.config.learningRate);
|
|
||||||
// Clamp values to configured bounds
|
|
||||||
weight = Math.max(this.config.minWeight, Math.min(this.config.maxWeight, weight));
|
|
||||||
confidence = Math.max(0, Math.min(1, confidence));
|
|
||||||
reasoning.push(`Final weight: ${weight.toFixed(3)}, confidence: ${confidence.toFixed(3)}`);
|
|
||||||
return { weight, confidence, reasoning };
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.warn('Error computing verb scores:', error);
|
|
||||||
return {
|
|
||||||
weight: existingWeight ?? 0.5,
|
|
||||||
confidence: this.config.baseConfidence,
|
|
||||||
reasoning: [`Error in scoring: ${error}`]
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Calculate semantic similarity between two entities using their embeddings
|
|
||||||
*/
|
|
||||||
async calculateSemanticScore(sourceId, targetId) {
|
|
||||||
try {
|
|
||||||
if (!this.brainyInstance?.storage)
|
|
||||||
return null;
|
|
||||||
// Get noun embeddings from storage
|
|
||||||
const sourceNoun = await this.brainyInstance.storage.getNoun(sourceId);
|
|
||||||
const targetNoun = await this.brainyInstance.storage.getNoun(targetId);
|
|
||||||
if (!sourceNoun?.vector || !targetNoun?.vector)
|
|
||||||
return null;
|
|
||||||
// Calculate cosine similarity (1 - distance)
|
|
||||||
const distance = cosineDistance(sourceNoun.vector, targetNoun.vector);
|
|
||||||
return Math.max(0, 1 - distance);
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.warn('Error calculating semantic score:', error);
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Calculate frequency-based boost for repeated relationships
|
|
||||||
*/
|
|
||||||
calculateFrequencyBoost(relationKey) {
|
|
||||||
const stats = this.relationshipStats.get(relationKey);
|
|
||||||
if (!stats || stats.count <= 1)
|
|
||||||
return 0.5;
|
|
||||||
// Logarithmic scaling: more occurrences = higher weight, but with diminishing returns
|
|
||||||
const boost = Math.log(stats.count + 1) / Math.log(10); // Log base 10
|
|
||||||
return Math.min(boost, 1.0);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Calculate temporal decay factor based on recency
|
|
||||||
*/
|
|
||||||
calculateTemporalFactor(relationKey) {
|
|
||||||
const stats = this.relationshipStats.get(relationKey);
|
|
||||||
if (!stats)
|
|
||||||
return 1.0;
|
|
||||||
const daysSinceLastSeen = (Date.now() - stats.lastSeen.getTime()) / (1000 * 60 * 60 * 24);
|
|
||||||
const decayFactor = Math.exp(-this.config.temporalDecayRate * daysSinceLastSeen);
|
|
||||||
return Math.max(0.1, decayFactor); // Minimum 10% of original weight
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Calculate learning-based adjustment using historical patterns
|
|
||||||
*/
|
|
||||||
calculateLearningAdjustment(relationKey) {
|
|
||||||
const stats = this.relationshipStats.get(relationKey);
|
|
||||||
if (!stats || stats.count <= 1)
|
|
||||||
return 0.5;
|
|
||||||
// Use moving average of weights as learned baseline
|
|
||||||
return Math.max(0, Math.min(1, stats.averageWeight));
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Update relationship statistics for learning
|
|
||||||
*/
|
|
||||||
updateRelationshipStats(relationKey, weight, metadata) {
|
|
||||||
const now = new Date();
|
|
||||||
const existing = this.relationshipStats.get(relationKey);
|
|
||||||
if (existing) {
|
|
||||||
// Update existing stats
|
|
||||||
existing.count++;
|
|
||||||
existing.totalWeight += weight;
|
|
||||||
existing.averageWeight = existing.totalWeight / existing.count;
|
|
||||||
existing.lastSeen = now;
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
// Create new stats entry
|
|
||||||
this.relationshipStats.set(relationKey, {
|
|
||||||
count: 1,
|
|
||||||
totalWeight: weight,
|
|
||||||
averageWeight: weight,
|
|
||||||
lastSeen: now,
|
|
||||||
firstSeen: now
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Blend two scores using a weighted average
|
|
||||||
*/
|
|
||||||
blendScores(score1, score2, weight2) {
|
|
||||||
const weight1 = 1 - weight2;
|
|
||||||
return score1 * weight1 + score2 * weight2;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get current configuration
|
|
||||||
*/
|
|
||||||
getConfig() {
|
|
||||||
return { ...this.config };
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Update configuration
|
|
||||||
*/
|
|
||||||
updateConfig(newConfig) {
|
|
||||||
this.config = { ...this.config, ...newConfig };
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get relationship statistics (for debugging/monitoring)
|
|
||||||
*/
|
|
||||||
getRelationshipStats() {
|
|
||||||
return new Map(this.relationshipStats);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Clear relationship statistics
|
|
||||||
*/
|
|
||||||
clearStats() {
|
|
||||||
this.relationshipStats.clear();
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Provide feedback to improve future scoring
|
|
||||||
* This allows the system to learn from user corrections or validation
|
|
||||||
*
|
|
||||||
* @param sourceId - Source entity ID
|
|
||||||
* @param targetId - Target entity ID
|
|
||||||
* @param verbType - Relationship type
|
|
||||||
* @param feedbackWeight - The corrected/validated weight (0-1)
|
|
||||||
* @param feedbackConfidence - The corrected/validated confidence (0-1)
|
|
||||||
* @param feedbackType - Type of feedback ('correction', 'validation', 'enhancement')
|
|
||||||
*/
|
|
||||||
async provideFeedback(sourceId, targetId, verbType, feedbackWeight, feedbackConfidence, feedbackType = 'correction') {
|
|
||||||
if (!this.enabled)
|
|
||||||
return;
|
|
||||||
const relationKey = `${sourceId}-${verbType}-${targetId}`;
|
|
||||||
const existing = this.relationshipStats.get(relationKey);
|
|
||||||
if (existing) {
|
|
||||||
// Apply feedback with learning rate
|
|
||||||
const newWeight = existing.averageWeight * (1 - this.config.learningRate) +
|
|
||||||
feedbackWeight * this.config.learningRate;
|
|
||||||
// Update the running average with feedback
|
|
||||||
existing.totalWeight = (existing.totalWeight * existing.count + feedbackWeight) / (existing.count + 1);
|
|
||||||
existing.averageWeight = existing.totalWeight / existing.count;
|
|
||||||
existing.count += 1;
|
|
||||||
existing.lastSeen = new Date();
|
|
||||||
if (this.brainyInstance?.loggingConfig?.verbose) {
|
|
||||||
console.log(`Feedback applied for ${relationKey}: ${feedbackType}, ` +
|
|
||||||
`old weight: ${existing.averageWeight.toFixed(3)}, ` +
|
|
||||||
`feedback: ${feedbackWeight.toFixed(3)}, ` +
|
|
||||||
`new weight: ${newWeight.toFixed(3)}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
// Create new entry with feedback as initial data
|
|
||||||
this.relationshipStats.set(relationKey, {
|
|
||||||
count: 1,
|
|
||||||
totalWeight: feedbackWeight,
|
|
||||||
averageWeight: feedbackWeight,
|
|
||||||
lastSeen: new Date(),
|
|
||||||
firstSeen: new Date()
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get learning statistics for monitoring and debugging
|
|
||||||
*/
|
|
||||||
getLearningStats() {
|
|
||||||
const relationships = Array.from(this.relationshipStats.entries());
|
|
||||||
const totalRelationships = relationships.length;
|
|
||||||
const feedbackCount = relationships.reduce((sum, [, stats]) => sum + stats.count, 0);
|
|
||||||
// Calculate average confidence (approximated from weight patterns)
|
|
||||||
const averageWeight = relationships.reduce((sum, [, stats]) => sum + stats.averageWeight, 0) / totalRelationships || 0;
|
|
||||||
const averageConfidence = Math.min(averageWeight + 0.2, 1.0); // Heuristic: confidence typically higher than weight
|
|
||||||
// Get top relationships by count
|
|
||||||
const topRelationships = relationships
|
|
||||||
.map(([key, stats]) => ({
|
|
||||||
relationship: key,
|
|
||||||
count: stats.count,
|
|
||||||
averageWeight: stats.averageWeight
|
|
||||||
}))
|
|
||||||
.sort((a, b) => b.count - a.count)
|
|
||||||
.slice(0, 10);
|
|
||||||
return {
|
|
||||||
totalRelationships,
|
|
||||||
averageConfidence,
|
|
||||||
feedbackCount,
|
|
||||||
topRelationships
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Export learning data for backup or analysis
|
|
||||||
*/
|
|
||||||
exportLearningData() {
|
|
||||||
const data = {
|
|
||||||
config: this.config,
|
|
||||||
stats: Array.from(this.relationshipStats.entries()).map(([key, stats]) => ({
|
|
||||||
relationship: key,
|
|
||||||
...stats,
|
|
||||||
firstSeen: stats.firstSeen.toISOString(),
|
|
||||||
lastSeen: stats.lastSeen.toISOString()
|
|
||||||
})),
|
|
||||||
exportedAt: new Date().toISOString(),
|
|
||||||
version: '1.0'
|
|
||||||
};
|
|
||||||
return JSON.stringify(data, null, 2);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Import learning data from backup
|
|
||||||
*/
|
|
||||||
importLearningData(jsonData) {
|
|
||||||
try {
|
|
||||||
const data = JSON.parse(jsonData);
|
|
||||||
if (data.version !== '1.0') {
|
|
||||||
console.warn('Learning data version mismatch, importing anyway');
|
|
||||||
}
|
|
||||||
// Update configuration if provided
|
|
||||||
if (data.config) {
|
|
||||||
this.config = { ...this.config, ...data.config };
|
|
||||||
}
|
|
||||||
// Import relationship statistics
|
|
||||||
if (data.stats && Array.isArray(data.stats)) {
|
|
||||||
for (const stat of data.stats) {
|
|
||||||
if (stat.relationship) {
|
|
||||||
this.relationshipStats.set(stat.relationship, {
|
|
||||||
count: stat.count || 1,
|
|
||||||
totalWeight: stat.totalWeight || stat.averageWeight || 0.5,
|
|
||||||
averageWeight: stat.averageWeight || 0.5,
|
|
||||||
firstSeen: new Date(stat.firstSeen || Date.now()),
|
|
||||||
lastSeen: new Date(stat.lastSeen || Date.now()),
|
|
||||||
semanticSimilarity: stat.semanticSimilarity
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
console.log(`Imported learning data: ${this.relationshipStats.size} relationships`);
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Failed to import learning data:', error);
|
|
||||||
throw new Error(`Failed to import learning data: ${error}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=intelligentVerbScoring.js.map
|
|
||||||
File diff suppressed because one or more lines are too long
72
dist/augmentations/memoryAugmentations.d.ts
vendored
72
dist/augmentations/memoryAugmentations.d.ts
vendored
|
|
@ -1,72 +0,0 @@
|
||||||
import { AugmentationType, IMemoryAugmentation, AugmentationResponse } from '../types/augmentations.js';
|
|
||||||
import { StorageAdapter } from '../coreTypes.js';
|
|
||||||
/**
|
|
||||||
* Base class for memory augmentations that wrap a StorageAdapter
|
|
||||||
*/
|
|
||||||
declare abstract class BaseMemoryAugmentation implements IMemoryAugmentation {
|
|
||||||
readonly name: string;
|
|
||||||
readonly description: string;
|
|
||||||
enabled: boolean;
|
|
||||||
protected storage: StorageAdapter;
|
|
||||||
protected isInitialized: boolean;
|
|
||||||
constructor(name: string, storage: StorageAdapter);
|
|
||||||
initialize(): Promise<void>;
|
|
||||||
shutDown(): Promise<void>;
|
|
||||||
getStatus(): Promise<'active' | 'inactive' | 'error'>;
|
|
||||||
storeData(key: string, data: unknown, options?: Record<string, unknown>): Promise<AugmentationResponse<boolean>>;
|
|
||||||
retrieveData(key: string, options?: Record<string, unknown>): Promise<AugmentationResponse<unknown>>;
|
|
||||||
updateData(key: string, data: unknown, options?: Record<string, unknown>): Promise<AugmentationResponse<boolean>>;
|
|
||||||
deleteData(key: string, options?: Record<string, unknown>): Promise<AugmentationResponse<boolean>>;
|
|
||||||
listDataKeys(pattern?: string, options?: Record<string, unknown>): Promise<AugmentationResponse<string[]>>;
|
|
||||||
/**
|
|
||||||
* Searches for data in the storage using vector similarity.
|
|
||||||
* Implements the findNearest functionality by calculating distances client-side.
|
|
||||||
* @param query The query vector or data to search for
|
|
||||||
* @param k Number of results to return (default: 10)
|
|
||||||
* @param options Optional search options
|
|
||||||
*/
|
|
||||||
search(query: unknown, k?: number, options?: Record<string, unknown>): Promise<AugmentationResponse<Array<{
|
|
||||||
id: string;
|
|
||||||
score: number;
|
|
||||||
data: unknown;
|
|
||||||
}>>>;
|
|
||||||
protected ensureInitialized(): Promise<void>;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Memory augmentation that uses in-memory storage
|
|
||||||
*/
|
|
||||||
export declare class MemoryStorageAugmentation extends BaseMemoryAugmentation {
|
|
||||||
readonly description = "Memory augmentation that stores data in memory";
|
|
||||||
enabled: boolean;
|
|
||||||
constructor(name: string);
|
|
||||||
getType(): AugmentationType;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Memory augmentation that uses file system storage
|
|
||||||
*/
|
|
||||||
export declare class FileSystemStorageAugmentation extends BaseMemoryAugmentation {
|
|
||||||
readonly description = "Memory augmentation that stores data in the file system";
|
|
||||||
enabled: boolean;
|
|
||||||
private rootDirectory;
|
|
||||||
constructor(name: string, rootDirectory?: string);
|
|
||||||
initialize(): Promise<void>;
|
|
||||||
getType(): AugmentationType;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Memory augmentation that uses OPFS (Origin Private File System) storage
|
|
||||||
*/
|
|
||||||
export declare class OPFSStorageAugmentation extends BaseMemoryAugmentation {
|
|
||||||
readonly description = "Memory augmentation that stores data in the Origin Private File System";
|
|
||||||
enabled: boolean;
|
|
||||||
constructor(name: string);
|
|
||||||
getType(): AugmentationType;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Factory function to create the appropriate memory augmentation based on the environment
|
|
||||||
*/
|
|
||||||
export declare function createMemoryAugmentation(name: string, options?: {
|
|
||||||
storageType?: 'memory' | 'filesystem' | 'opfs';
|
|
||||||
rootDirectory?: string;
|
|
||||||
requestPersistentStorage?: boolean;
|
|
||||||
}): Promise<IMemoryAugmentation>;
|
|
||||||
export {};
|
|
||||||
280
dist/augmentations/memoryAugmentations.js
vendored
280
dist/augmentations/memoryAugmentations.js
vendored
|
|
@ -1,280 +0,0 @@
|
||||||
import { AugmentationType } from '../types/augmentations.js';
|
|
||||||
import { MemoryStorage, OPFSStorage } from '../storage/storageFactory.js';
|
|
||||||
// FileSystemStorage will be dynamically imported when needed to avoid fs imports in browser
|
|
||||||
import { cosineDistance } from '../utils/distance.js';
|
|
||||||
/**
|
|
||||||
* Base class for memory augmentations that wrap a StorageAdapter
|
|
||||||
*/
|
|
||||||
class BaseMemoryAugmentation {
|
|
||||||
constructor(name, storage) {
|
|
||||||
this.description = 'Base memory augmentation';
|
|
||||||
this.enabled = true;
|
|
||||||
this.isInitialized = false;
|
|
||||||
this.name = name;
|
|
||||||
this.storage = storage;
|
|
||||||
}
|
|
||||||
async initialize() {
|
|
||||||
if (this.isInitialized) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
await this.storage.init();
|
|
||||||
this.isInitialized = true;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error(`Failed to initialize ${this.name}:`, error);
|
|
||||||
throw new Error(`Failed to initialize ${this.name}: ${error}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async shutDown() {
|
|
||||||
this.isInitialized = false;
|
|
||||||
}
|
|
||||||
async getStatus() {
|
|
||||||
return this.isInitialized ? 'active' : 'inactive';
|
|
||||||
}
|
|
||||||
async storeData(key, data, options) {
|
|
||||||
await this.ensureInitialized();
|
|
||||||
try {
|
|
||||||
await this.storage.saveMetadata(key, data);
|
|
||||||
return { success: true, data: true };
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error(`Failed to store data for key ${key}:`, error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: false,
|
|
||||||
error: `Failed to store data: ${error}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async retrieveData(key, options) {
|
|
||||||
await this.ensureInitialized();
|
|
||||||
try {
|
|
||||||
const data = await this.storage.getMetadata(key);
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error(`Failed to retrieve data for key ${key}:`, error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: `Failed to retrieve data: ${error}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async updateData(key, data, options) {
|
|
||||||
await this.ensureInitialized();
|
|
||||||
try {
|
|
||||||
await this.storage.saveMetadata(key, data);
|
|
||||||
return { success: true, data: true };
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error(`Failed to update data for key ${key}:`, error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: false,
|
|
||||||
error: `Failed to update data: ${error}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async deleteData(key, options) {
|
|
||||||
await this.ensureInitialized();
|
|
||||||
try {
|
|
||||||
// There's no direct deleteMetadata method, so we save null
|
|
||||||
await this.storage.saveMetadata(key, null);
|
|
||||||
return { success: true, data: true };
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error(`Failed to delete data for key ${key}:`, error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: false,
|
|
||||||
error: `Failed to delete data: ${error}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async listDataKeys(pattern, options) {
|
|
||||||
// This is a limitation of the current StorageAdapter interface
|
|
||||||
// It doesn't provide a way to list all metadata keys
|
|
||||||
// We could implement this in the future by extending the StorageAdapter interface
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: [],
|
|
||||||
error: 'listDataKeys is not supported by this storage adapter'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Searches for data in the storage using vector similarity.
|
|
||||||
* Implements the findNearest functionality by calculating distances client-side.
|
|
||||||
* @param query The query vector or data to search for
|
|
||||||
* @param k Number of results to return (default: 10)
|
|
||||||
* @param options Optional search options
|
|
||||||
*/
|
|
||||||
async search(query, k = 10, options) {
|
|
||||||
await this.ensureInitialized();
|
|
||||||
try {
|
|
||||||
// Check if query is a vector
|
|
||||||
let queryVector;
|
|
||||||
if (Array.isArray(query) && query.every(item => typeof item === 'number')) {
|
|
||||||
queryVector = query;
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
// If query is not a vector, we can't perform vector search
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: [],
|
|
||||||
error: 'Query must be a vector (array of numbers) for vector search'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// Process nodes in batches to avoid loading everything into memory
|
|
||||||
const allResults = [];
|
|
||||||
let hasMore = true;
|
|
||||||
let cursor;
|
|
||||||
while (hasMore) {
|
|
||||||
// Get a batch of nodes
|
|
||||||
const batchResult = await this.storage.getNouns({
|
|
||||||
pagination: { limit: 100, cursor }
|
|
||||||
});
|
|
||||||
// Process this batch
|
|
||||||
for (const noun of batchResult.items) {
|
|
||||||
// Skip nodes that don't have a vector
|
|
||||||
if (!noun.vector || !Array.isArray(noun.vector)) {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
// Get metadata for the node
|
|
||||||
const metadata = await this.storage.getMetadata(noun.id);
|
|
||||||
// Calculate distance between query vector and node vector
|
|
||||||
const distance = cosineDistance(queryVector, noun.vector);
|
|
||||||
// Convert distance to similarity score (1 - distance for cosine)
|
|
||||||
// This way higher scores are better (more similar)
|
|
||||||
const score = 1 - distance;
|
|
||||||
allResults.push({
|
|
||||||
id: noun.id,
|
|
||||||
score,
|
|
||||||
data: metadata
|
|
||||||
});
|
|
||||||
}
|
|
||||||
// Update pagination state
|
|
||||||
hasMore = batchResult.hasMore;
|
|
||||||
cursor = batchResult.nextCursor;
|
|
||||||
}
|
|
||||||
// Sort results by score (descending) and take top k
|
|
||||||
allResults.sort((a, b) => b.score - a.score);
|
|
||||||
const topResults = allResults.slice(0, k);
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: topResults
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error(`Failed to search in storage:`, error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: [],
|
|
||||||
error: `Failed to search in storage: ${error}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async ensureInitialized() {
|
|
||||||
if (!this.isInitialized) {
|
|
||||||
await this.initialize();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Memory augmentation that uses in-memory storage
|
|
||||||
*/
|
|
||||||
export class MemoryStorageAugmentation extends BaseMemoryAugmentation {
|
|
||||||
constructor(name) {
|
|
||||||
super(name, new MemoryStorage());
|
|
||||||
this.description = 'Memory augmentation that stores data in memory';
|
|
||||||
this.enabled = true;
|
|
||||||
}
|
|
||||||
getType() {
|
|
||||||
return AugmentationType.MEMORY;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Memory augmentation that uses file system storage
|
|
||||||
*/
|
|
||||||
export class FileSystemStorageAugmentation extends BaseMemoryAugmentation {
|
|
||||||
constructor(name, rootDirectory) {
|
|
||||||
// Temporarily use MemoryStorage, will be replaced in initialize()
|
|
||||||
super(name, new MemoryStorage());
|
|
||||||
this.description = 'Memory augmentation that stores data in the file system';
|
|
||||||
this.enabled = true;
|
|
||||||
this.rootDirectory = rootDirectory || '.';
|
|
||||||
}
|
|
||||||
async initialize() {
|
|
||||||
try {
|
|
||||||
// Dynamically import FileSystemStorage
|
|
||||||
const { FileSystemStorage } = await import('../storage/adapters/fileSystemStorage.js');
|
|
||||||
this.storage = new FileSystemStorage(this.rootDirectory);
|
|
||||||
await super.initialize();
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Failed to load FileSystemStorage:', error);
|
|
||||||
throw new Error(`Failed to initialize FileSystemStorage: ${error}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
getType() {
|
|
||||||
return AugmentationType.MEMORY;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Memory augmentation that uses OPFS (Origin Private File System) storage
|
|
||||||
*/
|
|
||||||
export class OPFSStorageAugmentation extends BaseMemoryAugmentation {
|
|
||||||
constructor(name) {
|
|
||||||
super(name, new OPFSStorage());
|
|
||||||
this.description = 'Memory augmentation that stores data in the Origin Private File System';
|
|
||||||
this.enabled = true;
|
|
||||||
}
|
|
||||||
getType() {
|
|
||||||
return AugmentationType.MEMORY;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Factory function to create the appropriate memory augmentation based on the environment
|
|
||||||
*/
|
|
||||||
export async function createMemoryAugmentation(name, options = {}) {
|
|
||||||
// If a specific storage type is requested, use that
|
|
||||||
if (options.storageType) {
|
|
||||||
switch (options.storageType) {
|
|
||||||
case 'memory':
|
|
||||||
return new MemoryStorageAugmentation(name);
|
|
||||||
case 'filesystem':
|
|
||||||
return new FileSystemStorageAugmentation(name, options.rootDirectory);
|
|
||||||
case 'opfs':
|
|
||||||
return new OPFSStorageAugmentation(name);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Otherwise, select based on environment
|
|
||||||
// Use the global isNode variable from the environment detection
|
|
||||||
const isNodeEnv = globalThis.__ENV__?.isNode || (typeof process !== 'undefined' &&
|
|
||||||
process.versions != null &&
|
|
||||||
process.versions.node != null);
|
|
||||||
if (isNodeEnv) {
|
|
||||||
// In Node.js, use FileSystemStorage
|
|
||||||
return new FileSystemStorageAugmentation(name, options.rootDirectory);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
// In browser, try OPFS first
|
|
||||||
const opfsStorage = new OPFSStorage();
|
|
||||||
if (opfsStorage.isOPFSAvailable()) {
|
|
||||||
// Request persistent storage if specified
|
|
||||||
if (options.requestPersistentStorage) {
|
|
||||||
await opfsStorage.requestPersistentStorage();
|
|
||||||
}
|
|
||||||
return new OPFSStorageAugmentation(name);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
// Fall back to memory storage
|
|
||||||
return new MemoryStorageAugmentation(name);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=memoryAugmentations.js.map
|
|
||||||
File diff suppressed because one or more lines are too long
199
dist/augmentations/neuralImport.d.ts
vendored
199
dist/augmentations/neuralImport.d.ts
vendored
|
|
@ -1,199 +0,0 @@
|
||||||
/**
|
|
||||||
* Neural Import Augmentation - AI-Powered Data Understanding
|
|
||||||
*
|
|
||||||
* 🧠 Built-in AI augmentation for intelligent data processing
|
|
||||||
* ⚛️ Always free, always included, always enabled
|
|
||||||
*
|
|
||||||
* This is the default AI-powered augmentation that comes with every Brainy installation.
|
|
||||||
* It provides intelligent data understanding, entity detection, and relationship analysis.
|
|
||||||
*/
|
|
||||||
import { ISenseAugmentation, AugmentationResponse } from '../types/augmentations.js';
|
|
||||||
import { BrainyData } from '../brainyData.js';
|
|
||||||
export interface NeuralAnalysisResult {
|
|
||||||
detectedEntities: DetectedEntity[];
|
|
||||||
detectedRelationships: DetectedRelationship[];
|
|
||||||
confidence: number;
|
|
||||||
insights: NeuralInsight[];
|
|
||||||
}
|
|
||||||
export interface DetectedEntity {
|
|
||||||
originalData: any;
|
|
||||||
nounType: string;
|
|
||||||
confidence: number;
|
|
||||||
suggestedId: string;
|
|
||||||
reasoning: string;
|
|
||||||
alternativeTypes: Array<{
|
|
||||||
type: string;
|
|
||||||
confidence: number;
|
|
||||||
}>;
|
|
||||||
}
|
|
||||||
export interface DetectedRelationship {
|
|
||||||
sourceId: string;
|
|
||||||
targetId: string;
|
|
||||||
verbType: string;
|
|
||||||
confidence: number;
|
|
||||||
weight: number;
|
|
||||||
reasoning: string;
|
|
||||||
context: string;
|
|
||||||
metadata?: Record<string, any>;
|
|
||||||
}
|
|
||||||
export interface NeuralInsight {
|
|
||||||
type: 'hierarchy' | 'cluster' | 'pattern' | 'anomaly' | 'opportunity';
|
|
||||||
description: string;
|
|
||||||
confidence: number;
|
|
||||||
affectedEntities: string[];
|
|
||||||
recommendation?: string;
|
|
||||||
}
|
|
||||||
export interface NeuralImportConfig {
|
|
||||||
confidenceThreshold: number;
|
|
||||||
enableWeights: boolean;
|
|
||||||
skipDuplicates: boolean;
|
|
||||||
categoryFilter?: string[];
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Neural Import SENSE Augmentation - The Brain's Perceptual System
|
|
||||||
*/
|
|
||||||
export declare class NeuralImportAugmentation implements ISenseAugmentation {
|
|
||||||
readonly name: string;
|
|
||||||
readonly description: string;
|
|
||||||
enabled: boolean;
|
|
||||||
private brainy;
|
|
||||||
private config;
|
|
||||||
constructor(brainy: BrainyData, config?: Partial<NeuralImportConfig>);
|
|
||||||
initialize(): Promise<void>;
|
|
||||||
shutDown(): Promise<void>;
|
|
||||||
getStatus(): Promise<'active' | 'inactive' | 'error'>;
|
|
||||||
/**
|
|
||||||
* Process raw data into structured nouns and verbs using neural analysis
|
|
||||||
*/
|
|
||||||
processRawData(rawData: Buffer | string, dataType: string, options?: Record<string, unknown>): Promise<AugmentationResponse<{
|
|
||||||
nouns: string[];
|
|
||||||
verbs: string[];
|
|
||||||
confidence?: number;
|
|
||||||
insights?: Array<{
|
|
||||||
type: string;
|
|
||||||
description: string;
|
|
||||||
confidence: number;
|
|
||||||
}>;
|
|
||||||
metadata?: Record<string, unknown>;
|
|
||||||
}>>;
|
|
||||||
/**
|
|
||||||
* Listen to real-time data feeds and process them
|
|
||||||
*/
|
|
||||||
listenToFeed(feedUrl: string, callback: (data: {
|
|
||||||
nouns: string[];
|
|
||||||
verbs: string[];
|
|
||||||
confidence?: number;
|
|
||||||
}) => void): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Analyze data structure without processing (preview mode)
|
|
||||||
*/
|
|
||||||
analyzeStructure(rawData: Buffer | string, dataType: string, options?: Record<string, unknown>): Promise<AugmentationResponse<{
|
|
||||||
entityTypes: Array<{
|
|
||||||
type: string;
|
|
||||||
count: number;
|
|
||||||
confidence: number;
|
|
||||||
}>;
|
|
||||||
relationshipTypes: Array<{
|
|
||||||
type: string;
|
|
||||||
count: number;
|
|
||||||
confidence: number;
|
|
||||||
}>;
|
|
||||||
dataQuality: {
|
|
||||||
completeness: number;
|
|
||||||
consistency: number;
|
|
||||||
accuracy: number;
|
|
||||||
};
|
|
||||||
recommendations: string[];
|
|
||||||
}>>;
|
|
||||||
/**
|
|
||||||
* Validate data compatibility with current knowledge base
|
|
||||||
*/
|
|
||||||
validateCompatibility(rawData: Buffer | string, dataType: string): Promise<AugmentationResponse<{
|
|
||||||
compatible: boolean;
|
|
||||||
issues: Array<{
|
|
||||||
type: string;
|
|
||||||
description: string;
|
|
||||||
severity: 'low' | 'medium' | 'high';
|
|
||||||
}>;
|
|
||||||
suggestions: string[];
|
|
||||||
}>>;
|
|
||||||
/**
|
|
||||||
* Get the full neural analysis result (custom method for Cortex integration)
|
|
||||||
*/
|
|
||||||
getNeuralAnalysis(rawData: Buffer | string, dataType: string): Promise<NeuralAnalysisResult>;
|
|
||||||
/**
|
|
||||||
* Parse raw data based on type
|
|
||||||
*/
|
|
||||||
private parseRawData;
|
|
||||||
/**
|
|
||||||
* Basic CSV parser
|
|
||||||
*/
|
|
||||||
private parseCSV;
|
|
||||||
/**
|
|
||||||
* Perform neural analysis on parsed data
|
|
||||||
*/
|
|
||||||
private performNeuralAnalysis;
|
|
||||||
/**
|
|
||||||
* Neural Entity Detection - The Core AI Engine
|
|
||||||
*/
|
|
||||||
private detectEntitiesWithNeuralAnalysis;
|
|
||||||
/**
|
|
||||||
* Calculate entity type confidence using AI
|
|
||||||
*/
|
|
||||||
private calculateEntityTypeConfidence;
|
|
||||||
/**
|
|
||||||
* Field-based confidence calculation
|
|
||||||
*/
|
|
||||||
private calculateFieldBasedConfidence;
|
|
||||||
/**
|
|
||||||
* Pattern-based confidence calculation
|
|
||||||
*/
|
|
||||||
private calculatePatternBasedConfidence;
|
|
||||||
/**
|
|
||||||
* Generate reasoning for entity type selection
|
|
||||||
*/
|
|
||||||
private generateEntityReasoning;
|
|
||||||
/**
|
|
||||||
* Neural Relationship Detection
|
|
||||||
*/
|
|
||||||
private detectRelationshipsWithNeuralAnalysis;
|
|
||||||
/**
|
|
||||||
* Calculate relationship confidence
|
|
||||||
*/
|
|
||||||
private calculateRelationshipConfidence;
|
|
||||||
/**
|
|
||||||
* Calculate relationship weight/strength
|
|
||||||
*/
|
|
||||||
private calculateRelationshipWeight;
|
|
||||||
/**
|
|
||||||
* Generate Neural Insights - The Intelligence Layer
|
|
||||||
*/
|
|
||||||
private generateNeuralInsights;
|
|
||||||
/**
|
|
||||||
* Helper methods for the neural system
|
|
||||||
*/
|
|
||||||
private extractMainText;
|
|
||||||
private generateSmartId;
|
|
||||||
private extractRelationshipContext;
|
|
||||||
private calculateTypeCompatibility;
|
|
||||||
private getVerbSpecificity;
|
|
||||||
private getRelevantFields;
|
|
||||||
private getMatchedPatterns;
|
|
||||||
private pruneRelationships;
|
|
||||||
private detectHierarchies;
|
|
||||||
private detectClusters;
|
|
||||||
private detectPatterns;
|
|
||||||
private calculateOverallConfidence;
|
|
||||||
private storeNeuralAnalysis;
|
|
||||||
private getDataTypeFromPath;
|
|
||||||
private generateRelationshipReasoning;
|
|
||||||
private extractRelationshipMetadata;
|
|
||||||
/**
|
|
||||||
* Assess data quality metrics
|
|
||||||
*/
|
|
||||||
private assessDataQuality;
|
|
||||||
/**
|
|
||||||
* Generate recommendations based on analysis
|
|
||||||
*/
|
|
||||||
private generateRecommendations;
|
|
||||||
}
|
|
||||||
750
dist/augmentations/neuralImport.js
vendored
750
dist/augmentations/neuralImport.js
vendored
|
|
@ -1,750 +0,0 @@
|
||||||
/**
|
|
||||||
* Neural Import Augmentation - AI-Powered Data Understanding
|
|
||||||
*
|
|
||||||
* 🧠 Built-in AI augmentation for intelligent data processing
|
|
||||||
* ⚛️ Always free, always included, always enabled
|
|
||||||
*
|
|
||||||
* This is the default AI-powered augmentation that comes with every Brainy installation.
|
|
||||||
* It provides intelligent data understanding, entity detection, and relationship analysis.
|
|
||||||
*/
|
|
||||||
import { NounType, VerbType } from '../types/graphTypes.js';
|
|
||||||
import * as fs from '../universal/fs.js';
|
|
||||||
import * as path from '../universal/path.js';
|
|
||||||
/**
|
|
||||||
* Neural Import SENSE Augmentation - The Brain's Perceptual System
|
|
||||||
*/
|
|
||||||
export class NeuralImportAugmentation {
|
|
||||||
constructor(brainy, config = {}) {
|
|
||||||
this.name = 'neural-import';
|
|
||||||
this.description = 'Built-in AI-powered data understanding and entity detection';
|
|
||||||
this.enabled = true;
|
|
||||||
this.brainy = brainy;
|
|
||||||
this.config = {
|
|
||||||
confidenceThreshold: 0.7,
|
|
||||||
enableWeights: true,
|
|
||||||
skipDuplicates: true,
|
|
||||||
...config
|
|
||||||
};
|
|
||||||
}
|
|
||||||
async initialize() {
|
|
||||||
// Initialize the cortex analysis system
|
|
||||||
console.log('🧠 Neural Import augmentation initialized');
|
|
||||||
}
|
|
||||||
async shutDown() {
|
|
||||||
console.log('🧠 Neural Import SENSE augmentation shut down');
|
|
||||||
}
|
|
||||||
async getStatus() {
|
|
||||||
return this.enabled ? 'active' : 'inactive';
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Process raw data into structured nouns and verbs using neural analysis
|
|
||||||
*/
|
|
||||||
async processRawData(rawData, dataType, options) {
|
|
||||||
try {
|
|
||||||
// Merge options with config
|
|
||||||
const mergedConfig = { ...this.config, ...options };
|
|
||||||
// Parse the raw data based on type
|
|
||||||
const parsedData = await this.parseRawData(rawData, dataType);
|
|
||||||
// Perform neural analysis
|
|
||||||
const analysis = await this.performNeuralAnalysis(parsedData, mergedConfig);
|
|
||||||
// Extract nouns and verbs for the ISenseAugmentation interface
|
|
||||||
const nouns = analysis.detectedEntities.map(entity => entity.suggestedId);
|
|
||||||
const verbs = analysis.detectedRelationships.map(rel => `${rel.sourceId}->${rel.verbType}->${rel.targetId}`);
|
|
||||||
// Store the full analysis for later retrieval
|
|
||||||
await this.storeNeuralAnalysis(analysis);
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: {
|
|
||||||
nouns,
|
|
||||||
verbs,
|
|
||||||
confidence: analysis.confidence,
|
|
||||||
insights: analysis.insights.map((insight) => ({
|
|
||||||
type: insight.type,
|
|
||||||
description: insight.description,
|
|
||||||
confidence: insight.confidence
|
|
||||||
})),
|
|
||||||
metadata: {
|
|
||||||
detectedEntities: analysis.detectedEntities.length,
|
|
||||||
detectedRelationships: analysis.detectedRelationships.length,
|
|
||||||
timestamp: new Date().toISOString(),
|
|
||||||
augmentation: 'neural-import-sense'
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: { nouns: [], verbs: [] },
|
|
||||||
error: error instanceof Error ? error.message : 'Neural analysis failed'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Listen to real-time data feeds and process them
|
|
||||||
*/
|
|
||||||
async listenToFeed(feedUrl, callback) {
|
|
||||||
// For file-based feeds, watch for changes
|
|
||||||
if (feedUrl.startsWith('file://')) {
|
|
||||||
const filePath = feedUrl.replace('file://', '');
|
|
||||||
// Watch file for changes using Node.js fs.watch
|
|
||||||
const fsWatch = require('fs');
|
|
||||||
const watcher = fsWatch.watch(filePath, async (eventType) => {
|
|
||||||
if (eventType === 'change') {
|
|
||||||
try {
|
|
||||||
const fileContent = await fs.readFile(filePath);
|
|
||||||
const result = await this.processRawData(fileContent, this.getDataTypeFromPath(filePath));
|
|
||||||
if (result.success) {
|
|
||||||
callback({
|
|
||||||
nouns: result.data.nouns,
|
|
||||||
verbs: result.data.verbs,
|
|
||||||
confidence: result.data.confidence
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Neural Import feed error:', error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
// For other feed types, implement appropriate listeners
|
|
||||||
console.log(`🧠 Neural Import listening to feed: ${feedUrl}`);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Analyze data structure without processing (preview mode)
|
|
||||||
*/
|
|
||||||
async analyzeStructure(rawData, dataType, options) {
|
|
||||||
try {
|
|
||||||
// Parse the raw data
|
|
||||||
const parsedData = await this.parseRawData(rawData, dataType);
|
|
||||||
// Perform lightweight analysis for structure detection
|
|
||||||
const analysis = await this.performNeuralAnalysis(parsedData, { ...this.config, ...options });
|
|
||||||
// Summarize entity types
|
|
||||||
const entityTypeCounts = new Map();
|
|
||||||
analysis.detectedEntities.forEach(entity => {
|
|
||||||
const existing = entityTypeCounts.get(entity.nounType) || { count: 0, totalConfidence: 0 };
|
|
||||||
entityTypeCounts.set(entity.nounType, {
|
|
||||||
count: existing.count + 1,
|
|
||||||
totalConfidence: existing.totalConfidence + entity.confidence
|
|
||||||
});
|
|
||||||
});
|
|
||||||
const entityTypes = Array.from(entityTypeCounts.entries()).map(([type, stats]) => ({
|
|
||||||
type,
|
|
||||||
count: stats.count,
|
|
||||||
confidence: stats.totalConfidence / stats.count
|
|
||||||
}));
|
|
||||||
// Summarize relationship types
|
|
||||||
const relationshipTypeCounts = new Map();
|
|
||||||
analysis.detectedRelationships.forEach(rel => {
|
|
||||||
const existing = relationshipTypeCounts.get(rel.verbType) || { count: 0, totalConfidence: 0 };
|
|
||||||
relationshipTypeCounts.set(rel.verbType, {
|
|
||||||
count: existing.count + 1,
|
|
||||||
totalConfidence: existing.totalConfidence + rel.confidence
|
|
||||||
});
|
|
||||||
});
|
|
||||||
const relationshipTypes = Array.from(relationshipTypeCounts.entries()).map(([type, stats]) => ({
|
|
||||||
type,
|
|
||||||
count: stats.count,
|
|
||||||
confidence: stats.totalConfidence / stats.count
|
|
||||||
}));
|
|
||||||
// Assess data quality
|
|
||||||
const dataQuality = this.assessDataQuality(parsedData, analysis);
|
|
||||||
// Generate recommendations
|
|
||||||
const recommendations = this.generateRecommendations(parsedData, analysis, entityTypes, relationshipTypes);
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: {
|
|
||||||
entityTypes,
|
|
||||||
relationshipTypes,
|
|
||||||
dataQuality,
|
|
||||||
recommendations
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: {
|
|
||||||
entityTypes: [],
|
|
||||||
relationshipTypes: [],
|
|
||||||
dataQuality: { completeness: 0, consistency: 0, accuracy: 0 },
|
|
||||||
recommendations: []
|
|
||||||
},
|
|
||||||
error: error instanceof Error ? error.message : 'Structure analysis failed'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Validate data compatibility with current knowledge base
|
|
||||||
*/
|
|
||||||
async validateCompatibility(rawData, dataType) {
|
|
||||||
try {
|
|
||||||
// Parse the raw data
|
|
||||||
const parsedData = await this.parseRawData(rawData, dataType);
|
|
||||||
// Perform neural analysis
|
|
||||||
const analysis = await this.performNeuralAnalysis(parsedData);
|
|
||||||
const issues = [];
|
|
||||||
const suggestions = [];
|
|
||||||
// Check for low confidence entities
|
|
||||||
const lowConfidenceEntities = analysis.detectedEntities.filter((e) => e.confidence < 0.5);
|
|
||||||
if (lowConfidenceEntities.length > 0) {
|
|
||||||
issues.push({
|
|
||||||
type: 'confidence',
|
|
||||||
description: `${lowConfidenceEntities.length} entities have low confidence scores`,
|
|
||||||
severity: 'medium'
|
|
||||||
});
|
|
||||||
suggestions.push('Consider reviewing field names and data structure for better entity detection');
|
|
||||||
}
|
|
||||||
// Check for missing relationships
|
|
||||||
if (analysis.detectedRelationships.length === 0 && analysis.detectedEntities.length > 1) {
|
|
||||||
issues.push({
|
|
||||||
type: 'relationships',
|
|
||||||
description: 'No relationships detected between entities',
|
|
||||||
severity: 'low'
|
|
||||||
});
|
|
||||||
suggestions.push('Consider adding contextual fields that describe entity relationships');
|
|
||||||
}
|
|
||||||
// Check for data type compatibility
|
|
||||||
const supportedTypes = ['json', 'csv', 'yaml', 'text'];
|
|
||||||
if (!supportedTypes.includes(dataType.toLowerCase())) {
|
|
||||||
issues.push({
|
|
||||||
type: 'format',
|
|
||||||
description: `Data type '${dataType}' may not be fully supported`,
|
|
||||||
severity: 'high'
|
|
||||||
});
|
|
||||||
suggestions.push(`Convert data to one of: ${supportedTypes.join(', ')}`);
|
|
||||||
}
|
|
||||||
// Check for data completeness
|
|
||||||
const incompleteEntities = analysis.detectedEntities.filter((e) => !e.originalData || Object.keys(e.originalData).length < 2);
|
|
||||||
if (incompleteEntities.length > 0) {
|
|
||||||
issues.push({
|
|
||||||
type: 'completeness',
|
|
||||||
description: `${incompleteEntities.length} entities have insufficient data`,
|
|
||||||
severity: 'medium'
|
|
||||||
});
|
|
||||||
suggestions.push('Ensure each entity has multiple descriptive fields');
|
|
||||||
}
|
|
||||||
const compatible = issues.filter(i => i.severity === 'high').length === 0;
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: {
|
|
||||||
compatible,
|
|
||||||
issues,
|
|
||||||
suggestions
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: {
|
|
||||||
compatible: false,
|
|
||||||
issues: [{
|
|
||||||
type: 'error',
|
|
||||||
description: error instanceof Error ? error.message : 'Validation failed',
|
|
||||||
severity: 'high'
|
|
||||||
}],
|
|
||||||
suggestions: []
|
|
||||||
},
|
|
||||||
error: error instanceof Error ? error.message : 'Compatibility validation failed'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get the full neural analysis result (custom method for Cortex integration)
|
|
||||||
*/
|
|
||||||
async getNeuralAnalysis(rawData, dataType) {
|
|
||||||
const parsedData = await this.parseRawData(rawData, dataType);
|
|
||||||
return await this.performNeuralAnalysis(parsedData);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Parse raw data based on type
|
|
||||||
*/
|
|
||||||
async parseRawData(rawData, dataType) {
|
|
||||||
const content = typeof rawData === 'string' ? rawData : rawData.toString('utf8');
|
|
||||||
switch (dataType.toLowerCase()) {
|
|
||||||
case 'json':
|
|
||||||
const jsonData = JSON.parse(content);
|
|
||||||
return Array.isArray(jsonData) ? jsonData : [jsonData];
|
|
||||||
case 'csv':
|
|
||||||
return this.parseCSV(content);
|
|
||||||
case 'yaml':
|
|
||||||
case 'yml':
|
|
||||||
// For now, basic YAML support - in full implementation would use yaml parser
|
|
||||||
return JSON.parse(content); // Placeholder
|
|
||||||
case 'txt':
|
|
||||||
case 'text':
|
|
||||||
// Split text into sentences/paragraphs for analysis
|
|
||||||
return content.split(/\n+/).filter(line => line.trim()).map(line => ({ text: line }));
|
|
||||||
default:
|
|
||||||
throw new Error(`Unsupported data type: ${dataType}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Basic CSV parser
|
|
||||||
*/
|
|
||||||
parseCSV(content) {
|
|
||||||
const lines = content.split('\n').filter(line => line.trim());
|
|
||||||
if (lines.length < 2)
|
|
||||||
return [];
|
|
||||||
const headers = lines[0].split(',').map(h => h.trim().replace(/"/g, ''));
|
|
||||||
const data = [];
|
|
||||||
for (let i = 1; i < lines.length; i++) {
|
|
||||||
const values = lines[i].split(',').map(v => v.trim().replace(/"/g, ''));
|
|
||||||
const row = {};
|
|
||||||
headers.forEach((header, index) => {
|
|
||||||
row[header] = values[index] || '';
|
|
||||||
});
|
|
||||||
data.push(row);
|
|
||||||
}
|
|
||||||
return data;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Perform neural analysis on parsed data
|
|
||||||
*/
|
|
||||||
async performNeuralAnalysis(parsedData, config = this.config) {
|
|
||||||
// Phase 1: Neural Entity Detection
|
|
||||||
const detectedEntities = await this.detectEntitiesWithNeuralAnalysis(parsedData, config);
|
|
||||||
// Phase 2: Neural Relationship Detection
|
|
||||||
const detectedRelationships = await this.detectRelationshipsWithNeuralAnalysis(detectedEntities, parsedData, config);
|
|
||||||
// Phase 3: Neural Insights Generation
|
|
||||||
const insights = await this.generateNeuralInsights(detectedEntities, detectedRelationships);
|
|
||||||
// Phase 4: Confidence Scoring
|
|
||||||
const overallConfidence = this.calculateOverallConfidence(detectedEntities, detectedRelationships);
|
|
||||||
return {
|
|
||||||
detectedEntities,
|
|
||||||
detectedRelationships,
|
|
||||||
confidence: overallConfidence,
|
|
||||||
insights
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Neural Entity Detection - The Core AI Engine
|
|
||||||
*/
|
|
||||||
async detectEntitiesWithNeuralAnalysis(rawData, config = this.config) {
|
|
||||||
const entities = [];
|
|
||||||
const nounTypes = Object.values(NounType);
|
|
||||||
for (const [index, dataItem] of rawData.entries()) {
|
|
||||||
const mainText = this.extractMainText(dataItem);
|
|
||||||
const detections = [];
|
|
||||||
// Test against all noun types using semantic similarity
|
|
||||||
for (const nounType of nounTypes) {
|
|
||||||
const confidence = await this.calculateEntityTypeConfidence(mainText, dataItem, nounType);
|
|
||||||
if (confidence >= config.confidenceThreshold - 0.2) { // Allow slightly lower for alternatives
|
|
||||||
const reasoning = await this.generateEntityReasoning(mainText, dataItem, nounType);
|
|
||||||
detections.push({ type: nounType, confidence, reasoning });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (detections.length > 0) {
|
|
||||||
// Sort by confidence
|
|
||||||
detections.sort((a, b) => b.confidence - a.confidence);
|
|
||||||
const primaryType = detections[0];
|
|
||||||
const alternatives = detections.slice(1, 3); // Top 2 alternatives
|
|
||||||
entities.push({
|
|
||||||
originalData: dataItem,
|
|
||||||
nounType: primaryType.type,
|
|
||||||
confidence: primaryType.confidence,
|
|
||||||
suggestedId: this.generateSmartId(dataItem, primaryType.type, index),
|
|
||||||
reasoning: primaryType.reasoning,
|
|
||||||
alternativeTypes: alternatives
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return entities;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Calculate entity type confidence using AI
|
|
||||||
*/
|
|
||||||
async calculateEntityTypeConfidence(text, data, nounType) {
|
|
||||||
// Base semantic similarity using search
|
|
||||||
const searchResults = await this.brainy.search(text + ' ' + nounType, 1);
|
|
||||||
const textSimilarity = searchResults.length > 0 ? searchResults[0].score : 0.5;
|
|
||||||
// Field-based confidence boost
|
|
||||||
const fieldBoost = this.calculateFieldBasedConfidence(data, nounType);
|
|
||||||
// Pattern-based confidence boost
|
|
||||||
const patternBoost = this.calculatePatternBasedConfidence(text, data, nounType);
|
|
||||||
// Combine confidences with weights
|
|
||||||
const combined = (textSimilarity * 0.5) + (fieldBoost * 0.3) + (patternBoost * 0.2);
|
|
||||||
return Math.min(combined, 1.0);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Field-based confidence calculation
|
|
||||||
*/
|
|
||||||
calculateFieldBasedConfidence(data, nounType) {
|
|
||||||
const fields = Object.keys(data);
|
|
||||||
let boost = 0;
|
|
||||||
// Field patterns that boost confidence for specific noun types
|
|
||||||
const fieldPatterns = {
|
|
||||||
[NounType.Person]: ['name', 'email', 'phone', 'age', 'firstname', 'lastname', 'employee'],
|
|
||||||
[NounType.Organization]: ['company', 'organization', 'corp', 'inc', 'ltd', 'department', 'team'],
|
|
||||||
[NounType.Project]: ['project', 'task', 'deadline', 'status', 'milestone', 'deliverable'],
|
|
||||||
[NounType.Location]: ['address', 'city', 'country', 'state', 'zip', 'location', 'coordinates'],
|
|
||||||
[NounType.Product]: ['product', 'price', 'sku', 'inventory', 'category', 'brand'],
|
|
||||||
[NounType.Event]: ['date', 'time', 'venue', 'event', 'meeting', 'conference', 'schedule']
|
|
||||||
};
|
|
||||||
const relevantPatterns = fieldPatterns[nounType] || [];
|
|
||||||
for (const field of fields) {
|
|
||||||
for (const pattern of relevantPatterns) {
|
|
||||||
if (field.toLowerCase().includes(pattern)) {
|
|
||||||
boost += 0.1;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return Math.min(boost, 0.5);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Pattern-based confidence calculation
|
|
||||||
*/
|
|
||||||
calculatePatternBasedConfidence(text, data, nounType) {
|
|
||||||
let boost = 0;
|
|
||||||
// Content patterns that indicate entity types
|
|
||||||
const patterns = {
|
|
||||||
[NounType.Person]: [
|
|
||||||
/@.*\.com/i, // Email pattern
|
|
||||||
/\b[A-Z][a-z]+ [A-Z][a-z]+\b/, // Name pattern
|
|
||||||
/Mr\.|Mrs\.|Dr\.|Prof\./i // Title pattern
|
|
||||||
],
|
|
||||||
[NounType.Organization]: [
|
|
||||||
/\bInc\.|Corp\.|LLC\.|Ltd\./i, // Corporate suffixes
|
|
||||||
/Company|Corporation|Enterprise/i
|
|
||||||
],
|
|
||||||
[NounType.Location]: [
|
|
||||||
/\b\d{5}(-\d{4})?\b/, // ZIP code
|
|
||||||
/Street|Ave|Road|Blvd/i
|
|
||||||
]
|
|
||||||
};
|
|
||||||
const relevantPatterns = patterns[nounType] || [];
|
|
||||||
for (const pattern of relevantPatterns) {
|
|
||||||
if (pattern.test(text)) {
|
|
||||||
boost += 0.15;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return Math.min(boost, 0.3);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Generate reasoning for entity type selection
|
|
||||||
*/
|
|
||||||
async generateEntityReasoning(text, data, nounType) {
|
|
||||||
const reasons = [];
|
|
||||||
// Semantic similarity reason
|
|
||||||
const searchResults = await this.brainy.search(text + ' ' + nounType, 1);
|
|
||||||
const similarity = searchResults.length > 0 ? searchResults[0].score : 0.5;
|
|
||||||
if (similarity > 0.7) {
|
|
||||||
reasons.push(`High semantic similarity (${(similarity * 100).toFixed(1)}%)`);
|
|
||||||
}
|
|
||||||
// Field-based reasons
|
|
||||||
const relevantFields = this.getRelevantFields(data, nounType);
|
|
||||||
if (relevantFields.length > 0) {
|
|
||||||
reasons.push(`Contains ${nounType}-specific fields: ${relevantFields.join(', ')}`);
|
|
||||||
}
|
|
||||||
// Pattern-based reasons
|
|
||||||
const matchedPatterns = this.getMatchedPatterns(text, data, nounType);
|
|
||||||
if (matchedPatterns.length > 0) {
|
|
||||||
reasons.push(`Matches ${nounType} patterns: ${matchedPatterns.join(', ')}`);
|
|
||||||
}
|
|
||||||
return reasons.length > 0 ? reasons.join('; ') : 'General semantic match';
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Neural Relationship Detection
|
|
||||||
*/
|
|
||||||
async detectRelationshipsWithNeuralAnalysis(entities, rawData, config = this.config) {
|
|
||||||
const relationships = [];
|
|
||||||
const verbTypes = Object.values(VerbType);
|
|
||||||
// For each pair of entities, test relationship possibilities
|
|
||||||
for (let i = 0; i < entities.length; i++) {
|
|
||||||
for (let j = i + 1; j < entities.length; j++) {
|
|
||||||
const sourceEntity = entities[i];
|
|
||||||
const targetEntity = entities[j];
|
|
||||||
// Extract context for relationship detection
|
|
||||||
const context = this.extractRelationshipContext(sourceEntity.originalData, targetEntity.originalData, rawData);
|
|
||||||
// Test all verb types
|
|
||||||
for (const verbType of verbTypes) {
|
|
||||||
const confidence = await this.calculateRelationshipConfidence(sourceEntity, targetEntity, verbType, context);
|
|
||||||
if (confidence >= config.confidenceThreshold - 0.1) { // Slightly lower threshold for relationships
|
|
||||||
const weight = config.enableWeights ?
|
|
||||||
this.calculateRelationshipWeight(sourceEntity, targetEntity, verbType, context) :
|
|
||||||
0.5;
|
|
||||||
const reasoning = await this.generateRelationshipReasoning(sourceEntity, targetEntity, verbType, context);
|
|
||||||
relationships.push({
|
|
||||||
sourceId: sourceEntity.suggestedId,
|
|
||||||
targetId: targetEntity.suggestedId,
|
|
||||||
verbType,
|
|
||||||
confidence,
|
|
||||||
weight,
|
|
||||||
reasoning,
|
|
||||||
context,
|
|
||||||
metadata: this.extractRelationshipMetadata(sourceEntity.originalData, targetEntity.originalData, verbType)
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Sort by confidence and remove duplicates/conflicts
|
|
||||||
return this.pruneRelationships(relationships);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Calculate relationship confidence
|
|
||||||
*/
|
|
||||||
async calculateRelationshipConfidence(source, target, verbType, context) {
|
|
||||||
// Semantic similarity between entities and verb type
|
|
||||||
const relationshipText = `${this.extractMainText(source.originalData)} ${verbType} ${this.extractMainText(target.originalData)}`;
|
|
||||||
const directResults = await this.brainy.search(relationshipText, 1);
|
|
||||||
const directSimilarity = directResults.length > 0 ? directResults[0].score : 0.5;
|
|
||||||
// Context-based similarity
|
|
||||||
const contextResults = await this.brainy.search(context + ' ' + verbType, 1);
|
|
||||||
const contextSimilarity = contextResults.length > 0 ? contextResults[0].score : 0.5;
|
|
||||||
// Entity type compatibility
|
|
||||||
const typeCompatibility = this.calculateTypeCompatibility(source.nounType, target.nounType, verbType);
|
|
||||||
// Combine with weights
|
|
||||||
return (directSimilarity * 0.4) + (contextSimilarity * 0.4) + (typeCompatibility * 0.2);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Calculate relationship weight/strength
|
|
||||||
*/
|
|
||||||
calculateRelationshipWeight(source, target, verbType, context) {
|
|
||||||
let weight = 0.5; // Base weight
|
|
||||||
// Context richness (more descriptive = stronger)
|
|
||||||
const contextWords = context.split(' ').length;
|
|
||||||
weight += Math.min(contextWords / 20, 0.2);
|
|
||||||
// Entity importance (higher confidence entities = stronger relationships)
|
|
||||||
const avgEntityConfidence = (source.confidence + target.confidence) / 2;
|
|
||||||
weight += avgEntityConfidence * 0.2;
|
|
||||||
// Verb type specificity (more specific verbs = stronger)
|
|
||||||
const verbSpecificity = this.getVerbSpecificity(verbType);
|
|
||||||
weight += verbSpecificity * 0.1;
|
|
||||||
return Math.min(weight, 1.0);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Generate Neural Insights - The Intelligence Layer
|
|
||||||
*/
|
|
||||||
async generateNeuralInsights(entities, relationships) {
|
|
||||||
const insights = [];
|
|
||||||
// Detect hierarchies
|
|
||||||
const hierarchies = this.detectHierarchies(relationships);
|
|
||||||
hierarchies.forEach(hierarchy => {
|
|
||||||
insights.push({
|
|
||||||
type: 'hierarchy',
|
|
||||||
description: `Detected ${hierarchy.type} hierarchy with ${hierarchy.levels} levels`,
|
|
||||||
confidence: hierarchy.confidence,
|
|
||||||
affectedEntities: hierarchy.entities,
|
|
||||||
recommendation: `Consider visualizing the ${hierarchy.type} structure`
|
|
||||||
});
|
|
||||||
});
|
|
||||||
// Detect clusters
|
|
||||||
const clusters = this.detectClusters(entities, relationships);
|
|
||||||
clusters.forEach(cluster => {
|
|
||||||
insights.push({
|
|
||||||
type: 'cluster',
|
|
||||||
description: `Found cluster of ${cluster.size} ${cluster.primaryType} entities`,
|
|
||||||
confidence: cluster.confidence,
|
|
||||||
affectedEntities: cluster.entities,
|
|
||||||
recommendation: `These ${cluster.primaryType}s might form a natural grouping`
|
|
||||||
});
|
|
||||||
});
|
|
||||||
// Detect patterns
|
|
||||||
const patterns = this.detectPatterns(relationships);
|
|
||||||
patterns.forEach(pattern => {
|
|
||||||
insights.push({
|
|
||||||
type: 'pattern',
|
|
||||||
description: `Common relationship pattern: ${pattern.description}`,
|
|
||||||
confidence: pattern.confidence,
|
|
||||||
affectedEntities: pattern.entities,
|
|
||||||
recommendation: pattern.recommendation
|
|
||||||
});
|
|
||||||
});
|
|
||||||
return insights;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Helper methods for the neural system
|
|
||||||
*/
|
|
||||||
extractMainText(data) {
|
|
||||||
// Extract the most relevant text from a data object
|
|
||||||
const textFields = ['name', 'title', 'description', 'content', 'text', 'label'];
|
|
||||||
for (const field of textFields) {
|
|
||||||
if (data[field] && typeof data[field] === 'string') {
|
|
||||||
return data[field];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Fallback: concatenate all string values
|
|
||||||
return Object.values(data)
|
|
||||||
.filter(v => typeof v === 'string')
|
|
||||||
.join(' ')
|
|
||||||
.substring(0, 200); // Limit length
|
|
||||||
}
|
|
||||||
generateSmartId(data, nounType, index) {
|
|
||||||
const mainText = this.extractMainText(data);
|
|
||||||
const cleanText = mainText.toLowerCase().replace(/[^a-z0-9]/g, '_').substring(0, 20);
|
|
||||||
return `${nounType}_${cleanText}_${index}`;
|
|
||||||
}
|
|
||||||
extractRelationshipContext(source, target, allData) {
|
|
||||||
// Extract context for relationship detection
|
|
||||||
return [
|
|
||||||
this.extractMainText(source),
|
|
||||||
this.extractMainText(target),
|
|
||||||
// Add more contextual information
|
|
||||||
].join(' ');
|
|
||||||
}
|
|
||||||
calculateTypeCompatibility(sourceType, targetType, verbType) {
|
|
||||||
// Define type compatibility matrix for relationships
|
|
||||||
const compatibilityMatrix = {
|
|
||||||
[NounType.Person]: {
|
|
||||||
[NounType.Organization]: [VerbType.MemberOf, VerbType.WorksWith],
|
|
||||||
[NounType.Project]: [VerbType.WorksWith, VerbType.Creates],
|
|
||||||
[NounType.Person]: [VerbType.WorksWith, VerbType.Mentors, VerbType.ReportsTo]
|
|
||||||
}
|
|
||||||
// Add more compatibility rules
|
|
||||||
};
|
|
||||||
const sourceCompatibility = compatibilityMatrix[sourceType];
|
|
||||||
if (sourceCompatibility && sourceCompatibility[targetType]) {
|
|
||||||
return sourceCompatibility[targetType].includes(verbType) ? 1.0 : 0.3;
|
|
||||||
}
|
|
||||||
return 0.5; // Default compatibility
|
|
||||||
}
|
|
||||||
getVerbSpecificity(verbType) {
|
|
||||||
// More specific verbs get higher scores
|
|
||||||
const specificityScores = {
|
|
||||||
[VerbType.RelatedTo]: 0.1, // Very generic
|
|
||||||
[VerbType.WorksWith]: 0.7, // Specific
|
|
||||||
[VerbType.Mentors]: 0.9, // Very specific
|
|
||||||
[VerbType.ReportsTo]: 0.9, // Very specific
|
|
||||||
[VerbType.Supervises]: 0.9 // Very specific
|
|
||||||
};
|
|
||||||
return specificityScores[verbType] || 0.5;
|
|
||||||
}
|
|
||||||
getRelevantFields(data, nounType) {
|
|
||||||
// Implementation for finding relevant fields
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
getMatchedPatterns(text, data, nounType) {
|
|
||||||
// Implementation for finding matched patterns
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
pruneRelationships(relationships) {
|
|
||||||
// Remove duplicates and low-confidence relationships
|
|
||||||
return relationships
|
|
||||||
.sort((a, b) => b.confidence - a.confidence)
|
|
||||||
.slice(0, 1000); // Limit to top 1000 relationships
|
|
||||||
}
|
|
||||||
detectHierarchies(relationships) {
|
|
||||||
// Detect hierarchical structures
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
detectClusters(entities, relationships) {
|
|
||||||
// Detect entity clusters
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
detectPatterns(relationships) {
|
|
||||||
// Detect relationship patterns
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
calculateOverallConfidence(entities, relationships) {
|
|
||||||
if (entities.length === 0)
|
|
||||||
return 0;
|
|
||||||
const entityConfidence = entities.reduce((sum, e) => sum + e.confidence, 0) / entities.length;
|
|
||||||
if (relationships.length === 0)
|
|
||||||
return entityConfidence;
|
|
||||||
const relationshipConfidence = relationships.reduce((sum, r) => sum + r.confidence, 0) / relationships.length;
|
|
||||||
return (entityConfidence + relationshipConfidence) / 2;
|
|
||||||
}
|
|
||||||
async storeNeuralAnalysis(analysis) {
|
|
||||||
// Store the full analysis result for later retrieval by Neural Import or other systems
|
|
||||||
// This could be stored in the brainy instance metadata or a separate analysis store
|
|
||||||
}
|
|
||||||
getDataTypeFromPath(filePath) {
|
|
||||||
const ext = path.extname(filePath).toLowerCase();
|
|
||||||
switch (ext) {
|
|
||||||
case '.json': return 'json';
|
|
||||||
case '.csv': return 'csv';
|
|
||||||
case '.yaml':
|
|
||||||
case '.yml': return 'yaml';
|
|
||||||
case '.txt': return 'text';
|
|
||||||
default: return 'text';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async generateRelationshipReasoning(source, target, verbType, context) {
|
|
||||||
return `Neural analysis detected ${verbType} relationship based on semantic context`;
|
|
||||||
}
|
|
||||||
extractRelationshipMetadata(sourceData, targetData, verbType) {
|
|
||||||
return {
|
|
||||||
sourceType: typeof sourceData,
|
|
||||||
targetType: typeof targetData,
|
|
||||||
detectedBy: 'neural-import-sense',
|
|
||||||
timestamp: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Assess data quality metrics
|
|
||||||
*/
|
|
||||||
assessDataQuality(parsedData, analysis) {
|
|
||||||
// Completeness: ratio of fields with data
|
|
||||||
let totalFields = 0;
|
|
||||||
let filledFields = 0;
|
|
||||||
parsedData.forEach(item => {
|
|
||||||
const fields = Object.keys(item);
|
|
||||||
totalFields += fields.length;
|
|
||||||
filledFields += fields.filter(field => item[field] !== null &&
|
|
||||||
item[field] !== undefined &&
|
|
||||||
item[field] !== '').length;
|
|
||||||
});
|
|
||||||
const completeness = totalFields > 0 ? filledFields / totalFields : 0;
|
|
||||||
// Consistency: variance in field structure
|
|
||||||
const fieldSets = parsedData.map(item => new Set(Object.keys(item)));
|
|
||||||
const allFields = new Set(fieldSets.flatMap(set => Array.from(set)));
|
|
||||||
let consistencyScore = 0;
|
|
||||||
if (fieldSets.length > 0) {
|
|
||||||
consistencyScore = Array.from(allFields).reduce((score, field) => {
|
|
||||||
const hasField = fieldSets.filter(set => set.has(field)).length;
|
|
||||||
return score + (hasField / fieldSets.length);
|
|
||||||
}, 0) / allFields.size;
|
|
||||||
}
|
|
||||||
// Accuracy: average confidence of detected entities
|
|
||||||
const accuracy = analysis.detectedEntities.length > 0 ?
|
|
||||||
analysis.detectedEntities.reduce((sum, e) => sum + e.confidence, 0) / analysis.detectedEntities.length :
|
|
||||||
0;
|
|
||||||
return {
|
|
||||||
completeness,
|
|
||||||
consistency: consistencyScore,
|
|
||||||
accuracy
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Generate recommendations based on analysis
|
|
||||||
*/
|
|
||||||
generateRecommendations(parsedData, analysis, entityTypes, relationshipTypes) {
|
|
||||||
const recommendations = [];
|
|
||||||
// Low entity confidence recommendations
|
|
||||||
const lowConfidenceEntities = entityTypes.filter(et => et.confidence < 0.7);
|
|
||||||
if (lowConfidenceEntities.length > 0) {
|
|
||||||
recommendations.push(`Consider improving field names for ${lowConfidenceEntities.map(e => e.type).join(', ')} entities`);
|
|
||||||
}
|
|
||||||
// Missing relationships recommendations
|
|
||||||
if (relationshipTypes.length === 0 && entityTypes.length > 1) {
|
|
||||||
recommendations.push('Add fields that describe how entities relate to each other');
|
|
||||||
}
|
|
||||||
// Data structure recommendations
|
|
||||||
if (parsedData.length > 0) {
|
|
||||||
const firstItem = parsedData[0];
|
|
||||||
const fieldCount = Object.keys(firstItem).length;
|
|
||||||
if (fieldCount < 3) {
|
|
||||||
recommendations.push('Consider adding more descriptive fields to each entity');
|
|
||||||
}
|
|
||||||
if (fieldCount > 20) {
|
|
||||||
recommendations.push('Consider grouping related fields or splitting complex entities');
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Entity distribution recommendations
|
|
||||||
const dominantEntityType = entityTypes.reduce((max, current) => current.count > max.count ? current : max, entityTypes[0] || { count: 0 });
|
|
||||||
if (dominantEntityType && dominantEntityType.count > parsedData.length * 0.8) {
|
|
||||||
recommendations.push(`Consider diversifying entity types - ${dominantEntityType.type} dominates the dataset`);
|
|
||||||
}
|
|
||||||
// Relationship quality recommendations
|
|
||||||
const lowWeightRelationships = relationshipTypes.filter(rt => rt.confidence < 0.6);
|
|
||||||
if (lowWeightRelationships.length > 0) {
|
|
||||||
recommendations.push('Consider adding more contextual information to strengthen relationship detection');
|
|
||||||
}
|
|
||||||
return recommendations;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=neuralImport.js.map
|
|
||||||
1
dist/augmentations/neuralImport.js.map
vendored
1
dist/augmentations/neuralImport.js.map
vendored
File diff suppressed because one or more lines are too long
167
dist/augmentations/serverSearchAugmentations.d.ts
vendored
167
dist/augmentations/serverSearchAugmentations.d.ts
vendored
|
|
@ -1,167 +0,0 @@
|
||||||
/**
|
|
||||||
* Server Search Augmentations
|
|
||||||
*
|
|
||||||
* This file implements conduit and activation augmentations for browser-server search functionality.
|
|
||||||
* It allows Brainy to search a server-hosted instance and store results locally.
|
|
||||||
*/
|
|
||||||
import { AugmentationType, IActivationAugmentation, AugmentationResponse, WebSocketConnection } from '../types/augmentations.js';
|
|
||||||
import { WebSocketConduitAugmentation } from './conduitAugmentations.js';
|
|
||||||
import { BrainyDataInterface } from '../types/brainyDataInterface.js';
|
|
||||||
/**
|
|
||||||
* ServerSearchConduitAugmentation
|
|
||||||
*
|
|
||||||
* A specialized conduit augmentation that provides functionality for searching
|
|
||||||
* a server-hosted Brainy instance and storing results locally.
|
|
||||||
*/
|
|
||||||
export declare class ServerSearchConduitAugmentation extends WebSocketConduitAugmentation {
|
|
||||||
private localDb;
|
|
||||||
constructor(name?: string);
|
|
||||||
/**
|
|
||||||
* Initialize the augmentation
|
|
||||||
*/
|
|
||||||
initialize(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Set the local Brainy instance
|
|
||||||
* @param db The Brainy instance to use for local storage
|
|
||||||
*/
|
|
||||||
setLocalDb(db: BrainyDataInterface): void;
|
|
||||||
/**
|
|
||||||
* Get the local Brainy instance
|
|
||||||
* @returns The local Brainy instance
|
|
||||||
*/
|
|
||||||
getLocalDb(): BrainyDataInterface | null;
|
|
||||||
/**
|
|
||||||
* Search the server-hosted Brainy instance and store results locally
|
|
||||||
* @param connectionId The ID of the established connection
|
|
||||||
* @param query The search query
|
|
||||||
* @param limit Maximum number of results to return
|
|
||||||
* @returns Search results
|
|
||||||
*/
|
|
||||||
searchServer(connectionId: string, query: string, limit?: number): Promise<AugmentationResponse<unknown>>;
|
|
||||||
/**
|
|
||||||
* Search the local Brainy instance
|
|
||||||
* @param query The search query
|
|
||||||
* @param limit Maximum number of results to return
|
|
||||||
* @returns Search results
|
|
||||||
*/
|
|
||||||
searchLocal(query: string, limit?: number): Promise<AugmentationResponse<unknown>>;
|
|
||||||
/**
|
|
||||||
* Search both server and local instances, combine results, and store server results locally
|
|
||||||
* @param connectionId The ID of the established connection
|
|
||||||
* @param query The search query
|
|
||||||
* @param limit Maximum number of results to return
|
|
||||||
* @returns Combined search results
|
|
||||||
*/
|
|
||||||
searchCombined(connectionId: string, query: string, limit?: number): Promise<AugmentationResponse<unknown>>;
|
|
||||||
/**
|
|
||||||
* Add data to both local and server instances
|
|
||||||
* @param connectionId The ID of the established connection
|
|
||||||
* @param data Text or vector to add
|
|
||||||
* @param metadata Metadata for the data
|
|
||||||
* @returns ID of the added data
|
|
||||||
*/
|
|
||||||
addToBoth(connectionId: string, data: string | any[], metadata?: any): Promise<AugmentationResponse<string>>;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* ServerSearchActivationAugmentation
|
|
||||||
*
|
|
||||||
* An activation augmentation that provides actions for server search functionality.
|
|
||||||
*/
|
|
||||||
export declare class ServerSearchActivationAugmentation implements IActivationAugmentation {
|
|
||||||
readonly name: string;
|
|
||||||
readonly description: string;
|
|
||||||
enabled: boolean;
|
|
||||||
private isInitialized;
|
|
||||||
private conduitAugmentation;
|
|
||||||
private connections;
|
|
||||||
constructor(name?: string);
|
|
||||||
getType(): AugmentationType;
|
|
||||||
/**
|
|
||||||
* Initialize the augmentation
|
|
||||||
*/
|
|
||||||
initialize(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Shut down the augmentation
|
|
||||||
*/
|
|
||||||
shutDown(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Get the status of the augmentation
|
|
||||||
*/
|
|
||||||
getStatus(): Promise<'active' | 'inactive' | 'error'>;
|
|
||||||
/**
|
|
||||||
* Set the conduit augmentation to use for server search
|
|
||||||
* @param conduit The ServerSearchConduitAugmentation to use
|
|
||||||
*/
|
|
||||||
setConduitAugmentation(conduit: ServerSearchConduitAugmentation): void;
|
|
||||||
/**
|
|
||||||
* Store a connection for later use
|
|
||||||
* @param connectionId The ID to use for the connection
|
|
||||||
* @param connection The WebSocket connection
|
|
||||||
*/
|
|
||||||
storeConnection(connectionId: string, connection: WebSocketConnection): void;
|
|
||||||
/**
|
|
||||||
* Get a stored connection
|
|
||||||
* @param connectionId The ID of the connection to retrieve
|
|
||||||
* @returns The WebSocket connection
|
|
||||||
*/
|
|
||||||
getConnection(connectionId: string): WebSocketConnection | undefined;
|
|
||||||
/**
|
|
||||||
* Trigger an action based on a processed command or internal state
|
|
||||||
* @param actionName The name of the action to trigger
|
|
||||||
* @param parameters Optional parameters for the action
|
|
||||||
*/
|
|
||||||
triggerAction(actionName: string, parameters?: Record<string, unknown>): AugmentationResponse<unknown>;
|
|
||||||
/**
|
|
||||||
* Handle the connectToServer action
|
|
||||||
* @param parameters Action parameters
|
|
||||||
*/
|
|
||||||
private handleConnectToServer;
|
|
||||||
/**
|
|
||||||
* Handle the searchServer action
|
|
||||||
* @param parameters Action parameters
|
|
||||||
*/
|
|
||||||
private handleSearchServer;
|
|
||||||
/**
|
|
||||||
* Handle the searchLocal action
|
|
||||||
* @param parameters Action parameters
|
|
||||||
*/
|
|
||||||
private handleSearchLocal;
|
|
||||||
/**
|
|
||||||
* Handle the searchCombined action
|
|
||||||
* @param parameters Action parameters
|
|
||||||
*/
|
|
||||||
private handleSearchCombined;
|
|
||||||
/**
|
|
||||||
* Handle the addToBoth action
|
|
||||||
* @param parameters Action parameters
|
|
||||||
*/
|
|
||||||
private handleAddToBoth;
|
|
||||||
/**
|
|
||||||
* Generates an expressive output or response from Brainy
|
|
||||||
* @param knowledgeId The identifier of the knowledge to express
|
|
||||||
* @param format The desired output format (e.g., 'text', 'json')
|
|
||||||
*/
|
|
||||||
generateOutput(knowledgeId: string, format: string): AugmentationResponse<string | Record<string, unknown>>;
|
|
||||||
/**
|
|
||||||
* Interacts with an external system or API
|
|
||||||
* @param systemId The identifier of the external system
|
|
||||||
* @param payload The data to send to the external system
|
|
||||||
*/
|
|
||||||
interactExternal(systemId: string, payload: Record<string, unknown>): AugmentationResponse<unknown>;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Factory function to create server search augmentations
|
|
||||||
* @param serverUrl The URL of the server to connect to
|
|
||||||
* @param options Additional options
|
|
||||||
* @returns An object containing the created augmentations
|
|
||||||
*/
|
|
||||||
export declare function createServerSearchAugmentations(serverUrl: string, options?: {
|
|
||||||
conduitName?: string;
|
|
||||||
activationName?: string;
|
|
||||||
protocols?: string | string[];
|
|
||||||
localDb?: BrainyDataInterface;
|
|
||||||
}): Promise<{
|
|
||||||
conduit: ServerSearchConduitAugmentation;
|
|
||||||
activation: ServerSearchActivationAugmentation;
|
|
||||||
connection: WebSocketConnection;
|
|
||||||
}>;
|
|
||||||
531
dist/augmentations/serverSearchAugmentations.js
vendored
531
dist/augmentations/serverSearchAugmentations.js
vendored
|
|
@ -1,531 +0,0 @@
|
||||||
/**
|
|
||||||
* Server Search Augmentations
|
|
||||||
*
|
|
||||||
* This file implements conduit and activation augmentations for browser-server search functionality.
|
|
||||||
* It allows Brainy to search a server-hosted instance and store results locally.
|
|
||||||
*/
|
|
||||||
import { AugmentationType } from '../types/augmentations.js';
|
|
||||||
import { WebSocketConduitAugmentation } from './conduitAugmentations.js';
|
|
||||||
/**
|
|
||||||
* ServerSearchConduitAugmentation
|
|
||||||
*
|
|
||||||
* A specialized conduit augmentation that provides functionality for searching
|
|
||||||
* a server-hosted Brainy instance and storing results locally.
|
|
||||||
*/
|
|
||||||
export class ServerSearchConduitAugmentation extends WebSocketConduitAugmentation {
|
|
||||||
constructor(name = 'server-search-conduit') {
|
|
||||||
super(name);
|
|
||||||
this.localDb = null;
|
|
||||||
// this.description = 'Conduit augmentation for server-hosted Brainy search'
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Initialize the augmentation
|
|
||||||
*/
|
|
||||||
async initialize() {
|
|
||||||
if (this.isInitialized) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
// Initialize the base conduit
|
|
||||||
await super.initialize();
|
|
||||||
// Local DB must be set before initialization
|
|
||||||
if (!this.localDb) {
|
|
||||||
throw new Error('Local database not set. Call setLocalDb before initializing.');
|
|
||||||
}
|
|
||||||
this.isInitialized = true;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error(`Failed to initialize ${this.name}:`, error);
|
|
||||||
throw new Error(`Failed to initialize ${this.name}: ${error}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Set the local Brainy instance
|
|
||||||
* @param db The Brainy instance to use for local storage
|
|
||||||
*/
|
|
||||||
setLocalDb(db) {
|
|
||||||
this.localDb = db;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get the local Brainy instance
|
|
||||||
* @returns The local Brainy instance
|
|
||||||
*/
|
|
||||||
getLocalDb() {
|
|
||||||
return this.localDb;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Search the server-hosted Brainy instance and store results locally
|
|
||||||
* @param connectionId The ID of the established connection
|
|
||||||
* @param query The search query
|
|
||||||
* @param limit Maximum number of results to return
|
|
||||||
* @returns Search results
|
|
||||||
*/
|
|
||||||
async searchServer(connectionId, query, limit = 10) {
|
|
||||||
await this.ensureInitialized();
|
|
||||||
try {
|
|
||||||
// Create a search request
|
|
||||||
const readResult = await this.readData({
|
|
||||||
connectionId,
|
|
||||||
query: {
|
|
||||||
type: 'search',
|
|
||||||
query,
|
|
||||||
limit
|
|
||||||
}
|
|
||||||
});
|
|
||||||
if (readResult.success && readResult.data) {
|
|
||||||
const searchResults = readResult.data;
|
|
||||||
// Store the results in the local Brainy instance
|
|
||||||
if (this.localDb) {
|
|
||||||
for (const result of searchResults) {
|
|
||||||
// Check if the noun already exists in the local database
|
|
||||||
const existingNoun = await this.localDb.get(result.id);
|
|
||||||
if (!existingNoun) {
|
|
||||||
// Add the noun to the local database
|
|
||||||
await this.localDb.add(result.vector, result.metadata);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: searchResults
|
|
||||||
};
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: readResult.error || 'Unknown error searching server'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error searching server:', error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: `Error searching server: ${error}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Search the local Brainy instance
|
|
||||||
* @param query The search query
|
|
||||||
* @param limit Maximum number of results to return
|
|
||||||
* @returns Search results
|
|
||||||
*/
|
|
||||||
async searchLocal(query, limit = 10) {
|
|
||||||
await this.ensureInitialized();
|
|
||||||
try {
|
|
||||||
if (!this.localDb) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'Local database not initialized'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
const results = await this.localDb.searchText(query, limit);
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: results
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error searching local database:', error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: `Error searching local database: ${error}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Search both server and local instances, combine results, and store server results locally
|
|
||||||
* @param connectionId The ID of the established connection
|
|
||||||
* @param query The search query
|
|
||||||
* @param limit Maximum number of results to return
|
|
||||||
* @returns Combined search results
|
|
||||||
*/
|
|
||||||
async searchCombined(connectionId, query, limit = 10) {
|
|
||||||
await this.ensureInitialized();
|
|
||||||
try {
|
|
||||||
// Search local first
|
|
||||||
const localSearchResult = await this.searchLocal(query, limit);
|
|
||||||
if (!localSearchResult.success) {
|
|
||||||
return localSearchResult;
|
|
||||||
}
|
|
||||||
const localResults = localSearchResult.data;
|
|
||||||
// If we have enough local results, return them
|
|
||||||
if (localResults.length >= limit) {
|
|
||||||
return localSearchResult;
|
|
||||||
}
|
|
||||||
// Otherwise, search server for additional results
|
|
||||||
const serverSearchResult = await this.searchServer(connectionId, query, limit - localResults.length);
|
|
||||||
if (!serverSearchResult.success) {
|
|
||||||
// If server search fails, return local results
|
|
||||||
return localSearchResult;
|
|
||||||
}
|
|
||||||
const serverResults = serverSearchResult.data;
|
|
||||||
// Combine results, removing duplicates
|
|
||||||
const combinedResults = [...localResults];
|
|
||||||
const localIds = new Set(localResults.map((r) => r.id));
|
|
||||||
for (const result of serverResults) {
|
|
||||||
if (!localIds.has(result.id)) {
|
|
||||||
combinedResults.push(result);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: combinedResults
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error performing combined search:', error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: `Error performing combined search: ${error}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Add data to both local and server instances
|
|
||||||
* @param connectionId The ID of the established connection
|
|
||||||
* @param data Text or vector to add
|
|
||||||
* @param metadata Metadata for the data
|
|
||||||
* @returns ID of the added data
|
|
||||||
*/
|
|
||||||
async addToBoth(connectionId, data, metadata = {}) {
|
|
||||||
await this.ensureInitialized();
|
|
||||||
try {
|
|
||||||
if (!this.localDb) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: '',
|
|
||||||
error: 'Local database not initialized'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// Add to local first
|
|
||||||
const id = await this.localDb.add(data, metadata);
|
|
||||||
// Get the vector and metadata
|
|
||||||
const noun = (await this.localDb.get(id));
|
|
||||||
if (!noun) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: '',
|
|
||||||
error: 'Failed to retrieve newly created noun'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// Add to server
|
|
||||||
const writeResult = await this.writeData({
|
|
||||||
connectionId,
|
|
||||||
data: {
|
|
||||||
type: 'addNoun',
|
|
||||||
vector: noun.vector,
|
|
||||||
metadata: noun.metadata
|
|
||||||
}
|
|
||||||
});
|
|
||||||
if (!writeResult.success) {
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: id,
|
|
||||||
error: `Added locally but failed to add to server: ${writeResult.error}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: id
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error adding data to both:', error);
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: '',
|
|
||||||
error: `Error adding data to both: ${error}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* ServerSearchActivationAugmentation
|
|
||||||
*
|
|
||||||
* An activation augmentation that provides actions for server search functionality.
|
|
||||||
*/
|
|
||||||
export class ServerSearchActivationAugmentation {
|
|
||||||
constructor(name = 'server-search-activation') {
|
|
||||||
this.enabled = true;
|
|
||||||
this.isInitialized = false;
|
|
||||||
this.conduitAugmentation = null;
|
|
||||||
this.connections = new Map();
|
|
||||||
this.name = name;
|
|
||||||
this.description = 'Activation augmentation for server-hosted Brainy search';
|
|
||||||
}
|
|
||||||
getType() {
|
|
||||||
return AugmentationType.ACTIVATION;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Initialize the augmentation
|
|
||||||
*/
|
|
||||||
async initialize() {
|
|
||||||
if (this.isInitialized) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
this.isInitialized = true;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Shut down the augmentation
|
|
||||||
*/
|
|
||||||
async shutDown() {
|
|
||||||
this.isInitialized = false;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get the status of the augmentation
|
|
||||||
*/
|
|
||||||
async getStatus() {
|
|
||||||
return this.isInitialized ? 'active' : 'inactive';
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Set the conduit augmentation to use for server search
|
|
||||||
* @param conduit The ServerSearchConduitAugmentation to use
|
|
||||||
*/
|
|
||||||
setConduitAugmentation(conduit) {
|
|
||||||
this.conduitAugmentation = conduit;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Store a connection for later use
|
|
||||||
* @param connectionId The ID to use for the connection
|
|
||||||
* @param connection The WebSocket connection
|
|
||||||
*/
|
|
||||||
storeConnection(connectionId, connection) {
|
|
||||||
this.connections.set(connectionId, connection);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get a stored connection
|
|
||||||
* @param connectionId The ID of the connection to retrieve
|
|
||||||
* @returns The WebSocket connection
|
|
||||||
*/
|
|
||||||
getConnection(connectionId) {
|
|
||||||
return this.connections.get(connectionId);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Trigger an action based on a processed command or internal state
|
|
||||||
* @param actionName The name of the action to trigger
|
|
||||||
* @param parameters Optional parameters for the action
|
|
||||||
*/
|
|
||||||
triggerAction(actionName, parameters) {
|
|
||||||
if (!this.conduitAugmentation) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'Conduit augmentation not set'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// Handle different actions
|
|
||||||
switch (actionName) {
|
|
||||||
case 'connectToServer':
|
|
||||||
return this.handleConnectToServer(parameters || {});
|
|
||||||
case 'searchServer':
|
|
||||||
return this.handleSearchServer(parameters || {});
|
|
||||||
case 'searchLocal':
|
|
||||||
return this.handleSearchLocal(parameters || {});
|
|
||||||
case 'searchCombined':
|
|
||||||
return this.handleSearchCombined(parameters || {});
|
|
||||||
case 'addToBoth':
|
|
||||||
return this.handleAddToBoth(parameters || {});
|
|
||||||
default:
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: `Unknown action: ${actionName}`
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Handle the connectToServer action
|
|
||||||
* @param parameters Action parameters
|
|
||||||
*/
|
|
||||||
handleConnectToServer(parameters) {
|
|
||||||
const serverUrl = parameters.serverUrl;
|
|
||||||
const protocols = parameters.protocols;
|
|
||||||
if (!serverUrl) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'serverUrl parameter is required'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// Return a promise that will be resolved when the connection is established
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: this.conduitAugmentation.establishConnection(serverUrl, {
|
|
||||||
protocols
|
|
||||||
})
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Handle the searchServer action
|
|
||||||
* @param parameters Action parameters
|
|
||||||
*/
|
|
||||||
handleSearchServer(parameters) {
|
|
||||||
const connectionId = parameters.connectionId;
|
|
||||||
const query = parameters.query;
|
|
||||||
const limit = parameters.limit || 10;
|
|
||||||
if (!connectionId) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'connectionId parameter is required'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
if (!query) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'query parameter is required'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// Return a promise that will be resolved when the search is complete
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: this.conduitAugmentation.searchServer(connectionId, query, limit)
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Handle the searchLocal action
|
|
||||||
* @param parameters Action parameters
|
|
||||||
*/
|
|
||||||
handleSearchLocal(parameters) {
|
|
||||||
const query = parameters.query;
|
|
||||||
const limit = parameters.limit || 10;
|
|
||||||
if (!query) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'query parameter is required'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// Return a promise that will be resolved when the search is complete
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: this.conduitAugmentation.searchLocal(query, limit)
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Handle the searchCombined action
|
|
||||||
* @param parameters Action parameters
|
|
||||||
*/
|
|
||||||
handleSearchCombined(parameters) {
|
|
||||||
const connectionId = parameters.connectionId;
|
|
||||||
const query = parameters.query;
|
|
||||||
const limit = parameters.limit || 10;
|
|
||||||
if (!connectionId) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'connectionId parameter is required'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
if (!query) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'query parameter is required'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// Return a promise that will be resolved when the search is complete
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: this.conduitAugmentation.searchCombined(connectionId, query, limit)
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Handle the addToBoth action
|
|
||||||
* @param parameters Action parameters
|
|
||||||
*/
|
|
||||||
handleAddToBoth(parameters) {
|
|
||||||
const connectionId = parameters.connectionId;
|
|
||||||
const data = parameters.data;
|
|
||||||
const metadata = parameters.metadata || {};
|
|
||||||
if (!connectionId) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'connectionId parameter is required'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
if (!data) {
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'data parameter is required'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// Return a promise that will be resolved when the add is complete
|
|
||||||
return {
|
|
||||||
success: true,
|
|
||||||
data: this.conduitAugmentation.addToBoth(connectionId, data, metadata)
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Generates an expressive output or response from Brainy
|
|
||||||
* @param knowledgeId The identifier of the knowledge to express
|
|
||||||
* @param format The desired output format (e.g., 'text', 'json')
|
|
||||||
*/
|
|
||||||
generateOutput(knowledgeId, format) {
|
|
||||||
// This method is not used for server search functionality
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: '',
|
|
||||||
error: 'generateOutput is not implemented for ServerSearchActivationAugmentation'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Interacts with an external system or API
|
|
||||||
* @param systemId The identifier of the external system
|
|
||||||
* @param payload The data to send to the external system
|
|
||||||
*/
|
|
||||||
interactExternal(systemId, payload) {
|
|
||||||
// This method is not used for server search functionality
|
|
||||||
return {
|
|
||||||
success: false,
|
|
||||||
data: null,
|
|
||||||
error: 'interactExternal is not implemented for ServerSearchActivationAugmentation'
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Factory function to create server search augmentations
|
|
||||||
* @param serverUrl The URL of the server to connect to
|
|
||||||
* @param options Additional options
|
|
||||||
* @returns An object containing the created augmentations
|
|
||||||
*/
|
|
||||||
export async function createServerSearchAugmentations(serverUrl, options = {}) {
|
|
||||||
// Create the conduit augmentation
|
|
||||||
const conduit = new ServerSearchConduitAugmentation(options.conduitName);
|
|
||||||
await conduit.initialize();
|
|
||||||
// Set the local database if provided
|
|
||||||
if (options.localDb) {
|
|
||||||
conduit.setLocalDb(options.localDb);
|
|
||||||
}
|
|
||||||
// Create the activation augmentation
|
|
||||||
const activation = new ServerSearchActivationAugmentation(options.activationName);
|
|
||||||
await activation.initialize();
|
|
||||||
// Link the augmentations
|
|
||||||
activation.setConduitAugmentation(conduit);
|
|
||||||
// Connect to the server
|
|
||||||
const connectionResult = await conduit.establishConnection(serverUrl, {
|
|
||||||
protocols: options.protocols
|
|
||||||
});
|
|
||||||
if (!connectionResult.success || !connectionResult.data) {
|
|
||||||
throw new Error(`Failed to connect to server: ${connectionResult.error}`);
|
|
||||||
}
|
|
||||||
const connection = connectionResult.data;
|
|
||||||
// Store the connection in the activation augmentation
|
|
||||||
activation.storeConnection(connection.connectionId, connection);
|
|
||||||
return {
|
|
||||||
conduit,
|
|
||||||
activation,
|
|
||||||
connection
|
|
||||||
};
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=serverSearchAugmentations.js.map
|
|
||||||
File diff suppressed because one or more lines are too long
1584
dist/brainyData.d.ts
vendored
1584
dist/brainyData.d.ts
vendored
File diff suppressed because it is too large
Load diff
5630
dist/brainyData.js
vendored
5630
dist/brainyData.js
vendored
File diff suppressed because it is too large
Load diff
1
dist/brainyData.js.map
vendored
1
dist/brainyData.js.map
vendored
File diff suppressed because one or more lines are too long
15
dist/browserFramework.d.ts
vendored
15
dist/browserFramework.d.ts
vendored
|
|
@ -1,15 +0,0 @@
|
||||||
/**
|
|
||||||
* Browser Framework Entry Point for Brainy
|
|
||||||
* Optimized for modern frameworks like Angular, React, Vue, etc.
|
|
||||||
* Auto-detects environment and uses optimal storage (OPFS in browsers)
|
|
||||||
*/
|
|
||||||
import { BrainyData, BrainyDataConfig } from './brainyData.js';
|
|
||||||
import { VerbType, NounType } from './types/graphTypes.js';
|
|
||||||
/**
|
|
||||||
* Create a BrainyData instance optimized for browser frameworks
|
|
||||||
* Auto-detects environment and selects optimal storage and settings
|
|
||||||
*/
|
|
||||||
export declare function createBrowserBrainyData(config?: Partial<BrainyDataConfig>): Promise<BrainyData>;
|
|
||||||
export { VerbType, NounType, BrainyData };
|
|
||||||
export type { BrainyDataConfig };
|
|
||||||
export default createBrowserBrainyData;
|
|
||||||
31
dist/browserFramework.js
vendored
31
dist/browserFramework.js
vendored
|
|
@ -1,31 +0,0 @@
|
||||||
/**
|
|
||||||
* Browser Framework Entry Point for Brainy
|
|
||||||
* Optimized for modern frameworks like Angular, React, Vue, etc.
|
|
||||||
* Auto-detects environment and uses optimal storage (OPFS in browsers)
|
|
||||||
*/
|
|
||||||
import { BrainyData } from './brainyData.js';
|
|
||||||
import { VerbType, NounType } from './types/graphTypes.js';
|
|
||||||
/**
|
|
||||||
* Create a BrainyData instance optimized for browser frameworks
|
|
||||||
* Auto-detects environment and selects optimal storage and settings
|
|
||||||
*/
|
|
||||||
export async function createBrowserBrainyData(config = {}) {
|
|
||||||
// BrainyData already has environment detection and will automatically:
|
|
||||||
// - Use OPFS storage in browsers with fallback to Memory
|
|
||||||
// - Use FileSystem storage in Node.js
|
|
||||||
// - Request persistent storage when appropriate
|
|
||||||
const browserConfig = {
|
|
||||||
storage: {
|
|
||||||
requestPersistentStorage: true // Request persistent storage for better performance
|
|
||||||
},
|
|
||||||
...config
|
|
||||||
};
|
|
||||||
const brainyData = new BrainyData(browserConfig);
|
|
||||||
await brainyData.init();
|
|
||||||
return brainyData;
|
|
||||||
}
|
|
||||||
// Re-export types and constants for framework use
|
|
||||||
export { VerbType, NounType, BrainyData };
|
|
||||||
// Default export for easy importing
|
|
||||||
export default createBrowserBrainyData;
|
|
||||||
//# sourceMappingURL=browserFramework.js.map
|
|
||||||
1
dist/browserFramework.js.map
vendored
1
dist/browserFramework.js.map
vendored
|
|
@ -1 +0,0 @@
|
||||||
{"version":3,"file":"browserFramework.js","sourceRoot":"","sources":["../src/browserFramework.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAE,UAAU,EAAoB,MAAM,iBAAiB,CAAA;AAC9D,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,MAAM,uBAAuB,CAAA;AAE1D;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,uBAAuB,CAAC,SAAoC,EAAE;IAClF,uEAAuE;IACvE,yDAAyD;IACzD,sCAAsC;IACtC,gDAAgD;IAChD,MAAM,aAAa,GAAqB;QACtC,OAAO,EAAE;YACP,wBAAwB,EAAE,IAAI,CAAC,oDAAoD;SACpF;QACD,GAAG,MAAM;KACV,CAAA;IAED,MAAM,UAAU,GAAG,IAAI,UAAU,CAAC,aAAa,CAAC,CAAA;IAChD,MAAM,UAAU,CAAC,IAAI,EAAE,CAAA;IAEvB,OAAO,UAAU,CAAA;AACnB,CAAC;AAED,kDAAkD;AAClD,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,UAAU,EAAE,CAAA;AAGzC,oCAAoC;AACpC,eAAe,uBAAuB,CAAA"}
|
|
||||||
14
dist/browserFramework.minimal.d.ts
vendored
14
dist/browserFramework.minimal.d.ts
vendored
|
|
@ -1,14 +0,0 @@
|
||||||
/**
|
|
||||||
* Minimal Browser Framework Entry Point for Brainy
|
|
||||||
* Core MIT open source functionality only - no enterprise features
|
|
||||||
* Optimized for browser usage with all dependencies bundled
|
|
||||||
*/
|
|
||||||
import { BrainyData } from './brainyData.js';
|
|
||||||
import { VerbType, NounType } from './types/graphTypes.js';
|
|
||||||
/**
|
|
||||||
* Create a BrainyData instance optimized for browser usage
|
|
||||||
* Auto-detects environment and selects optimal storage and settings
|
|
||||||
*/
|
|
||||||
export declare function createBrowserBrainyData(config?: {}): Promise<BrainyData<any>>;
|
|
||||||
export { VerbType, NounType, BrainyData };
|
|
||||||
export default createBrowserBrainyData;
|
|
||||||
31
dist/browserFramework.minimal.js
vendored
31
dist/browserFramework.minimal.js
vendored
|
|
@ -1,31 +0,0 @@
|
||||||
/**
|
|
||||||
* Minimal Browser Framework Entry Point for Brainy
|
|
||||||
* Core MIT open source functionality only - no enterprise features
|
|
||||||
* Optimized for browser usage with all dependencies bundled
|
|
||||||
*/
|
|
||||||
import { BrainyData } from './brainyData.js';
|
|
||||||
import { VerbType, NounType } from './types/graphTypes.js';
|
|
||||||
/**
|
|
||||||
* Create a BrainyData instance optimized for browser usage
|
|
||||||
* Auto-detects environment and selects optimal storage and settings
|
|
||||||
*/
|
|
||||||
export async function createBrowserBrainyData(config = {}) {
|
|
||||||
// BrainyData already has environment detection and will automatically:
|
|
||||||
// - Use OPFS storage in browsers with fallback to Memory
|
|
||||||
// - Use FileSystem storage in Node.js
|
|
||||||
// - Request persistent storage when appropriate
|
|
||||||
const browserConfig = {
|
|
||||||
storage: {
|
|
||||||
requestPersistentStorage: true // Request persistent storage for better performance
|
|
||||||
},
|
|
||||||
...config
|
|
||||||
};
|
|
||||||
const brainyData = new BrainyData(browserConfig);
|
|
||||||
await brainyData.init();
|
|
||||||
return brainyData;
|
|
||||||
}
|
|
||||||
// Re-export core types and classes for browser use
|
|
||||||
export { VerbType, NounType, BrainyData };
|
|
||||||
// Default export for easy importing
|
|
||||||
export default createBrowserBrainyData;
|
|
||||||
//# sourceMappingURL=browserFramework.minimal.js.map
|
|
||||||
1
dist/browserFramework.minimal.js.map
vendored
1
dist/browserFramework.minimal.js.map
vendored
|
|
@ -1 +0,0 @@
|
||||||
{"version":3,"file":"browserFramework.minimal.js","sourceRoot":"","sources":["../src/browserFramework.minimal.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAA;AAC5C,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,MAAM,uBAAuB,CAAA;AAE1D;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,uBAAuB,CAAC,MAAM,GAAG,EAAE;IACrD,uEAAuE;IACvE,yDAAyD;IACzD,sCAAsC;IACtC,gDAAgD;IAChD,MAAM,aAAa,GAAG;QAClB,OAAO,EAAE;YACL,wBAAwB,EAAE,IAAI,CAAC,oDAAoD;SACtF;QACD,GAAG,MAAM;KACZ,CAAA;IAED,MAAM,UAAU,GAAG,IAAI,UAAU,CAAC,aAAa,CAAC,CAAA;IAChD,MAAM,UAAU,CAAC,IAAI,EAAE,CAAA;IACvB,OAAO,UAAU,CAAA;AACrB,CAAC;AAED,mDAAmD;AACnD,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,UAAU,EAAE,CAAA;AAEzC,oCAAoC;AACpC,eAAe,uBAAuB,CAAA"}
|
|
||||||
113
dist/chat/BrainyChat.d.ts
vendored
113
dist/chat/BrainyChat.d.ts
vendored
|
|
@ -1,113 +0,0 @@
|
||||||
/**
|
|
||||||
* BrainyChat - Magical Chat Command Center
|
|
||||||
*
|
|
||||||
* A smart chat system that leverages Brainy's standard noun/verb types
|
|
||||||
* to create intelligent, persistent conversations with automatic context loading.
|
|
||||||
*
|
|
||||||
* Key Features:
|
|
||||||
* - Uses standard NounType.Message for all chat messages
|
|
||||||
* - Employs VerbType.Communicates and VerbType.Precedes for conversation flow
|
|
||||||
* - Auto-discovery of previous sessions using Brainy's search capabilities
|
|
||||||
* - Hybrid architecture: basic chat (open source) + premium memory sync
|
|
||||||
*/
|
|
||||||
import { BrainyData } from '../brainyData.js';
|
|
||||||
export interface ChatMessage {
|
|
||||||
id: string;
|
|
||||||
content: string;
|
|
||||||
speaker: 'user' | 'assistant' | string;
|
|
||||||
sessionId: string;
|
|
||||||
timestamp: Date;
|
|
||||||
metadata?: {
|
|
||||||
model?: string;
|
|
||||||
usage?: {
|
|
||||||
prompt_tokens?: number;
|
|
||||||
completion_tokens?: number;
|
|
||||||
};
|
|
||||||
context?: Record<string, any>;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
export interface ChatSession {
|
|
||||||
id: string;
|
|
||||||
title?: string;
|
|
||||||
createdAt: Date;
|
|
||||||
lastMessageAt: Date;
|
|
||||||
messageCount: number;
|
|
||||||
participants: string[];
|
|
||||||
metadata?: {
|
|
||||||
tags?: string[];
|
|
||||||
summary?: string;
|
|
||||||
archived?: boolean;
|
|
||||||
premium?: boolean;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Enhanced BrainyChat with automatic context loading and intelligent memory
|
|
||||||
*
|
|
||||||
* This extends basic chat functionality with premium features when available
|
|
||||||
*/
|
|
||||||
export declare class BrainyChat {
|
|
||||||
private brainy;
|
|
||||||
private currentSessionId;
|
|
||||||
private sessionCache;
|
|
||||||
constructor(brainy: BrainyData);
|
|
||||||
/**
|
|
||||||
* Initialize chat system and auto-discover last session
|
|
||||||
* Uses Brainy's advanced search to find the most recent conversation
|
|
||||||
*/
|
|
||||||
initialize(): Promise<ChatSession | null>;
|
|
||||||
/**
|
|
||||||
* Start a new chat session
|
|
||||||
* Automatically generates a session ID and stores session metadata
|
|
||||||
*/
|
|
||||||
startNewSession(title?: string, participants?: string[]): Promise<ChatSession>;
|
|
||||||
/**
|
|
||||||
* Add a message to the current session
|
|
||||||
* Stores using standard NounType.Message and creates conversation flow relationships
|
|
||||||
*/
|
|
||||||
addMessage(content: string, speaker?: string, metadata?: ChatMessage['metadata']): Promise<ChatMessage>;
|
|
||||||
/**
|
|
||||||
* Get conversation history for current session
|
|
||||||
* Uses Brainy's graph traversal to get messages in chronological order
|
|
||||||
*/
|
|
||||||
getHistory(limit?: number): Promise<ChatMessage[]>;
|
|
||||||
/**
|
|
||||||
* Search across all chat sessions and messages
|
|
||||||
* Leverages Brainy's powerful vector and semantic search
|
|
||||||
*/
|
|
||||||
searchMessages(query: string, options?: {
|
|
||||||
sessionId?: string;
|
|
||||||
speaker?: string;
|
|
||||||
limit?: number;
|
|
||||||
semanticSearch?: boolean;
|
|
||||||
}): Promise<ChatMessage[]>;
|
|
||||||
/**
|
|
||||||
* Get all chat sessions
|
|
||||||
* Uses Brainy's search to find all conversation sessions
|
|
||||||
*/
|
|
||||||
getSessions(limit?: number): Promise<ChatSession[]>;
|
|
||||||
/**
|
|
||||||
* Switch to a different session
|
|
||||||
* Automatically loads context and history
|
|
||||||
*/
|
|
||||||
switchToSession(sessionId: string): Promise<ChatSession | null>;
|
|
||||||
/**
|
|
||||||
* Archive a session (premium feature)
|
|
||||||
* Maintains full searchability while organizing conversations
|
|
||||||
*/
|
|
||||||
archiveSession(sessionId: string): Promise<boolean>;
|
|
||||||
/**
|
|
||||||
* Generate session summary using AI (premium feature)
|
|
||||||
* Intelligently summarizes long conversations
|
|
||||||
*/
|
|
||||||
generateSessionSummary(sessionId: string): Promise<string | null>;
|
|
||||||
private createMessageRelationships;
|
|
||||||
private loadSession;
|
|
||||||
private getHistoryForSession;
|
|
||||||
private updateSessionMetadata;
|
|
||||||
private nounToChatMessage;
|
|
||||||
private nounToChatSession;
|
|
||||||
private toTimestamp;
|
|
||||||
private isPremiumEnabled;
|
|
||||||
getCurrentSessionId(): string | null;
|
|
||||||
getCurrentSession(): ChatSession | null;
|
|
||||||
}
|
|
||||||
368
dist/chat/BrainyChat.js
vendored
368
dist/chat/BrainyChat.js
vendored
|
|
@ -1,368 +0,0 @@
|
||||||
/**
|
|
||||||
* BrainyChat - Magical Chat Command Center
|
|
||||||
*
|
|
||||||
* A smart chat system that leverages Brainy's standard noun/verb types
|
|
||||||
* to create intelligent, persistent conversations with automatic context loading.
|
|
||||||
*
|
|
||||||
* Key Features:
|
|
||||||
* - Uses standard NounType.Message for all chat messages
|
|
||||||
* - Employs VerbType.Communicates and VerbType.Precedes for conversation flow
|
|
||||||
* - Auto-discovery of previous sessions using Brainy's search capabilities
|
|
||||||
* - Hybrid architecture: basic chat (open source) + premium memory sync
|
|
||||||
*/
|
|
||||||
import { NounType, VerbType } from '../types/graphTypes.js';
|
|
||||||
/**
|
|
||||||
* Enhanced BrainyChat with automatic context loading and intelligent memory
|
|
||||||
*
|
|
||||||
* This extends basic chat functionality with premium features when available
|
|
||||||
*/
|
|
||||||
export class BrainyChat {
|
|
||||||
constructor(brainy) {
|
|
||||||
this.currentSessionId = null;
|
|
||||||
this.sessionCache = new Map();
|
|
||||||
this.brainy = brainy;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Initialize chat system and auto-discover last session
|
|
||||||
* Uses Brainy's advanced search to find the most recent conversation
|
|
||||||
*/
|
|
||||||
async initialize() {
|
|
||||||
try {
|
|
||||||
// Search for the most recent chat message using Brainy's search
|
|
||||||
const recentMessages = await this.brainy.search('recent chat conversation', 1, {
|
|
||||||
nounTypes: [NounType.Message],
|
|
||||||
metadata: {
|
|
||||||
messageType: 'chat'
|
|
||||||
}
|
|
||||||
});
|
|
||||||
if (recentMessages.length > 0) {
|
|
||||||
const lastMessage = recentMessages[0];
|
|
||||||
const sessionId = lastMessage.metadata?.sessionId;
|
|
||||||
if (sessionId) {
|
|
||||||
this.currentSessionId = sessionId;
|
|
||||||
return await this.loadSession(sessionId);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.debug('No previous session found, starting fresh:', error?.message);
|
|
||||||
}
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Start a new chat session
|
|
||||||
* Automatically generates a session ID and stores session metadata
|
|
||||||
*/
|
|
||||||
async startNewSession(title, participants = ['user', 'assistant']) {
|
|
||||||
const sessionId = `chat-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
|
|
||||||
const session = {
|
|
||||||
id: sessionId,
|
|
||||||
title,
|
|
||||||
createdAt: new Date(),
|
|
||||||
lastMessageAt: new Date(),
|
|
||||||
messageCount: 0,
|
|
||||||
participants,
|
|
||||||
metadata: {
|
|
||||||
tags: ['active'],
|
|
||||||
premium: await this.isPremiumEnabled()
|
|
||||||
}
|
|
||||||
};
|
|
||||||
// Store session using BrainyData add() method
|
|
||||||
await this.brainy.add({
|
|
||||||
sessionType: 'chat',
|
|
||||||
title: title || `Chat Session ${new Date().toLocaleDateString()}`,
|
|
||||||
createdAt: session.createdAt.toISOString(),
|
|
||||||
lastMessageAt: session.lastMessageAt.toISOString(),
|
|
||||||
messageCount: session.messageCount,
|
|
||||||
participants: session.participants
|
|
||||||
}, {
|
|
||||||
id: sessionId,
|
|
||||||
nounType: NounType.Concept,
|
|
||||||
sessionType: 'chat'
|
|
||||||
});
|
|
||||||
this.currentSessionId = sessionId;
|
|
||||||
this.sessionCache.set(sessionId, session);
|
|
||||||
return session;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Add a message to the current session
|
|
||||||
* Stores using standard NounType.Message and creates conversation flow relationships
|
|
||||||
*/
|
|
||||||
async addMessage(content, speaker = 'user', metadata) {
|
|
||||||
if (!this.currentSessionId) {
|
|
||||||
await this.startNewSession();
|
|
||||||
}
|
|
||||||
const messageId = `msg-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
|
|
||||||
const timestamp = new Date();
|
|
||||||
const message = {
|
|
||||||
id: messageId,
|
|
||||||
content,
|
|
||||||
speaker,
|
|
||||||
sessionId: this.currentSessionId,
|
|
||||||
timestamp,
|
|
||||||
metadata
|
|
||||||
};
|
|
||||||
// Store message using BrainyData add() method
|
|
||||||
await this.brainy.add({
|
|
||||||
messageType: 'chat',
|
|
||||||
content,
|
|
||||||
speaker,
|
|
||||||
sessionId: this.currentSessionId,
|
|
||||||
timestamp: timestamp.toISOString(),
|
|
||||||
...metadata
|
|
||||||
}, {
|
|
||||||
id: messageId,
|
|
||||||
nounType: NounType.Message,
|
|
||||||
messageType: 'chat',
|
|
||||||
sessionId: this.currentSessionId,
|
|
||||||
speaker
|
|
||||||
});
|
|
||||||
// Create relationships using standard verb types
|
|
||||||
await this.createMessageRelationships(messageId);
|
|
||||||
// Update session metadata
|
|
||||||
await this.updateSessionMetadata();
|
|
||||||
return message;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get conversation history for current session
|
|
||||||
* Uses Brainy's graph traversal to get messages in chronological order
|
|
||||||
*/
|
|
||||||
async getHistory(limit = 50) {
|
|
||||||
if (!this.currentSessionId)
|
|
||||||
return [];
|
|
||||||
try {
|
|
||||||
// Search for messages in this session using Brainy's search
|
|
||||||
const messageNouns = await this.brainy.search('', // Empty query to get all messages
|
|
||||||
limit, {
|
|
||||||
nounTypes: [NounType.Message],
|
|
||||||
metadata: {
|
|
||||||
sessionId: this.currentSessionId,
|
|
||||||
messageType: 'chat'
|
|
||||||
}
|
|
||||||
});
|
|
||||||
return messageNouns.map((noun) => this.nounToChatMessage(noun));
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error retrieving chat history:', error);
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Search across all chat sessions and messages
|
|
||||||
* Leverages Brainy's powerful vector and semantic search
|
|
||||||
*/
|
|
||||||
async searchMessages(query, options) {
|
|
||||||
const metadata = {
|
|
||||||
messageType: 'chat'
|
|
||||||
};
|
|
||||||
if (options?.sessionId) {
|
|
||||||
metadata.sessionId = options.sessionId;
|
|
||||||
}
|
|
||||||
if (options?.speaker) {
|
|
||||||
metadata.speaker = options.speaker;
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
const results = await this.brainy.search(options?.semanticSearch !== false ? query : '', options?.limit || 20, {
|
|
||||||
nounTypes: [NounType.Message],
|
|
||||||
metadata
|
|
||||||
});
|
|
||||||
return results.map((noun) => this.nounToChatMessage(noun));
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error searching messages:', error);
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get all chat sessions
|
|
||||||
* Uses Brainy's search to find all conversation sessions
|
|
||||||
*/
|
|
||||||
async getSessions(limit = 20) {
|
|
||||||
try {
|
|
||||||
const sessionNouns = await this.brainy.search('', limit, {
|
|
||||||
nounTypes: [NounType.Concept],
|
|
||||||
metadata: {
|
|
||||||
sessionType: 'chat'
|
|
||||||
}
|
|
||||||
});
|
|
||||||
return sessionNouns.map((noun) => this.nounToChatSession(noun));
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error retrieving sessions:', error);
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Switch to a different session
|
|
||||||
* Automatically loads context and history
|
|
||||||
*/
|
|
||||||
async switchToSession(sessionId) {
|
|
||||||
try {
|
|
||||||
const session = await this.loadSession(sessionId);
|
|
||||||
if (session) {
|
|
||||||
this.currentSessionId = sessionId;
|
|
||||||
this.sessionCache.set(sessionId, session);
|
|
||||||
}
|
|
||||||
return session;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error switching to session:', error);
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Archive a session (premium feature)
|
|
||||||
* Maintains full searchability while organizing conversations
|
|
||||||
*/
|
|
||||||
async archiveSession(sessionId) {
|
|
||||||
if (!await this.isPremiumEnabled()) {
|
|
||||||
throw new Error('Session archiving requires premium Brain Cloud subscription');
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
// Since BrainyData doesn't have update, add an archive marker
|
|
||||||
await this.brainy.add({
|
|
||||||
archivedSessionId: sessionId,
|
|
||||||
archivedAt: new Date().toISOString(),
|
|
||||||
action: 'archive'
|
|
||||||
}, {
|
|
||||||
nounType: NounType.State,
|
|
||||||
sessionId,
|
|
||||||
archived: true
|
|
||||||
});
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error archiving session:', error);
|
|
||||||
}
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Generate session summary using AI (premium feature)
|
|
||||||
* Intelligently summarizes long conversations
|
|
||||||
*/
|
|
||||||
async generateSessionSummary(sessionId) {
|
|
||||||
if (!await this.isPremiumEnabled()) {
|
|
||||||
throw new Error('AI session summaries require premium Brain Cloud subscription');
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
const messages = await this.getHistoryForSession(sessionId, 100);
|
|
||||||
const content = messages
|
|
||||||
.map(msg => `${msg.speaker}: ${msg.content}`)
|
|
||||||
.join('\n');
|
|
||||||
// Use Brainy's AI to generate summary (placeholder - would need actual AI integration)
|
|
||||||
const summaryResponse = `Summary of ${messages.length} messages discussing various topics in ${sessionId}`;
|
|
||||||
return summaryResponse || null;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error generating session summary:', error);
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Private helper methods
|
|
||||||
async createMessageRelationships(messageId) {
|
|
||||||
// Link message to session using unified addVerb API
|
|
||||||
await this.brainy.addVerb(messageId, this.currentSessionId, VerbType.PartOf, {
|
|
||||||
relationship: 'message-in-session'
|
|
||||||
});
|
|
||||||
// Find previous message to create conversation flow using VerbType.Precedes
|
|
||||||
const previousMessages = await this.brainy.search('', 1, {
|
|
||||||
nounTypes: [NounType.Message],
|
|
||||||
metadata: {
|
|
||||||
sessionId: this.currentSessionId,
|
|
||||||
messageType: 'chat'
|
|
||||||
}
|
|
||||||
});
|
|
||||||
if (previousMessages.length > 0 && previousMessages[0].id !== messageId) {
|
|
||||||
await this.brainy.addVerb(previousMessages[0].id, messageId, VerbType.Precedes, {
|
|
||||||
relationship: 'message-sequence'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async loadSession(sessionId) {
|
|
||||||
try {
|
|
||||||
const sessionNouns = await this.brainy.search('', 1, {
|
|
||||||
nounTypes: [NounType.Concept],
|
|
||||||
metadata: {
|
|
||||||
sessionType: 'chat'
|
|
||||||
}
|
|
||||||
});
|
|
||||||
// Filter by session ID manually since BrainyData search may not support ID filtering
|
|
||||||
const matchingSession = sessionNouns.find(noun => noun.id === sessionId);
|
|
||||||
if (matchingSession) {
|
|
||||||
return this.nounToChatSession(matchingSession);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error loading session:', error);
|
|
||||||
}
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
async getHistoryForSession(sessionId, limit = 50) {
|
|
||||||
try {
|
|
||||||
const messageNouns = await this.brainy.search('', limit, {
|
|
||||||
nounTypes: [NounType.Message],
|
|
||||||
metadata: {
|
|
||||||
sessionId: sessionId,
|
|
||||||
messageType: 'chat'
|
|
||||||
}
|
|
||||||
});
|
|
||||||
return messageNouns.map((noun) => this.nounToChatMessage(noun));
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error('Error retrieving session history:', error);
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async updateSessionMetadata() {
|
|
||||||
if (!this.currentSessionId)
|
|
||||||
return;
|
|
||||||
// Since BrainyData doesn't have update functionality, we'll skip this
|
|
||||||
// In a real implementation, you'd need update capabilities
|
|
||||||
console.debug('Session metadata update skipped - BrainyData lacks update API');
|
|
||||||
}
|
|
||||||
nounToChatMessage(noun) {
|
|
||||||
return {
|
|
||||||
id: noun.id,
|
|
||||||
content: noun.metadata?.content || noun.data?.content || '',
|
|
||||||
speaker: noun.metadata?.speaker || noun.data?.speaker || 'unknown',
|
|
||||||
sessionId: noun.metadata?.sessionId || noun.data?.sessionId || '',
|
|
||||||
timestamp: new Date(noun.metadata?.timestamp || noun.data?.timestamp || Date.now()),
|
|
||||||
metadata: noun.metadata
|
|
||||||
};
|
|
||||||
}
|
|
||||||
nounToChatSession(noun) {
|
|
||||||
return {
|
|
||||||
id: noun.id,
|
|
||||||
title: noun.metadata?.title || noun.data?.title || 'Untitled Session',
|
|
||||||
createdAt: new Date(noun.metadata?.createdAt || noun.data?.createdAt || Date.now()),
|
|
||||||
lastMessageAt: new Date(noun.metadata?.lastMessageAt || noun.data?.lastMessageAt || Date.now()),
|
|
||||||
messageCount: noun.metadata?.messageCount || noun.data?.messageCount || 0,
|
|
||||||
participants: noun.metadata?.participants || noun.data?.participants || ['user', 'assistant'],
|
|
||||||
metadata: noun.metadata
|
|
||||||
};
|
|
||||||
}
|
|
||||||
toTimestamp(date) {
|
|
||||||
const seconds = Math.floor(date.getTime() / 1000);
|
|
||||||
const nanoseconds = (date.getTime() % 1000) * 1000000;
|
|
||||||
return { seconds, nanoseconds };
|
|
||||||
}
|
|
||||||
async isPremiumEnabled() {
|
|
||||||
// Check if premium augmentations are available
|
|
||||||
// This would integrate with the license validation system
|
|
||||||
try {
|
|
||||||
const augmentations = await this.brainy.listAugmentations();
|
|
||||||
return augmentations.some((aug) => aug.premium === true && aug.enabled === true);
|
|
||||||
}
|
|
||||||
catch {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Public API methods for CLI integration
|
|
||||||
getCurrentSessionId() {
|
|
||||||
return this.currentSessionId;
|
|
||||||
}
|
|
||||||
getCurrentSession() {
|
|
||||||
return this.currentSessionId ? this.sessionCache.get(this.currentSessionId) || null : null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=BrainyChat.js.map
|
|
||||||
1
dist/chat/BrainyChat.js.map
vendored
1
dist/chat/BrainyChat.js.map
vendored
File diff suppressed because one or more lines are too long
61
dist/chat/ChatCLI.d.ts
vendored
61
dist/chat/ChatCLI.d.ts
vendored
|
|
@ -1,61 +0,0 @@
|
||||||
/**
|
|
||||||
* ChatCLI - Command Line Interface for BrainyChat
|
|
||||||
*
|
|
||||||
* Provides a magical chat experience through the Brainy CLI with:
|
|
||||||
* - Auto-discovery of previous sessions
|
|
||||||
* - Intelligent context loading
|
|
||||||
* - Multi-agent coordination support
|
|
||||||
* - Premium memory sync integration
|
|
||||||
*/
|
|
||||||
import { type ChatMessage } from './BrainyChat.js';
|
|
||||||
import { BrainyData } from '../brainyData.js';
|
|
||||||
export declare class ChatCLI {
|
|
||||||
private brainyChat;
|
|
||||||
private brainy;
|
|
||||||
constructor(brainy: BrainyData);
|
|
||||||
/**
|
|
||||||
* Start an interactive chat session
|
|
||||||
* Automatically discovers and loads previous context
|
|
||||||
*/
|
|
||||||
startInteractiveChat(options?: {
|
|
||||||
sessionId?: string;
|
|
||||||
speaker?: string;
|
|
||||||
memory?: boolean;
|
|
||||||
newSession?: boolean;
|
|
||||||
}): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Send a single message and get response
|
|
||||||
*/
|
|
||||||
sendMessage(message: string, options?: {
|
|
||||||
sessionId?: string;
|
|
||||||
speaker?: string;
|
|
||||||
noResponse?: boolean;
|
|
||||||
}): Promise<ChatMessage[]>;
|
|
||||||
/**
|
|
||||||
* Show conversation history
|
|
||||||
*/
|
|
||||||
showHistory(limit?: number): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Search across all conversations
|
|
||||||
*/
|
|
||||||
searchConversations(query: string, options?: {
|
|
||||||
limit?: number;
|
|
||||||
sessionId?: string;
|
|
||||||
semantic?: boolean;
|
|
||||||
}): Promise<void>;
|
|
||||||
/**
|
|
||||||
* List all chat sessions
|
|
||||||
*/
|
|
||||||
listSessions(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Switch to a different session
|
|
||||||
*/
|
|
||||||
switchSession(sessionId: string): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Show help for chat commands
|
|
||||||
*/
|
|
||||||
showHelp(): void;
|
|
||||||
private interactiveLoop;
|
|
||||||
private showRecentContext;
|
|
||||||
private generateResponse;
|
|
||||||
}
|
|
||||||
351
dist/chat/ChatCLI.js
vendored
351
dist/chat/ChatCLI.js
vendored
|
|
@ -1,351 +0,0 @@
|
||||||
/**
|
|
||||||
* ChatCLI - Command Line Interface for BrainyChat
|
|
||||||
*
|
|
||||||
* Provides a magical chat experience through the Brainy CLI with:
|
|
||||||
* - Auto-discovery of previous sessions
|
|
||||||
* - Intelligent context loading
|
|
||||||
* - Multi-agent coordination support
|
|
||||||
* - Premium memory sync integration
|
|
||||||
*/
|
|
||||||
import { BrainyChat } from './BrainyChat.js';
|
|
||||||
// Simple color utility without external dependencies
|
|
||||||
const colors = {
|
|
||||||
cyan: (text) => `\x1b[36m${text}\x1b[0m`,
|
|
||||||
green: (text) => `\x1b[32m${text}\x1b[0m`,
|
|
||||||
yellow: (text) => `\x1b[33m${text}\x1b[0m`,
|
|
||||||
blue: (text) => `\x1b[34m${text}\x1b[0m`,
|
|
||||||
gray: (text) => `\x1b[90m${text}\x1b[0m`,
|
|
||||||
red: (text) => `\x1b[31m${text}\x1b[0m`
|
|
||||||
};
|
|
||||||
export class ChatCLI {
|
|
||||||
constructor(brainy) {
|
|
||||||
this.brainy = brainy;
|
|
||||||
this.brainyChat = new BrainyChat(brainy);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Start an interactive chat session
|
|
||||||
* Automatically discovers and loads previous context
|
|
||||||
*/
|
|
||||||
async startInteractiveChat(options) {
|
|
||||||
console.log(colors.cyan('🧠 Brainy Chat - Local Memory & Intelligence'));
|
|
||||||
console.log();
|
|
||||||
let session = null;
|
|
||||||
if (options?.sessionId) {
|
|
||||||
// Load specific session
|
|
||||||
session = await this.brainyChat.switchToSession(options.sessionId);
|
|
||||||
if (session) {
|
|
||||||
console.log(colors.green(`📂 Loaded session: ${session.title || session.id}`));
|
|
||||||
console.log(colors.gray(` Created: ${session.createdAt.toLocaleDateString()}`));
|
|
||||||
console.log(colors.gray(` Messages: ${session.messageCount}`));
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
console.log(colors.yellow(`⚠️ Session ${options.sessionId} not found, starting new session`));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
else if (!options?.newSession) {
|
|
||||||
// Auto-discover last session
|
|
||||||
console.log(colors.gray('🔍 Looking for your last conversation...'));
|
|
||||||
session = await this.brainyChat.initialize();
|
|
||||||
if (session) {
|
|
||||||
console.log(colors.green(`✨ Found your last session: ${session.title || 'Untitled'}`));
|
|
||||||
console.log(colors.gray(` Last active: ${session.lastMessageAt.toLocaleString()}`));
|
|
||||||
console.log(colors.gray(` Messages: ${session.messageCount}`));
|
|
||||||
// Show recent context if memory option is enabled
|
|
||||||
if (options?.memory !== false) {
|
|
||||||
await this.showRecentContext();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
console.log(colors.blue('🆕 No previous sessions found, starting fresh!'));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (!session) {
|
|
||||||
session = await this.brainyChat.startNewSession(`Chat ${new Date().toLocaleDateString()}`, ['user', options?.speaker || 'assistant']);
|
|
||||||
console.log(colors.green(`🎉 Started new session: ${session.id}`));
|
|
||||||
}
|
|
||||||
console.log();
|
|
||||||
console.log(colors.gray('💡 Tips:'));
|
|
||||||
console.log(colors.gray(' - Type /history to see conversation history'));
|
|
||||||
console.log(colors.gray(' - Type /search <query> to search all conversations'));
|
|
||||||
console.log(colors.gray(' - Type /sessions to list all sessions'));
|
|
||||||
console.log(colors.gray(' - Type /help for more commands'));
|
|
||||||
console.log(colors.gray(' - Type /quit to exit'));
|
|
||||||
console.log();
|
|
||||||
console.log(colors.blue('🚀 Want multi-agent coordination? Try: brainy cloud auth'));
|
|
||||||
console.log();
|
|
||||||
// Start interactive loop
|
|
||||||
await this.interactiveLoop(options?.speaker || 'assistant');
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Send a single message and get response
|
|
||||||
*/
|
|
||||||
async sendMessage(message, options) {
|
|
||||||
if (options?.sessionId) {
|
|
||||||
await this.brainyChat.switchToSession(options.sessionId);
|
|
||||||
}
|
|
||||||
// Add user message
|
|
||||||
const userMessage = await this.brainyChat.addMessage(message, 'user');
|
|
||||||
console.log(colors.blue(`👤 You: ${message}`));
|
|
||||||
if (options?.noResponse) {
|
|
||||||
return [userMessage];
|
|
||||||
}
|
|
||||||
// For CLI usage, we'd integrate with whatever AI service is configured
|
|
||||||
// This is a placeholder showing the architecture
|
|
||||||
const response = await this.generateResponse(message, options?.speaker || 'assistant');
|
|
||||||
const assistantMessage = await this.brainyChat.addMessage(response, options?.speaker || 'assistant', {
|
|
||||||
model: 'claude-3-sonnet',
|
|
||||||
context: { userMessage: userMessage.id }
|
|
||||||
});
|
|
||||||
console.log(colors.green(`🤖 ${options?.speaker || 'Assistant'}: ${response}`));
|
|
||||||
return [userMessage, assistantMessage];
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Show conversation history
|
|
||||||
*/
|
|
||||||
async showHistory(limit = 10) {
|
|
||||||
const messages = await this.brainyChat.getHistory(limit);
|
|
||||||
if (messages.length === 0) {
|
|
||||||
console.log(colors.yellow('📭 No messages in current session'));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
console.log(colors.cyan(`📜 Last ${Math.min(limit, messages.length)} messages:`));
|
|
||||||
console.log();
|
|
||||||
for (const message of messages.slice(-limit)) {
|
|
||||||
const timestamp = message.timestamp.toLocaleTimeString();
|
|
||||||
const speakerColor = message.speaker === 'user' ? colors.blue : colors.green;
|
|
||||||
const icon = message.speaker === 'user' ? '👤' : '🤖';
|
|
||||||
console.log(speakerColor(`${icon} ${message.speaker} (${timestamp}):`));
|
|
||||||
console.log(colors.gray(` ${message.content}`));
|
|
||||||
console.log();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Search across all conversations
|
|
||||||
*/
|
|
||||||
async searchConversations(query, options) {
|
|
||||||
console.log(colors.cyan(`🔍 Searching for: "${query}"`));
|
|
||||||
const results = await this.brainyChat.searchMessages(query, {
|
|
||||||
limit: options?.limit || 10,
|
|
||||||
sessionId: options?.sessionId,
|
|
||||||
semanticSearch: options?.semantic !== false
|
|
||||||
});
|
|
||||||
if (results.length === 0) {
|
|
||||||
console.log(colors.yellow('🤷 No matching messages found'));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
console.log(colors.green(`✨ Found ${results.length} matches:`));
|
|
||||||
console.log();
|
|
||||||
for (const message of results) {
|
|
||||||
const date = message.timestamp.toLocaleDateString();
|
|
||||||
const time = message.timestamp.toLocaleTimeString();
|
|
||||||
const speakerColor = message.speaker === 'user' ? colors.blue : colors.green;
|
|
||||||
const icon = message.speaker === 'user' ? '👤' : '🤖';
|
|
||||||
console.log(colors.gray(`📅 ${date} ${time} - Session: ${message.sessionId.substring(0, 8)}...`));
|
|
||||||
console.log(speakerColor(`${icon} ${message.speaker}: ${message.content}`));
|
|
||||||
console.log();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* List all chat sessions
|
|
||||||
*/
|
|
||||||
async listSessions() {
|
|
||||||
const sessions = await this.brainyChat.getSessions();
|
|
||||||
if (sessions.length === 0) {
|
|
||||||
console.log(colors.yellow('📭 No chat sessions found'));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
console.log(colors.cyan(`💬 Your chat sessions (${sessions.length}):`));
|
|
||||||
console.log();
|
|
||||||
for (const session of sessions) {
|
|
||||||
const isActive = session.id === this.brainyChat.getCurrentSessionId();
|
|
||||||
const activeIndicator = isActive ? colors.green(' ● ACTIVE') : '';
|
|
||||||
const archived = session.metadata?.archived ? colors.gray(' [ARCHIVED]') : '';
|
|
||||||
console.log(colors.blue(`📂 ${session.title || 'Untitled'}${activeIndicator}${archived}`));
|
|
||||||
console.log(colors.gray(` ID: ${session.id}`));
|
|
||||||
console.log(colors.gray(` Created: ${session.createdAt.toLocaleDateString()}`));
|
|
||||||
console.log(colors.gray(` Last active: ${session.lastMessageAt.toLocaleDateString()}`));
|
|
||||||
console.log(colors.gray(` Messages: ${session.messageCount}`));
|
|
||||||
console.log(colors.gray(` Participants: ${session.participants.join(', ')}`));
|
|
||||||
console.log();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Switch to a different session
|
|
||||||
*/
|
|
||||||
async switchSession(sessionId) {
|
|
||||||
const session = await this.brainyChat.switchToSession(sessionId);
|
|
||||||
if (session) {
|
|
||||||
console.log(colors.green(`✅ Switched to session: ${session.title || session.id}`));
|
|
||||||
console.log(colors.gray(` Messages: ${session.messageCount}`));
|
|
||||||
console.log(colors.gray(` Last active: ${session.lastMessageAt.toLocaleString()}`));
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
console.log(colors.red(`❌ Session ${sessionId} not found`));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Show help for chat commands
|
|
||||||
*/
|
|
||||||
showHelp() {
|
|
||||||
console.log(colors.cyan('🧠 Brainy Chat Commands:'));
|
|
||||||
console.log();
|
|
||||||
console.log(colors.blue('Basic Commands:'));
|
|
||||||
console.log(' /history [limit] - Show conversation history (default: 10 messages)');
|
|
||||||
console.log(' /search <query> - Search all conversations');
|
|
||||||
console.log(' /sessions - List all chat sessions');
|
|
||||||
console.log(' /switch <id> - Switch to a specific session');
|
|
||||||
console.log(' /new - Start a new session');
|
|
||||||
console.log(' /help - Show this help');
|
|
||||||
console.log(' /quit - Exit chat');
|
|
||||||
console.log();
|
|
||||||
console.log(colors.yellow('Local Features:'));
|
|
||||||
console.log(' ✨ Automatic session discovery');
|
|
||||||
console.log(' 🧠 Local memory across all conversations');
|
|
||||||
console.log(' 🔍 Semantic search using vector similarity');
|
|
||||||
console.log(' 📊 Standard noun/verb graph relationships');
|
|
||||||
console.log();
|
|
||||||
console.log(colors.green('Want More? Premium Features:'));
|
|
||||||
console.log(' 🤝 Multi-agent coordination');
|
|
||||||
console.log(' ☁️ Cross-device memory sync');
|
|
||||||
console.log(' 🎨 Rich web coordination UI');
|
|
||||||
console.log(' 🔄 Real-time team collaboration');
|
|
||||||
console.log();
|
|
||||||
console.log(colors.blue('Get premium: brainy cloud auth'));
|
|
||||||
console.log();
|
|
||||||
}
|
|
||||||
// Private methods
|
|
||||||
async interactiveLoop(assistantSpeaker = 'assistant') {
|
|
||||||
const readline = require('readline');
|
|
||||||
const rl = readline.createInterface({
|
|
||||||
input: process.stdin,
|
|
||||||
output: process.stdout
|
|
||||||
});
|
|
||||||
const askQuestion = () => {
|
|
||||||
return new Promise((resolve) => {
|
|
||||||
rl.question(colors.blue('💬 You: '), resolve);
|
|
||||||
});
|
|
||||||
};
|
|
||||||
while (true) {
|
|
||||||
try {
|
|
||||||
const input = await askQuestion();
|
|
||||||
if (input.trim() === '')
|
|
||||||
continue;
|
|
||||||
// Handle commands
|
|
||||||
if (input.startsWith('/')) {
|
|
||||||
const [command, ...args] = input.slice(1).split(' ');
|
|
||||||
switch (command.toLowerCase()) {
|
|
||||||
case 'quit':
|
|
||||||
case 'exit':
|
|
||||||
console.log(colors.cyan('👋 Thanks for chatting! Your conversation is saved.'));
|
|
||||||
rl.close();
|
|
||||||
return;
|
|
||||||
case 'history':
|
|
||||||
const limit = args[0] ? parseInt(args[0]) : 10;
|
|
||||||
await this.showHistory(limit);
|
|
||||||
break;
|
|
||||||
case 'search':
|
|
||||||
if (args.length === 0) {
|
|
||||||
console.log(colors.yellow('Usage: /search <query>'));
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
await this.searchConversations(args.join(' '));
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
case 'sessions':
|
|
||||||
await this.listSessions();
|
|
||||||
break;
|
|
||||||
case 'switch':
|
|
||||||
if (args.length === 0) {
|
|
||||||
console.log(colors.yellow('Usage: /switch <session-id>'));
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
await this.switchSession(args[0]);
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
case 'new':
|
|
||||||
const newSession = await this.brainyChat.startNewSession(`Chat ${new Date().toLocaleDateString()}`);
|
|
||||||
console.log(colors.green(`🆕 Started new session: ${newSession.id}`));
|
|
||||||
break;
|
|
||||||
case 'archive':
|
|
||||||
const sessionToArchive = args[0] || this.brainyChat.getCurrentSessionId();
|
|
||||||
if (sessionToArchive) {
|
|
||||||
try {
|
|
||||||
await this.brainyChat.archiveSession(sessionToArchive);
|
|
||||||
console.log(colors.green(`📁 Session archived: ${sessionToArchive}`));
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.log(colors.red(`❌ ${error?.message}`));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
console.log(colors.yellow('No session to archive'));
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
case 'summary':
|
|
||||||
const sessionToSummarize = args[0] || this.brainyChat.getCurrentSessionId();
|
|
||||||
if (sessionToSummarize) {
|
|
||||||
try {
|
|
||||||
const summary = await this.brainyChat.generateSessionSummary(sessionToSummarize);
|
|
||||||
if (summary) {
|
|
||||||
console.log(colors.green('📋 Session Summary:'));
|
|
||||||
console.log(colors.gray(summary));
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
console.log(colors.yellow('No summary could be generated'));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.log(colors.red(`❌ ${error?.message}`));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
console.log(colors.yellow('No session to summarize'));
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
case 'help':
|
|
||||||
this.showHelp();
|
|
||||||
break;
|
|
||||||
default:
|
|
||||||
console.log(colors.yellow(`Unknown command: ${command}`));
|
|
||||||
console.log(colors.gray('Type /help for available commands'));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
// Regular message
|
|
||||||
await this.sendMessage(input, { speaker: assistantSpeaker });
|
|
||||||
}
|
|
||||||
console.log();
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.error(colors.red(`Error: ${error?.message}`));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async showRecentContext(limit = 3) {
|
|
||||||
const recentMessages = await this.brainyChat.getHistory(limit);
|
|
||||||
if (recentMessages.length > 0) {
|
|
||||||
console.log(colors.gray('💭 Recent context:'));
|
|
||||||
for (const msg of recentMessages.slice(-limit)) {
|
|
||||||
const preview = msg.content.length > 60
|
|
||||||
? msg.content.substring(0, 60) + '...'
|
|
||||||
: msg.content;
|
|
||||||
console.log(colors.gray(` ${msg.speaker}: ${preview}`));
|
|
||||||
}
|
|
||||||
console.log();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async generateResponse(message, speaker) {
|
|
||||||
// This is a placeholder for AI integration
|
|
||||||
// In a real implementation, this would call the configured AI service
|
|
||||||
// and could include multi-agent coordination
|
|
||||||
// Example responses for demonstration
|
|
||||||
const responses = [
|
|
||||||
"I remember our conversation and can help with that!",
|
|
||||||
"Based on our previous discussions, I think...",
|
|
||||||
"Let me search through our chat history for relevant context.",
|
|
||||||
"I can coordinate with other AI agents if needed for this task."
|
|
||||||
];
|
|
||||||
return responses[Math.floor(Math.random() * responses.length)];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=ChatCLI.js.map
|
|
||||||
1
dist/chat/ChatCLI.js.map
vendored
1
dist/chat/ChatCLI.js.map
vendored
File diff suppressed because one or more lines are too long
47
dist/cli/catalog.d.ts
vendored
47
dist/cli/catalog.d.ts
vendored
|
|
@ -1,47 +0,0 @@
|
||||||
/**
|
|
||||||
* Brain Cloud Catalog Integration for CLI
|
|
||||||
*
|
|
||||||
* Fetches and displays augmentation catalog
|
|
||||||
* Falls back to local cache if API is unavailable
|
|
||||||
*/
|
|
||||||
interface Augmentation {
|
|
||||||
id: string;
|
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
category: string;
|
|
||||||
status: 'available' | 'coming_soon' | 'deprecated';
|
|
||||||
popular?: boolean;
|
|
||||||
eta?: string;
|
|
||||||
}
|
|
||||||
interface Category {
|
|
||||||
id: string;
|
|
||||||
name: string;
|
|
||||||
icon: string;
|
|
||||||
description: string;
|
|
||||||
}
|
|
||||||
interface Catalog {
|
|
||||||
version: string;
|
|
||||||
categories: Category[];
|
|
||||||
augmentations: Augmentation[];
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Fetch catalog from API with caching
|
|
||||||
*/
|
|
||||||
export declare function fetchCatalog(): Promise<Catalog | null>;
|
|
||||||
/**
|
|
||||||
* Display catalog in CLI
|
|
||||||
*/
|
|
||||||
export declare function showCatalog(options: {
|
|
||||||
category?: string;
|
|
||||||
search?: string;
|
|
||||||
detailed?: boolean;
|
|
||||||
}): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Show detailed info about an augmentation
|
|
||||||
*/
|
|
||||||
export declare function showAugmentationInfo(id: string): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Show user's available augmentations
|
|
||||||
*/
|
|
||||||
export declare function showAvailable(licenseKey?: string): Promise<void>;
|
|
||||||
export {};
|
|
||||||
325
dist/cli/catalog.js
vendored
325
dist/cli/catalog.js
vendored
|
|
@ -1,325 +0,0 @@
|
||||||
/**
|
|
||||||
* Brain Cloud Catalog Integration for CLI
|
|
||||||
*
|
|
||||||
* Fetches and displays augmentation catalog
|
|
||||||
* Falls back to local cache if API is unavailable
|
|
||||||
*/
|
|
||||||
import chalk from 'chalk';
|
|
||||||
import { readFileSync, writeFileSync, existsSync } from 'fs';
|
|
||||||
import { join } from 'path';
|
|
||||||
import { homedir } from 'os';
|
|
||||||
const CATALOG_API = process.env.BRAIN_CLOUD_CATALOG_URL || 'https://catalog.brain-cloud.soulcraft.com';
|
|
||||||
const CACHE_PATH = join(homedir(), '.brainy', 'catalog-cache.json');
|
|
||||||
const CACHE_TTL = 24 * 60 * 60 * 1000; // 24 hours
|
|
||||||
/**
|
|
||||||
* Fetch catalog from API with caching
|
|
||||||
*/
|
|
||||||
export async function fetchCatalog() {
|
|
||||||
try {
|
|
||||||
// Check cache first
|
|
||||||
const cached = loadCache();
|
|
||||||
if (cached)
|
|
||||||
return cached;
|
|
||||||
// Fetch from API
|
|
||||||
const response = await fetch(`${CATALOG_API}/api/catalog/cli`);
|
|
||||||
if (!response.ok)
|
|
||||||
throw new Error('API unavailable');
|
|
||||||
const catalog = await response.json();
|
|
||||||
// Save to cache
|
|
||||||
saveCache(catalog);
|
|
||||||
return catalog;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
// Try loading from cache even if expired
|
|
||||||
const cached = loadCache(true);
|
|
||||||
if (cached) {
|
|
||||||
console.log(chalk.yellow('📡 Using cached catalog (API unavailable)'));
|
|
||||||
return cached;
|
|
||||||
}
|
|
||||||
// Fall back to hardcoded catalog
|
|
||||||
return getDefaultCatalog();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Display catalog in CLI
|
|
||||||
*/
|
|
||||||
export async function showCatalog(options) {
|
|
||||||
const catalog = await fetchCatalog();
|
|
||||||
if (!catalog) {
|
|
||||||
console.log(chalk.red('❌ Could not load augmentation catalog'));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
console.log(chalk.cyan.bold('🧠 Brain Cloud Augmentation Catalog'));
|
|
||||||
console.log(chalk.gray(`Version ${catalog.version}`));
|
|
||||||
console.log('');
|
|
||||||
// Filter augmentations
|
|
||||||
let augmentations = catalog.augmentations;
|
|
||||||
if (options.category) {
|
|
||||||
augmentations = augmentations.filter(a => a.category === options.category);
|
|
||||||
}
|
|
||||||
if (options.search) {
|
|
||||||
const query = options.search.toLowerCase();
|
|
||||||
augmentations = augmentations.filter(a => a.name.toLowerCase().includes(query) ||
|
|
||||||
a.description.toLowerCase().includes(query));
|
|
||||||
}
|
|
||||||
// Group by category
|
|
||||||
const grouped = groupByCategory(augmentations, catalog.categories);
|
|
||||||
// Display
|
|
||||||
for (const [category, augs] of Object.entries(grouped)) {
|
|
||||||
if (augs.length === 0)
|
|
||||||
continue;
|
|
||||||
const cat = catalog.categories.find(c => c.id === category);
|
|
||||||
console.log(chalk.bold(`${cat?.icon || '📦'} ${cat?.name || category}`));
|
|
||||||
for (const aug of augs) {
|
|
||||||
const status = getStatusIcon(aug.status);
|
|
||||||
const popular = aug.popular ? chalk.yellow(' ⭐') : '';
|
|
||||||
const eta = aug.eta ? chalk.gray(` (${aug.eta})`) : '';
|
|
||||||
console.log(` ${status} ${aug.name}${popular}${eta}`);
|
|
||||||
if (options.detailed) {
|
|
||||||
console.log(chalk.gray(` ${aug.description}`));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
console.log('');
|
|
||||||
}
|
|
||||||
// Show summary
|
|
||||||
const available = augmentations.filter(a => a.status === 'available').length;
|
|
||||||
const coming = augmentations.filter(a => a.status === 'coming_soon').length;
|
|
||||||
console.log(chalk.gray('─'.repeat(50)));
|
|
||||||
console.log(chalk.green(`✅ ${available} available`) + chalk.gray(` • `) +
|
|
||||||
chalk.yellow(`🔜 ${coming} coming soon`));
|
|
||||||
console.log('');
|
|
||||||
console.log(chalk.dim('Sign up at app.soulcraft.com to activate'));
|
|
||||||
console.log(chalk.dim('Run "brainy augment info <name>" for details'));
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Show detailed info about an augmentation
|
|
||||||
*/
|
|
||||||
export async function showAugmentationInfo(id) {
|
|
||||||
const catalog = await fetchCatalog();
|
|
||||||
if (!catalog) {
|
|
||||||
console.log(chalk.red('❌ Could not load augmentation catalog'));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const aug = catalog.augmentations.find(a => a.id === id);
|
|
||||||
if (!aug) {
|
|
||||||
console.log(chalk.red(`❌ Augmentation not found: ${id}`));
|
|
||||||
console.log('');
|
|
||||||
console.log('Available augmentations:');
|
|
||||||
catalog.augmentations.forEach(a => {
|
|
||||||
console.log(` • ${a.id}`);
|
|
||||||
});
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
// Fetch full details from API
|
|
||||||
try {
|
|
||||||
const response = await fetch(`${CATALOG_API}/api/catalog/augmentation/${id}`);
|
|
||||||
const details = await response.json();
|
|
||||||
console.log(chalk.cyan.bold(`📦 ${details.name}`));
|
|
||||||
if (details.popular)
|
|
||||||
console.log(chalk.yellow('⭐ Popular'));
|
|
||||||
console.log('');
|
|
||||||
console.log(chalk.bold('Category:'), getCategoryName(details.category, catalog.categories));
|
|
||||||
console.log(chalk.bold('Status:'), getStatusText(details.status));
|
|
||||||
if (details.eta)
|
|
||||||
console.log(chalk.bold('Expected:'), details.eta);
|
|
||||||
console.log('');
|
|
||||||
console.log(chalk.bold('Description:'));
|
|
||||||
console.log(details.longDescription || details.description);
|
|
||||||
console.log('');
|
|
||||||
if (details.features) {
|
|
||||||
console.log(chalk.bold('Features:'));
|
|
||||||
details.features.forEach((f) => console.log(` ✓ ${f}`));
|
|
||||||
console.log('');
|
|
||||||
}
|
|
||||||
if (details.example) {
|
|
||||||
console.log(chalk.bold('Example:'));
|
|
||||||
console.log(chalk.gray('─'.repeat(50)));
|
|
||||||
console.log(details.example.code);
|
|
||||||
console.log(chalk.gray('─'.repeat(50)));
|
|
||||||
console.log('');
|
|
||||||
}
|
|
||||||
if (details.requirements?.config) {
|
|
||||||
console.log(chalk.bold('Required Configuration:'));
|
|
||||||
details.requirements.config.forEach((c) => console.log(` • ${c}`));
|
|
||||||
console.log('');
|
|
||||||
}
|
|
||||||
if (details.pricing) {
|
|
||||||
console.log(chalk.bold('Available in:'));
|
|
||||||
details.pricing.tiers.forEach((t) => console.log(` • ${t}`));
|
|
||||||
console.log('');
|
|
||||||
}
|
|
||||||
console.log(chalk.dim('To activate: brainy augment activate'));
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
// Show basic info if API fails
|
|
||||||
console.log(chalk.cyan.bold(`📦 ${aug.name}`));
|
|
||||||
console.log(aug.description);
|
|
||||||
console.log('');
|
|
||||||
console.log(chalk.dim('Full details unavailable (API offline)'));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Show user's available augmentations
|
|
||||||
*/
|
|
||||||
export async function showAvailable(licenseKey) {
|
|
||||||
const key = licenseKey || process.env.BRAINY_LICENSE_KEY || readLicenseFile();
|
|
||||||
if (!key) {
|
|
||||||
console.log(chalk.yellow('⚠️ No license key found'));
|
|
||||||
console.log('');
|
|
||||||
console.log('To see your available augmentations:');
|
|
||||||
console.log(' 1. Sign up at app.soulcraft.com');
|
|
||||||
console.log(' 2. Run: brainy augment activate');
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
const response = await fetch(`${CATALOG_API}/api/catalog/available`, {
|
|
||||||
headers: { 'x-license-key': key }
|
|
||||||
});
|
|
||||||
if (!response.ok) {
|
|
||||||
throw new Error('Invalid license');
|
|
||||||
}
|
|
||||||
const data = await response.json();
|
|
||||||
console.log(chalk.cyan.bold('🧠 Your Available Augmentations'));
|
|
||||||
console.log(chalk.gray(`Plan: ${data.plan}`));
|
|
||||||
console.log('');
|
|
||||||
const grouped = groupByCategory(data.augmentations, []);
|
|
||||||
for (const [category, augs] of Object.entries(grouped)) {
|
|
||||||
console.log(chalk.bold(category));
|
|
||||||
augs.forEach(aug => {
|
|
||||||
console.log(` ✅ ${aug.name}`);
|
|
||||||
});
|
|
||||||
console.log('');
|
|
||||||
}
|
|
||||||
if (data.operations) {
|
|
||||||
const used = data.operations.used || 0;
|
|
||||||
const limit = data.operations.limit;
|
|
||||||
const percent = limit === 'unlimited' ? 0 : Math.round((used / limit) * 100);
|
|
||||||
console.log(chalk.bold('Usage:'));
|
|
||||||
if (limit === 'unlimited') {
|
|
||||||
console.log(` Unlimited operations`);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
console.log(` ${used.toLocaleString()} / ${limit.toLocaleString()} operations (${percent}%)`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
console.log(chalk.red('❌ Could not fetch available augmentations'));
|
|
||||||
console.log(chalk.gray(error.message));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Helper functions
|
|
||||||
function loadCache(ignoreExpiry = false) {
|
|
||||||
try {
|
|
||||||
if (!existsSync(CACHE_PATH))
|
|
||||||
return null;
|
|
||||||
const data = JSON.parse(readFileSync(CACHE_PATH, 'utf8'));
|
|
||||||
if (!ignoreExpiry && Date.now() - data.timestamp > CACHE_TTL) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
return data.catalog;
|
|
||||||
}
|
|
||||||
catch {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
function saveCache(catalog) {
|
|
||||||
try {
|
|
||||||
const dir = join(homedir(), '.brainy');
|
|
||||||
if (!existsSync(dir)) {
|
|
||||||
require('fs').mkdirSync(dir, { recursive: true });
|
|
||||||
}
|
|
||||||
writeFileSync(CACHE_PATH, JSON.stringify({
|
|
||||||
catalog,
|
|
||||||
timestamp: Date.now()
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
catch {
|
|
||||||
// Ignore cache save errors
|
|
||||||
}
|
|
||||||
}
|
|
||||||
function groupByCategory(augmentations, categories) {
|
|
||||||
const grouped = {};
|
|
||||||
for (const aug of augmentations) {
|
|
||||||
if (!grouped[aug.category]) {
|
|
||||||
grouped[aug.category] = [];
|
|
||||||
}
|
|
||||||
grouped[aug.category].push(aug);
|
|
||||||
}
|
|
||||||
// Sort by category order
|
|
||||||
const ordered = {};
|
|
||||||
const categoryOrder = ['memory', 'coordination', 'enterprise', 'perception', 'dialog', 'activation', 'cognition', 'websocket'];
|
|
||||||
for (const cat of categoryOrder) {
|
|
||||||
if (grouped[cat]) {
|
|
||||||
ordered[cat] = grouped[cat];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return ordered;
|
|
||||||
}
|
|
||||||
function getStatusIcon(status) {
|
|
||||||
switch (status) {
|
|
||||||
case 'available': return chalk.green('✅');
|
|
||||||
case 'coming_soon': return chalk.yellow('🔜');
|
|
||||||
case 'deprecated': return chalk.red('⚠️');
|
|
||||||
default: return '❓';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
function getStatusText(status) {
|
|
||||||
switch (status) {
|
|
||||||
case 'available': return chalk.green('Available');
|
|
||||||
case 'coming_soon': return chalk.yellow('Coming Soon');
|
|
||||||
case 'deprecated': return chalk.red('Deprecated');
|
|
||||||
default: return 'Unknown';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
function getCategoryName(categoryId, categories) {
|
|
||||||
const cat = categories.find(c => c.id === categoryId);
|
|
||||||
return cat ? `${cat.icon} ${cat.name}` : categoryId;
|
|
||||||
}
|
|
||||||
function readLicenseFile() {
|
|
||||||
try {
|
|
||||||
const licensePath = join(homedir(), '.brainy', 'license');
|
|
||||||
if (existsSync(licensePath)) {
|
|
||||||
return readFileSync(licensePath, 'utf8').trim();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch { }
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
function getDefaultCatalog() {
|
|
||||||
// Hardcoded fallback catalog
|
|
||||||
return {
|
|
||||||
version: '1.0.0',
|
|
||||||
categories: [
|
|
||||||
{ id: 'memory', name: 'Memory', icon: '🧠', description: 'AI memory and persistence' },
|
|
||||||
{ id: 'coordination', name: 'Coordination', icon: '🤝', description: 'Multi-agent orchestration' },
|
|
||||||
{ id: 'enterprise', name: 'Enterprise', icon: '🏢', description: 'Business integrations' }
|
|
||||||
],
|
|
||||||
augmentations: [
|
|
||||||
{
|
|
||||||
id: 'ai-memory',
|
|
||||||
name: 'AI Memory',
|
|
||||||
category: 'memory',
|
|
||||||
description: 'Persistent memory across all AI sessions',
|
|
||||||
status: 'available',
|
|
||||||
popular: true
|
|
||||||
},
|
|
||||||
{
|
|
||||||
id: 'agent-coordinator',
|
|
||||||
name: 'Agent Coordinator',
|
|
||||||
category: 'coordination',
|
|
||||||
description: 'Multi-agent handoffs and orchestration',
|
|
||||||
status: 'available',
|
|
||||||
popular: true
|
|
||||||
},
|
|
||||||
{
|
|
||||||
id: 'notion-sync',
|
|
||||||
name: 'Notion Sync',
|
|
||||||
category: 'enterprise',
|
|
||||||
description: 'Bidirectional Notion database sync',
|
|
||||||
status: 'available'
|
|
||||||
}
|
|
||||||
]
|
|
||||||
};
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=catalog.js.map
|
|
||||||
1
dist/cli/catalog.js.map
vendored
1
dist/cli/catalog.js.map
vendored
File diff suppressed because one or more lines are too long
143
dist/connectors/interfaces/IConnector.d.ts
vendored
143
dist/connectors/interfaces/IConnector.d.ts
vendored
|
|
@ -1,143 +0,0 @@
|
||||||
/**
|
|
||||||
* Brainy Connector Interface - Atomic Age Integration Framework
|
|
||||||
*
|
|
||||||
* 🧠 Base interface for all premium connectors in Brain Cloud
|
|
||||||
* ⚛️ Open source interface, implementations are premium-only
|
|
||||||
*/
|
|
||||||
export interface ConnectorConfig {
|
|
||||||
/** Connector identifier (e.g., 'notion', 'salesforce') */
|
|
||||||
connectorId: string;
|
|
||||||
/** Premium license key (required for Brain Cloud connectors) */
|
|
||||||
licenseKey: string;
|
|
||||||
/** API credentials for the external service */
|
|
||||||
credentials: {
|
|
||||||
apiKey?: string;
|
|
||||||
accessToken?: string;
|
|
||||||
refreshToken?: string;
|
|
||||||
clientId?: string;
|
|
||||||
clientSecret?: string;
|
|
||||||
[key: string]: any;
|
|
||||||
};
|
|
||||||
/** Connector-specific configuration */
|
|
||||||
options?: {
|
|
||||||
syncInterval?: number;
|
|
||||||
batchSize?: number;
|
|
||||||
retryAttempts?: number;
|
|
||||||
[key: string]: any;
|
|
||||||
};
|
|
||||||
/** Brainy database instance configuration */
|
|
||||||
brainy?: {
|
|
||||||
endpoint?: string;
|
|
||||||
storage?: string;
|
|
||||||
[key: string]: any;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
export interface SyncResult {
|
|
||||||
/** Number of items successfully synced */
|
|
||||||
synced: number;
|
|
||||||
/** Number of items that failed to sync */
|
|
||||||
failed: number;
|
|
||||||
/** Number of items skipped (duplicates, etc.) */
|
|
||||||
skipped: number;
|
|
||||||
/** Total processing time in milliseconds */
|
|
||||||
duration: number;
|
|
||||||
/** Sync operation timestamp */
|
|
||||||
timestamp: string;
|
|
||||||
/** Error details for failed items */
|
|
||||||
errors?: Array<{
|
|
||||||
item: string;
|
|
||||||
error: string;
|
|
||||||
retryable: boolean;
|
|
||||||
}>;
|
|
||||||
/** Metadata about the sync operation */
|
|
||||||
metadata?: {
|
|
||||||
lastSyncId?: string;
|
|
||||||
nextPageToken?: string;
|
|
||||||
hasMore?: boolean;
|
|
||||||
[key: string]: any;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
export interface ConnectorStatus {
|
|
||||||
/** Current connector state */
|
|
||||||
status: 'connected' | 'disconnected' | 'error' | 'syncing' | 'paused';
|
|
||||||
/** Human-readable status message */
|
|
||||||
message: string;
|
|
||||||
/** Last successful sync timestamp */
|
|
||||||
lastSync?: string;
|
|
||||||
/** Next scheduled sync timestamp */
|
|
||||||
nextSync?: string;
|
|
||||||
/** Connection health indicators */
|
|
||||||
health: {
|
|
||||||
apiReachable: boolean;
|
|
||||||
credentialsValid: boolean;
|
|
||||||
licenseValid: boolean;
|
|
||||||
quotaRemaining?: number;
|
|
||||||
};
|
|
||||||
/** Usage statistics */
|
|
||||||
stats?: {
|
|
||||||
totalSyncs: number;
|
|
||||||
totalItems: number;
|
|
||||||
averageDuration: number;
|
|
||||||
errorRate: number;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Base interface for all Brainy premium connectors
|
|
||||||
*
|
|
||||||
* Implementations auto-load with Brain Cloud subscription after auth
|
|
||||||
*/
|
|
||||||
export interface IConnector {
|
|
||||||
/** Unique connector identifier */
|
|
||||||
readonly id: string;
|
|
||||||
/** Human-readable connector name */
|
|
||||||
readonly name: string;
|
|
||||||
/** Connector version */
|
|
||||||
readonly version: string;
|
|
||||||
/** Supported data types this connector can handle */
|
|
||||||
readonly supportedTypes: string[];
|
|
||||||
/**
|
|
||||||
* Initialize the connector with configuration
|
|
||||||
*/
|
|
||||||
initialize(config: ConnectorConfig): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Test connection to the external service
|
|
||||||
*/
|
|
||||||
testConnection(): Promise<boolean>;
|
|
||||||
/**
|
|
||||||
* Get current connector status and health
|
|
||||||
*/
|
|
||||||
getStatus(): Promise<ConnectorStatus>;
|
|
||||||
/**
|
|
||||||
* Start syncing data from the external service
|
|
||||||
*/
|
|
||||||
startSync(): Promise<SyncResult>;
|
|
||||||
/**
|
|
||||||
* Stop any ongoing sync operations
|
|
||||||
*/
|
|
||||||
stopSync(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Perform incremental sync (delta changes only)
|
|
||||||
*/
|
|
||||||
incrementalSync(): Promise<SyncResult>;
|
|
||||||
/**
|
|
||||||
* Perform full sync (all data)
|
|
||||||
*/
|
|
||||||
fullSync(): Promise<SyncResult>;
|
|
||||||
/**
|
|
||||||
* Preview what would be synced without actually syncing
|
|
||||||
*/
|
|
||||||
previewSync(limit?: number): Promise<{
|
|
||||||
items: Array<{
|
|
||||||
type: string;
|
|
||||||
title: string;
|
|
||||||
preview: string;
|
|
||||||
relationships: string[];
|
|
||||||
}>;
|
|
||||||
totalCount: number;
|
|
||||||
estimatedDuration: number;
|
|
||||||
}>;
|
|
||||||
/**
|
|
||||||
* Clean up resources and disconnect
|
|
||||||
*/
|
|
||||||
disconnect(): Promise<void>;
|
|
||||||
}
|
|
||||||
8
dist/connectors/interfaces/IConnector.js
vendored
8
dist/connectors/interfaces/IConnector.js
vendored
|
|
@ -1,8 +0,0 @@
|
||||||
/**
|
|
||||||
* Brainy Connector Interface - Atomic Age Integration Framework
|
|
||||||
*
|
|
||||||
* 🧠 Base interface for all premium connectors in Brain Cloud
|
|
||||||
* ⚛️ Open source interface, implementations are premium-only
|
|
||||||
*/
|
|
||||||
export {};
|
|
||||||
//# sourceMappingURL=IConnector.js.map
|
|
||||||
1
dist/connectors/interfaces/IConnector.js.map
vendored
1
dist/connectors/interfaces/IConnector.js.map
vendored
|
|
@ -1 +0,0 @@
|
||||||
{"version":3,"file":"IConnector.js","sourceRoot":"","sources":["../../../src/connectors/interfaces/IConnector.ts"],"names":[],"mappings":"AAAA;;;;;GAKG"}
|
|
||||||
515
dist/coreTypes.d.ts
vendored
515
dist/coreTypes.d.ts
vendored
|
|
@ -1,515 +0,0 @@
|
||||||
/**
|
|
||||||
* Type definitions for the Soulcraft Brainy
|
|
||||||
*/
|
|
||||||
/**
|
|
||||||
* Vector representation - an array of numbers
|
|
||||||
*/
|
|
||||||
export type Vector = number[];
|
|
||||||
/**
|
|
||||||
* A document with a vector embedding and optional metadata
|
|
||||||
*/
|
|
||||||
export interface VectorDocument<T = any> {
|
|
||||||
id: string;
|
|
||||||
vector: Vector;
|
|
||||||
metadata?: T;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Search result with similarity score
|
|
||||||
*/
|
|
||||||
export interface SearchResult<T = any> {
|
|
||||||
id: string;
|
|
||||||
score: number;
|
|
||||||
vector: Vector;
|
|
||||||
metadata?: T;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Cursor for pagination through search results
|
|
||||||
*/
|
|
||||||
export interface SearchCursor {
|
|
||||||
lastId: string;
|
|
||||||
lastScore: number;
|
|
||||||
position: number;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Paginated search result with cursor support
|
|
||||||
*/
|
|
||||||
export interface PaginatedSearchResult<T = any> {
|
|
||||||
results: SearchResult<T>[];
|
|
||||||
cursor?: SearchCursor;
|
|
||||||
hasMore: boolean;
|
|
||||||
totalEstimate?: number;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Distance function for comparing vectors
|
|
||||||
*/
|
|
||||||
export type DistanceFunction = (a: Vector, b: Vector) => number;
|
|
||||||
/**
|
|
||||||
* Embedding function for converting data to vectors
|
|
||||||
*/
|
|
||||||
export type EmbeddingFunction = (data: any) => Promise<Vector>;
|
|
||||||
/**
|
|
||||||
* Embedding model interface
|
|
||||||
*/
|
|
||||||
export interface EmbeddingModel {
|
|
||||||
/**
|
|
||||||
* Initialize the embedding model
|
|
||||||
*/
|
|
||||||
init(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Embed data into a vector
|
|
||||||
*/
|
|
||||||
embed(data: any): Promise<Vector>;
|
|
||||||
/**
|
|
||||||
* Dispose of the model resources
|
|
||||||
*/
|
|
||||||
dispose(): Promise<void>;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* HNSW graph noun
|
|
||||||
*/
|
|
||||||
export interface HNSWNoun {
|
|
||||||
id: string;
|
|
||||||
vector: Vector;
|
|
||||||
connections: Map<number, Set<string>>;
|
|
||||||
level: number;
|
|
||||||
metadata?: any;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Lightweight verb for HNSW index storage
|
|
||||||
* Contains only essential data needed for vector operations
|
|
||||||
*/
|
|
||||||
export interface HNSWVerb {
|
|
||||||
id: string;
|
|
||||||
vector: Vector;
|
|
||||||
connections: Map<number, Set<string>>;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Verb representing a relationship between nouns
|
|
||||||
* Stored separately from HNSW index for lightweight performance
|
|
||||||
*/
|
|
||||||
export interface GraphVerb {
|
|
||||||
id: string;
|
|
||||||
sourceId: string;
|
|
||||||
targetId: string;
|
|
||||||
vector: Vector;
|
|
||||||
connections?: Map<number, Set<string>>;
|
|
||||||
type?: string;
|
|
||||||
weight?: number;
|
|
||||||
metadata?: any;
|
|
||||||
source?: string;
|
|
||||||
target?: string;
|
|
||||||
verb?: string;
|
|
||||||
data?: Record<string, any>;
|
|
||||||
embedding?: Vector;
|
|
||||||
createdAt?: {
|
|
||||||
seconds: number;
|
|
||||||
nanoseconds: number;
|
|
||||||
};
|
|
||||||
updatedAt?: {
|
|
||||||
seconds: number;
|
|
||||||
nanoseconds: number;
|
|
||||||
};
|
|
||||||
createdBy?: {
|
|
||||||
augmentation: string;
|
|
||||||
version: string;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* HNSW index configuration
|
|
||||||
*/
|
|
||||||
export interface HNSWConfig {
|
|
||||||
M: number;
|
|
||||||
efConstruction: number;
|
|
||||||
efSearch: number;
|
|
||||||
ml: number;
|
|
||||||
useDiskBasedIndex?: boolean;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Storage interface for persistence
|
|
||||||
*/
|
|
||||||
/**
|
|
||||||
* Statistics data structure for tracking counts by service
|
|
||||||
*/
|
|
||||||
/**
|
|
||||||
* Per-service statistics tracking
|
|
||||||
*/
|
|
||||||
export interface ServiceStatistics {
|
|
||||||
/**
|
|
||||||
* Service name
|
|
||||||
*/
|
|
||||||
name: string;
|
|
||||||
/**
|
|
||||||
* Total number of nouns created by this service
|
|
||||||
*/
|
|
||||||
totalNouns: number;
|
|
||||||
/**
|
|
||||||
* Total number of verbs created by this service
|
|
||||||
*/
|
|
||||||
totalVerbs: number;
|
|
||||||
/**
|
|
||||||
* Total number of metadata entries created by this service
|
|
||||||
*/
|
|
||||||
totalMetadata: number;
|
|
||||||
/**
|
|
||||||
* First activity timestamp for this service
|
|
||||||
*/
|
|
||||||
firstActivity?: string;
|
|
||||||
/**
|
|
||||||
* Last activity timestamp for this service
|
|
||||||
*/
|
|
||||||
lastActivity?: string;
|
|
||||||
/**
|
|
||||||
* Error count for this service
|
|
||||||
*/
|
|
||||||
errorCount?: number;
|
|
||||||
/**
|
|
||||||
* Operation breakdown for this service
|
|
||||||
*/
|
|
||||||
operations?: {
|
|
||||||
adds: number;
|
|
||||||
updates: number;
|
|
||||||
deletes: number;
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Status of the service (active, inactive, read-only)
|
|
||||||
*/
|
|
||||||
status?: 'active' | 'inactive' | 'read-only';
|
|
||||||
}
|
|
||||||
export interface StatisticsData {
|
|
||||||
/**
|
|
||||||
* Count of nouns by service
|
|
||||||
*/
|
|
||||||
nounCount: Record<string, number>;
|
|
||||||
/**
|
|
||||||
* Count of verbs by service
|
|
||||||
*/
|
|
||||||
verbCount: Record<string, number>;
|
|
||||||
/**
|
|
||||||
* Count of metadata entries by service
|
|
||||||
*/
|
|
||||||
metadataCount: Record<string, number>;
|
|
||||||
/**
|
|
||||||
* Size of the HNSW index
|
|
||||||
*/
|
|
||||||
hnswIndexSize: number;
|
|
||||||
/**
|
|
||||||
* Total number of nodes
|
|
||||||
*/
|
|
||||||
totalNodes?: number;
|
|
||||||
/**
|
|
||||||
* Total number of edges
|
|
||||||
*/
|
|
||||||
totalEdges?: number;
|
|
||||||
/**
|
|
||||||
* Total metadata count
|
|
||||||
*/
|
|
||||||
totalMetadata?: number;
|
|
||||||
/**
|
|
||||||
* Operation counts
|
|
||||||
*/
|
|
||||||
operations?: {
|
|
||||||
add: number;
|
|
||||||
search: number;
|
|
||||||
delete: number;
|
|
||||||
update: number;
|
|
||||||
relate: number;
|
|
||||||
total: number;
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Field names available for searching, organized by service
|
|
||||||
* This helps users understand what fields are available from different data sources
|
|
||||||
*/
|
|
||||||
fieldNames?: Record<string, string[]>;
|
|
||||||
/**
|
|
||||||
* Standard field mappings for common field names across services
|
|
||||||
* Maps standard field names to the actual field names used by each service
|
|
||||||
*/
|
|
||||||
standardFieldMappings?: Record<string, Record<string, string[]>>;
|
|
||||||
/**
|
|
||||||
* Content type breakdown (e.g., Person, Repository, Issue, etc.)
|
|
||||||
*/
|
|
||||||
contentTypes?: Record<string, number>;
|
|
||||||
/**
|
|
||||||
* Data freshness metrics
|
|
||||||
*/
|
|
||||||
dataFreshness?: {
|
|
||||||
oldestEntry: string;
|
|
||||||
newestEntry: string;
|
|
||||||
updatesLastHour: number;
|
|
||||||
updatesLastDay: number;
|
|
||||||
ageDistribution: {
|
|
||||||
last24h: number;
|
|
||||||
last7d: number;
|
|
||||||
last30d: number;
|
|
||||||
older: number;
|
|
||||||
};
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Storage utilization metrics
|
|
||||||
*/
|
|
||||||
storageMetrics?: {
|
|
||||||
totalSizeBytes: number;
|
|
||||||
nounsSizeBytes: number;
|
|
||||||
verbsSizeBytes: number;
|
|
||||||
metadataSizeBytes: number;
|
|
||||||
indexSizeBytes: number;
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Search performance metrics
|
|
||||||
*/
|
|
||||||
searchMetrics?: {
|
|
||||||
totalSearches: number;
|
|
||||||
averageSearchTimeMs: number;
|
|
||||||
searchesLastHour: number;
|
|
||||||
searchesLastDay: number;
|
|
||||||
topSearchTerms?: string[];
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Verb statistics similar to nouns
|
|
||||||
*/
|
|
||||||
verbStatistics?: {
|
|
||||||
totalVerbs: number;
|
|
||||||
verbTypes: Record<string, number>;
|
|
||||||
averageConnectionsPerVerb: number;
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Service-level activity timestamps
|
|
||||||
*/
|
|
||||||
serviceActivity?: Record<string, {
|
|
||||||
firstActivity: string;
|
|
||||||
lastActivity: string;
|
|
||||||
totalOperations: number;
|
|
||||||
}>;
|
|
||||||
/**
|
|
||||||
* List of all services that have written data
|
|
||||||
*/
|
|
||||||
services?: ServiceStatistics[];
|
|
||||||
/**
|
|
||||||
* Throttling metrics for storage operations
|
|
||||||
*/
|
|
||||||
throttlingMetrics?: {
|
|
||||||
/**
|
|
||||||
* Storage-level throttling information
|
|
||||||
*/
|
|
||||||
storage?: {
|
|
||||||
currentlyThrottled: boolean;
|
|
||||||
lastThrottleTime?: string;
|
|
||||||
consecutiveThrottleEvents: number;
|
|
||||||
currentBackoffMs: number;
|
|
||||||
totalThrottleEvents: number;
|
|
||||||
throttleEventsByHour?: number[];
|
|
||||||
throttleReasons?: Record<string, number>;
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Operation impact metrics
|
|
||||||
*/
|
|
||||||
operationImpact?: {
|
|
||||||
delayedOperations: number;
|
|
||||||
retriedOperations: number;
|
|
||||||
failedDueToThrottling: number;
|
|
||||||
averageDelayMs: number;
|
|
||||||
totalDelayMs: number;
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Service-level throttling breakdown
|
|
||||||
*/
|
|
||||||
serviceThrottling?: Record<string, {
|
|
||||||
throttleCount: number;
|
|
||||||
lastThrottle: string;
|
|
||||||
status: 'normal' | 'throttled' | 'recovering';
|
|
||||||
}>;
|
|
||||||
};
|
|
||||||
/**
|
|
||||||
* Last updated timestamp
|
|
||||||
*/
|
|
||||||
lastUpdated: string;
|
|
||||||
/**
|
|
||||||
* Distributed configuration (stored in index folder for easy access)
|
|
||||||
* This is used for distributed Brainy instances coordination
|
|
||||||
*/
|
|
||||||
distributedConfig?: import('./types/distributedTypes.js').SharedConfig;
|
|
||||||
}
|
|
||||||
export interface StorageAdapter {
|
|
||||||
init(): Promise<void>;
|
|
||||||
saveNoun(noun: HNSWNoun): Promise<void>;
|
|
||||||
getNoun(id: string): Promise<HNSWNoun | null>;
|
|
||||||
/**
|
|
||||||
* Get nouns with pagination and filtering
|
|
||||||
* @param options Pagination and filtering options
|
|
||||||
* @returns Promise that resolves to a paginated result of nouns
|
|
||||||
*/
|
|
||||||
getNouns(options?: {
|
|
||||||
pagination?: {
|
|
||||||
offset?: number;
|
|
||||||
limit?: number;
|
|
||||||
cursor?: string;
|
|
||||||
};
|
|
||||||
filter?: {
|
|
||||||
nounType?: string | string[];
|
|
||||||
service?: string | string[];
|
|
||||||
metadata?: Record<string, any>;
|
|
||||||
};
|
|
||||||
}): Promise<{
|
|
||||||
items: HNSWNoun[];
|
|
||||||
totalCount?: number;
|
|
||||||
hasMore: boolean;
|
|
||||||
nextCursor?: string;
|
|
||||||
}>;
|
|
||||||
/**
|
|
||||||
* Get nouns by noun type
|
|
||||||
* @param nounType The noun type to filter by
|
|
||||||
* @returns Promise that resolves to an array of nouns of the specified noun type
|
|
||||||
* @deprecated Use getNouns() with filter.nounType instead
|
|
||||||
*/
|
|
||||||
getNounsByNounType(nounType: string): Promise<HNSWNoun[]>;
|
|
||||||
deleteNoun(id: string): Promise<void>;
|
|
||||||
saveVerb(verb: GraphVerb): Promise<void>;
|
|
||||||
getVerb(id: string): Promise<GraphVerb | null>;
|
|
||||||
/**
|
|
||||||
* Get verbs with pagination and filtering
|
|
||||||
* @param options Pagination and filtering options
|
|
||||||
* @returns Promise that resolves to a paginated result of verbs
|
|
||||||
*/
|
|
||||||
getVerbs(options?: {
|
|
||||||
pagination?: {
|
|
||||||
offset?: number;
|
|
||||||
limit?: number;
|
|
||||||
cursor?: string;
|
|
||||||
};
|
|
||||||
filter?: {
|
|
||||||
verbType?: string | string[];
|
|
||||||
sourceId?: string | string[];
|
|
||||||
targetId?: string | string[];
|
|
||||||
service?: string | string[];
|
|
||||||
metadata?: Record<string, any>;
|
|
||||||
};
|
|
||||||
}): Promise<{
|
|
||||||
items: GraphVerb[];
|
|
||||||
totalCount?: number;
|
|
||||||
hasMore: boolean;
|
|
||||||
nextCursor?: string;
|
|
||||||
}>;
|
|
||||||
/**
|
|
||||||
* Get verbs by source
|
|
||||||
* @param sourceId The source ID to filter by
|
|
||||||
* @returns Promise that resolves to an array of verbs with the specified source ID
|
|
||||||
* @deprecated Use getVerbs() with filter.sourceId instead
|
|
||||||
*/
|
|
||||||
getVerbsBySource(sourceId: string): Promise<GraphVerb[]>;
|
|
||||||
/**
|
|
||||||
* Get verbs by target
|
|
||||||
* @param targetId The target ID to filter by
|
|
||||||
* @returns Promise that resolves to an array of verbs with the specified target ID
|
|
||||||
* @deprecated Use getVerbs() with filter.targetId instead
|
|
||||||
*/
|
|
||||||
getVerbsByTarget(targetId: string): Promise<GraphVerb[]>;
|
|
||||||
/**
|
|
||||||
* Get verbs by type
|
|
||||||
* @param type The verb type to filter by
|
|
||||||
* @returns Promise that resolves to an array of verbs with the specified type
|
|
||||||
* @deprecated Use getVerbs() with filter.verbType instead
|
|
||||||
*/
|
|
||||||
getVerbsByType(type: string): Promise<GraphVerb[]>;
|
|
||||||
deleteVerb(id: string): Promise<void>;
|
|
||||||
saveMetadata(id: string, metadata: any): Promise<void>;
|
|
||||||
getMetadata(id: string): Promise<any | null>;
|
|
||||||
/**
|
|
||||||
* Get multiple metadata objects in batches (prevents socket exhaustion)
|
|
||||||
* @param ids Array of IDs to get metadata for
|
|
||||||
* @returns Promise that resolves to a Map of id -> metadata
|
|
||||||
*/
|
|
||||||
getMetadataBatch?(ids: string[]): Promise<Map<string, any>>;
|
|
||||||
/**
|
|
||||||
* Save verb metadata to storage
|
|
||||||
* @param id The ID of the verb
|
|
||||||
* @param metadata The metadata to save
|
|
||||||
* @returns Promise that resolves when the metadata is saved
|
|
||||||
*/
|
|
||||||
saveVerbMetadata(id: string, metadata: any): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Get verb metadata from storage
|
|
||||||
* @param id The ID of the verb
|
|
||||||
* @returns Promise that resolves to the metadata or null if not found
|
|
||||||
*/
|
|
||||||
getVerbMetadata(id: string): Promise<any | null>;
|
|
||||||
clear(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Get information about storage usage and capacity
|
|
||||||
* @returns Promise that resolves to an object containing storage status information
|
|
||||||
*/
|
|
||||||
getStorageStatus(): Promise<{
|
|
||||||
/**
|
|
||||||
* The type of storage being used (e.g., 'filesystem', 'opfs', 'memory')
|
|
||||||
*/
|
|
||||||
type: string;
|
|
||||||
/**
|
|
||||||
* The amount of storage being used in bytes
|
|
||||||
*/
|
|
||||||
used: number;
|
|
||||||
/**
|
|
||||||
* The total amount of storage available in bytes, or null if unknown
|
|
||||||
*/
|
|
||||||
quota: number | null;
|
|
||||||
/**
|
|
||||||
* Additional storage-specific information
|
|
||||||
*/
|
|
||||||
details?: Record<string, any>;
|
|
||||||
}>;
|
|
||||||
/**
|
|
||||||
* Save statistics data
|
|
||||||
* @param statistics The statistics data to save
|
|
||||||
*/
|
|
||||||
saveStatistics(statistics: StatisticsData): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Get statistics data
|
|
||||||
* @returns Promise that resolves to the statistics data
|
|
||||||
*/
|
|
||||||
getStatistics(): Promise<StatisticsData | null>;
|
|
||||||
/**
|
|
||||||
* Increment a statistic counter
|
|
||||||
* @param type The type of statistic to increment ('noun', 'verb', 'metadata')
|
|
||||||
* @param service The service that inserted the data
|
|
||||||
* @param amount The amount to increment by (default: 1)
|
|
||||||
*/
|
|
||||||
incrementStatistic(type: 'noun' | 'verb' | 'metadata', service: string, amount?: number): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Decrement a statistic counter
|
|
||||||
* @param type The type of statistic to decrement ('noun', 'verb', 'metadata')
|
|
||||||
* @param service The service that inserted the data
|
|
||||||
* @param amount The amount to decrement by (default: 1)
|
|
||||||
*/
|
|
||||||
decrementStatistic(type: 'noun' | 'verb' | 'metadata', service: string, amount?: number): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Update the HNSW index size statistic
|
|
||||||
* @param size The new size of the HNSW index
|
|
||||||
*/
|
|
||||||
updateHnswIndexSize(size: number): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Force an immediate flush of statistics to storage
|
|
||||||
* This ensures that any pending statistics updates are written to persistent storage
|
|
||||||
*/
|
|
||||||
flushStatisticsToStorage(): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Track field names from a JSON document
|
|
||||||
* @param jsonDocument The JSON document to extract field names from
|
|
||||||
* @param service The service that inserted the data
|
|
||||||
*/
|
|
||||||
trackFieldNames(jsonDocument: any, service: string): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Get available field names by service
|
|
||||||
* @returns Record of field names by service
|
|
||||||
*/
|
|
||||||
getAvailableFieldNames(): Promise<Record<string, string[]>>;
|
|
||||||
/**
|
|
||||||
* Get standard field mappings
|
|
||||||
* @returns Record of standard field mappings
|
|
||||||
*/
|
|
||||||
getStandardFieldMappings(): Promise<Record<string, Record<string, string[]>>>;
|
|
||||||
/**
|
|
||||||
* Get changes since a specific timestamp
|
|
||||||
* @param timestamp The timestamp to get changes since
|
|
||||||
* @param limit Optional limit on the number of changes to return
|
|
||||||
* @returns Promise that resolves to an array of changes
|
|
||||||
*/
|
|
||||||
getChangesSince?(timestamp: number, limit?: number): Promise<any[]>;
|
|
||||||
}
|
|
||||||
5
dist/coreTypes.js
vendored
5
dist/coreTypes.js
vendored
|
|
@ -1,5 +0,0 @@
|
||||||
/**
|
|
||||||
* Type definitions for the Soulcraft Brainy
|
|
||||||
*/
|
|
||||||
export {};
|
|
||||||
//# sourceMappingURL=coreTypes.js.map
|
|
||||||
1
dist/coreTypes.js.map
vendored
1
dist/coreTypes.js.map
vendored
|
|
@ -1 +0,0 @@
|
||||||
{"version":3,"file":"coreTypes.js","sourceRoot":"","sources":["../src/coreTypes.ts"],"names":[],"mappings":"AAAA;;GAEG"}
|
|
||||||
11
dist/cortex.d.ts
vendored
11
dist/cortex.d.ts
vendored
|
|
@ -1,11 +0,0 @@
|
||||||
/**
|
|
||||||
* Cortex - The Brain's Central Orchestration System
|
|
||||||
*
|
|
||||||
* 🧠⚛️ The cerebral cortex that coordinates all augmentations
|
|
||||||
*
|
|
||||||
* This is the main export for the Cortex system. It provides the central
|
|
||||||
* coordination for all augmentations, managing their registration, execution,
|
|
||||||
* and pipeline orchestration.
|
|
||||||
*/
|
|
||||||
export { Cortex, cortex, ExecutionMode, PipelineOptions, AugmentationPipeline, augmentationPipeline } from './augmentationPipeline.js';
|
|
||||||
export type { BrainyAugmentations, IAugmentation, ISenseAugmentation, IConduitAugmentation, ICognitionAugmentation, IMemoryAugmentation, IPerceptionAugmentation, IDialogAugmentation, IActivationAugmentation, IWebSocketSupport, AugmentationResponse, AugmentationType } from './types/augmentations.js';
|
|
||||||
14
dist/cortex.js
vendored
14
dist/cortex.js
vendored
|
|
@ -1,14 +0,0 @@
|
||||||
/**
|
|
||||||
* Cortex - The Brain's Central Orchestration System
|
|
||||||
*
|
|
||||||
* 🧠⚛️ The cerebral cortex that coordinates all augmentations
|
|
||||||
*
|
|
||||||
* This is the main export for the Cortex system. It provides the central
|
|
||||||
* coordination for all augmentations, managing their registration, execution,
|
|
||||||
* and pipeline orchestration.
|
|
||||||
*/
|
|
||||||
// Re-export from augmentationPipeline (which contains the Cortex class)
|
|
||||||
export { Cortex, cortex, ExecutionMode,
|
|
||||||
// Backward compatibility
|
|
||||||
AugmentationPipeline, augmentationPipeline } from './augmentationPipeline.js';
|
|
||||||
//# sourceMappingURL=cortex.js.map
|
|
||||||
1
dist/cortex.js.map
vendored
1
dist/cortex.js.map
vendored
|
|
@ -1 +0,0 @@
|
||||||
{"version":3,"file":"cortex.js","sourceRoot":"","sources":["../src/cortex.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,wEAAwE;AACxE,OAAO,EACL,MAAM,EACN,MAAM,EACN,aAAa;AAEb,yBAAyB;AACzB,oBAAoB,EACpB,oBAAoB,EACrB,MAAM,2BAA2B,CAAA"}
|
|
||||||
85
dist/cortex/backupRestore.d.ts
vendored
85
dist/cortex/backupRestore.d.ts
vendored
|
|
@ -1,85 +0,0 @@
|
||||||
/**
|
|
||||||
* Backup & Restore System - Atomic Age Data Preservation Protocol
|
|
||||||
*
|
|
||||||
* 🧠 Complete backup/restore with compression and verification
|
|
||||||
* ⚛️ 1950s retro sci-fi aesthetic maintained throughout
|
|
||||||
*/
|
|
||||||
import { BrainyData } from '../brainyData.js';
|
|
||||||
export interface BackupOptions {
|
|
||||||
compress?: boolean;
|
|
||||||
output?: string;
|
|
||||||
includeMetadata?: boolean;
|
|
||||||
includeStatistics?: boolean;
|
|
||||||
verify?: boolean;
|
|
||||||
password?: string;
|
|
||||||
}
|
|
||||||
export interface RestoreOptions {
|
|
||||||
verify?: boolean;
|
|
||||||
overwrite?: boolean;
|
|
||||||
password?: string;
|
|
||||||
dryRun?: boolean;
|
|
||||||
}
|
|
||||||
export interface BackupManifest {
|
|
||||||
version: string;
|
|
||||||
timestamp: string;
|
|
||||||
brainyVersion: string;
|
|
||||||
entityCount: number;
|
|
||||||
relationshipCount: number;
|
|
||||||
storageType: string;
|
|
||||||
compressed: boolean;
|
|
||||||
encrypted: boolean;
|
|
||||||
checksum: string;
|
|
||||||
metadata: {
|
|
||||||
created: string;
|
|
||||||
description?: string;
|
|
||||||
tags?: string[];
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Backup & Restore Engine - The Brain's Memory Preservation System
|
|
||||||
*/
|
|
||||||
export declare class BackupRestore {
|
|
||||||
private brainy;
|
|
||||||
private colors;
|
|
||||||
private emojis;
|
|
||||||
constructor(brainy: BrainyData);
|
|
||||||
/**
|
|
||||||
* Create a complete backup of Brainy data
|
|
||||||
*/
|
|
||||||
createBackup(options?: BackupOptions): Promise<string>;
|
|
||||||
/**
|
|
||||||
* Restore Brainy data from backup
|
|
||||||
*/
|
|
||||||
restoreBackup(backupPath: string, options?: RestoreOptions): Promise<void>;
|
|
||||||
/**
|
|
||||||
* List available backups in a directory
|
|
||||||
*/
|
|
||||||
listBackups(directory?: string): Promise<BackupManifest[]>;
|
|
||||||
/**
|
|
||||||
* Get backup manifest without loading full backup
|
|
||||||
*/
|
|
||||||
private getBackupManifest;
|
|
||||||
/**
|
|
||||||
* Collect all data for backup
|
|
||||||
*/
|
|
||||||
private collectBackupData;
|
|
||||||
/**
|
|
||||||
* Create backup manifest
|
|
||||||
*/
|
|
||||||
private createManifest;
|
|
||||||
/**
|
|
||||||
* Helper methods
|
|
||||||
*/
|
|
||||||
private generateBackupPath;
|
|
||||||
private compressData;
|
|
||||||
private decompressData;
|
|
||||||
private encryptData;
|
|
||||||
private decryptData;
|
|
||||||
private verifyBackup;
|
|
||||||
private verifyRestoreData;
|
|
||||||
private executeRestore;
|
|
||||||
private collectMetadata;
|
|
||||||
private restoreMetadata;
|
|
||||||
private calculateChecksum;
|
|
||||||
private formatFileSize;
|
|
||||||
}
|
|
||||||
326
dist/cortex/backupRestore.js
vendored
326
dist/cortex/backupRestore.js
vendored
|
|
@ -1,326 +0,0 @@
|
||||||
/**
|
|
||||||
* Backup & Restore System - Atomic Age Data Preservation Protocol
|
|
||||||
*
|
|
||||||
* 🧠 Complete backup/restore with compression and verification
|
|
||||||
* ⚛️ 1950s retro sci-fi aesthetic maintained throughout
|
|
||||||
*/
|
|
||||||
import * as fs from '../universal/fs.js';
|
|
||||||
import * as path from '../universal/path.js';
|
|
||||||
// @ts-ignore
|
|
||||||
import chalk from 'chalk';
|
|
||||||
// @ts-ignore
|
|
||||||
import ora from 'ora';
|
|
||||||
// @ts-ignore
|
|
||||||
import boxen from 'boxen';
|
|
||||||
// @ts-ignore
|
|
||||||
import prompts from 'prompts';
|
|
||||||
/**
|
|
||||||
* Backup & Restore Engine - The Brain's Memory Preservation System
|
|
||||||
*/
|
|
||||||
export class BackupRestore {
|
|
||||||
constructor(brainy) {
|
|
||||||
this.colors = {
|
|
||||||
primary: chalk.hex('#3A5F4A'),
|
|
||||||
success: chalk.hex('#2D4A3A'),
|
|
||||||
warning: chalk.hex('#D67441'),
|
|
||||||
error: chalk.hex('#B85C35'),
|
|
||||||
info: chalk.hex('#4A6B5A'),
|
|
||||||
dim: chalk.hex('#8A9B8A'),
|
|
||||||
highlight: chalk.hex('#E88B5A'),
|
|
||||||
accent: chalk.hex('#F5E6D3'),
|
|
||||||
brain: chalk.hex('#E88B5A')
|
|
||||||
};
|
|
||||||
this.emojis = {
|
|
||||||
brain: '🧠',
|
|
||||||
atom: '⚛️',
|
|
||||||
disk: '💾',
|
|
||||||
archive: '📦',
|
|
||||||
shield: '🛡️',
|
|
||||||
check: '✅',
|
|
||||||
warning: '⚠️',
|
|
||||||
sparkle: '✨',
|
|
||||||
rocket: '🚀',
|
|
||||||
gear: '⚙️',
|
|
||||||
time: '⏰'
|
|
||||||
};
|
|
||||||
this.brainy = brainy;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Create a complete backup of Brainy data
|
|
||||||
*/
|
|
||||||
async createBackup(options = {}) {
|
|
||||||
const outputPath = options.output || this.generateBackupPath();
|
|
||||||
console.log(boxen(`${this.emojis.archive} ${this.colors.brain('ATOMIC DATA PRESERVATION PROTOCOL')} ${this.emojis.atom}\n\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Initiating brain backup sequence')}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Output:')} ${this.colors.highlight(outputPath)}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Compression:')} ${this.colors.highlight(options.compress ? 'Enabled' : 'Disabled')}`, { padding: 1, borderStyle: 'round', borderColor: '#E88B5A' }));
|
|
||||||
const spinner = ora(`${this.emojis.brain} Scanning neural pathways...`).start();
|
|
||||||
try {
|
|
||||||
// Phase 1: Collect data
|
|
||||||
spinner.text = `${this.emojis.gear} Extracting neural data...`;
|
|
||||||
const backupData = await this.collectBackupData(options);
|
|
||||||
// Phase 2: Create manifest
|
|
||||||
spinner.text = `${this.emojis.atom} Generating quantum manifest...`;
|
|
||||||
const manifest = await this.createManifest(backupData, options);
|
|
||||||
// Phase 3: Package data
|
|
||||||
spinner.text = `${this.emojis.archive} Packaging atomic data...`;
|
|
||||||
const packagedData = {
|
|
||||||
manifest,
|
|
||||||
data: backupData
|
|
||||||
};
|
|
||||||
// Phase 4: Compress if requested
|
|
||||||
let finalData = JSON.stringify(packagedData, null, 2);
|
|
||||||
if (options.compress) {
|
|
||||||
spinner.text = `${this.emojis.gear} Applying quantum compression...`;
|
|
||||||
finalData = await this.compressData(finalData);
|
|
||||||
}
|
|
||||||
// Phase 5: Encrypt if password provided
|
|
||||||
if (options.password) {
|
|
||||||
spinner.text = `${this.emojis.shield} Applying atomic encryption...`;
|
|
||||||
finalData = await this.encryptData(finalData, options.password);
|
|
||||||
}
|
|
||||||
// Phase 6: Write to file
|
|
||||||
spinner.text = `${this.emojis.disk} Storing in atomic vault...`;
|
|
||||||
await fs.writeFile(outputPath, finalData);
|
|
||||||
// Phase 7: Verify if requested
|
|
||||||
if (options.verify) {
|
|
||||||
spinner.text = `${this.emojis.check} Verifying atomic integrity...`;
|
|
||||||
await this.verifyBackup(outputPath, options);
|
|
||||||
}
|
|
||||||
spinner.succeed(this.colors.success(`${this.emojis.sparkle} Backup complete! Neural pathways preserved in atomic vault.`));
|
|
||||||
console.log(boxen(`${this.emojis.brain} ${this.colors.brain('BACKUP SUMMARY')}\n\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Entities:')} ${this.colors.primary(manifest.entityCount.toLocaleString())}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Relationships:')} ${this.colors.primary(manifest.relationshipCount.toLocaleString())}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Size:')} ${this.colors.highlight(this.formatFileSize(finalData.length))}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Location:')} ${this.colors.highlight(outputPath)}`, { padding: 1, borderStyle: 'round', borderColor: '#2D4A3A' }));
|
|
||||||
return outputPath;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
spinner.fail('Backup failed - atomic vault compromised!');
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Restore Brainy data from backup
|
|
||||||
*/
|
|
||||||
async restoreBackup(backupPath, options = {}) {
|
|
||||||
console.log(boxen(`${this.emojis.rocket} ${this.colors.brain('ATOMIC RESTORATION PROTOCOL')} ${this.emojis.atom}\n\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Initiating neural restoration sequence')}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Source:')} ${this.colors.highlight(backupPath)}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Mode:')} ${this.colors.highlight(options.dryRun ? 'Simulation' : 'Full Restore')}`, { padding: 1, borderStyle: 'round', borderColor: '#E88B5A' }));
|
|
||||||
const spinner = ora(`${this.emojis.brain} Loading atomic vault...`).start();
|
|
||||||
try {
|
|
||||||
// Phase 1: Load backup file
|
|
||||||
spinner.text = `${this.emojis.disk} Reading atomic data...`;
|
|
||||||
let rawData = await fs.readFile(backupPath, 'utf8');
|
|
||||||
// Phase 2: Decrypt if needed
|
|
||||||
if (options.password) {
|
|
||||||
spinner.text = `${this.emojis.shield} Decrypting atomic data...`;
|
|
||||||
rawData = await this.decryptData(rawData, options.password);
|
|
||||||
}
|
|
||||||
// Phase 3: Decompress if needed
|
|
||||||
spinner.text = `${this.emojis.gear} Decompressing quantum data...`;
|
|
||||||
const decompressedData = await this.decompressData(rawData);
|
|
||||||
// Phase 4: Parse backup data
|
|
||||||
const backupPackage = JSON.parse(decompressedData);
|
|
||||||
const { manifest, data } = backupPackage;
|
|
||||||
// Phase 5: Verify integrity
|
|
||||||
if (options.verify) {
|
|
||||||
spinner.text = `${this.emojis.check} Verifying atomic integrity...`;
|
|
||||||
await this.verifyRestoreData(data, manifest);
|
|
||||||
}
|
|
||||||
// Phase 6: Display what will be restored
|
|
||||||
console.log('\n' + boxen(`${this.emojis.brain} ${this.colors.brain('RESTORATION PREVIEW')}\n\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Backup Date:')} ${this.colors.highlight(new Date(manifest.timestamp).toLocaleString())}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Entities:')} ${this.colors.primary(manifest.entityCount.toLocaleString())}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Relationships:')} ${this.colors.primary(manifest.relationshipCount.toLocaleString())}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Storage Type:')} ${this.colors.highlight(manifest.storageType)}`, { padding: 1, borderStyle: 'round', borderColor: '#D67441' }));
|
|
||||||
if (options.dryRun) {
|
|
||||||
spinner.succeed(this.colors.success('Dry run complete - restoration simulation successful'));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
// Phase 7: Confirm restoration
|
|
||||||
if (!options.overwrite) {
|
|
||||||
const { confirm } = await prompts({
|
|
||||||
type: 'confirm',
|
|
||||||
name: 'confirm',
|
|
||||||
message: `${this.emojis.warning} This will replace current data. Continue?`,
|
|
||||||
initial: false
|
|
||||||
});
|
|
||||||
if (!confirm) {
|
|
||||||
spinner.info('Restoration cancelled by user');
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Phase 8: Restore data
|
|
||||||
spinner.text = `${this.emojis.rocket} Restoring neural pathways...`;
|
|
||||||
await this.executeRestore(data, manifest);
|
|
||||||
spinner.succeed(this.colors.success(`${this.emojis.sparkle} Restoration complete! Neural pathways successfully reconstructed.`));
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
spinner.fail('Restoration failed - atomic vault corrupted!');
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* List available backups in a directory
|
|
||||||
*/
|
|
||||||
async listBackups(directory = './backups') {
|
|
||||||
try {
|
|
||||||
const files = await fs.readdir(directory);
|
|
||||||
const backupFiles = files.filter(f => f.endsWith('.brainy') || f.endsWith('.json'));
|
|
||||||
const manifests = [];
|
|
||||||
for (const file of backupFiles) {
|
|
||||||
try {
|
|
||||||
const filePath = path.join(directory, file);
|
|
||||||
const manifest = await this.getBackupManifest(filePath);
|
|
||||||
if (manifest)
|
|
||||||
manifests.push(manifest);
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
// Skip invalid backup files
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return manifests.sort((a, b) => new Date(b.timestamp).getTime() - new Date(a.timestamp).getTime());
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get backup manifest without loading full backup
|
|
||||||
*/
|
|
||||||
async getBackupManifest(backupPath) {
|
|
||||||
try {
|
|
||||||
const rawData = await fs.readFile(backupPath, 'utf8');
|
|
||||||
const decompressedData = await this.decompressData(rawData);
|
|
||||||
const backupPackage = JSON.parse(decompressedData);
|
|
||||||
return backupPackage.manifest || null;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Collect all data for backup
|
|
||||||
*/
|
|
||||||
async collectBackupData(options) {
|
|
||||||
const data = {
|
|
||||||
entities: [],
|
|
||||||
relationships: [],
|
|
||||||
metadata: {},
|
|
||||||
statistics: null
|
|
||||||
};
|
|
||||||
// For now, we'll create a simplified backup that just captures the current state
|
|
||||||
// In a full implementation, this would use internal storage methods
|
|
||||||
console.log(this.colors.warning('Note: Backup system is in beta - captures basic data only'));
|
|
||||||
// Placeholder data collection
|
|
||||||
data.entities = [];
|
|
||||||
data.relationships = [];
|
|
||||||
// Collect metadata if requested
|
|
||||||
if (options.includeMetadata) {
|
|
||||||
data.metadata = await this.collectMetadata();
|
|
||||||
}
|
|
||||||
// Statistics placeholder
|
|
||||||
if (options.includeStatistics) {
|
|
||||||
data.statistics = {
|
|
||||||
timestamp: new Date().toISOString(),
|
|
||||||
placeholder: true
|
|
||||||
};
|
|
||||||
}
|
|
||||||
return data;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Create backup manifest
|
|
||||||
*/
|
|
||||||
async createManifest(data, options) {
|
|
||||||
return {
|
|
||||||
version: '1.0.0',
|
|
||||||
timestamp: new Date().toISOString(),
|
|
||||||
brainyVersion: '0.55.0', // Would come from package.json
|
|
||||||
entityCount: data.entities.length,
|
|
||||||
relationshipCount: data.relationships.length,
|
|
||||||
storageType: 'unknown', // Would detect from brainy instance
|
|
||||||
compressed: options.compress || false,
|
|
||||||
encrypted: !!options.password,
|
|
||||||
checksum: await this.calculateChecksum(JSON.stringify(data)),
|
|
||||||
metadata: {
|
|
||||||
created: new Date().toISOString(),
|
|
||||||
description: 'Atomic age brain backup',
|
|
||||||
tags: ['brainy', 'neural-backup', 'atomic-data']
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Helper methods
|
|
||||||
*/
|
|
||||||
generateBackupPath() {
|
|
||||||
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
|
|
||||||
return `./brainy-backup-${timestamp}.brainy`;
|
|
||||||
}
|
|
||||||
async compressData(data) {
|
|
||||||
// Placeholder - would use zlib or similar
|
|
||||||
return data; // For now, no compression
|
|
||||||
}
|
|
||||||
async decompressData(data) {
|
|
||||||
// Placeholder - would use zlib or similar
|
|
||||||
return data; // For now, no decompression
|
|
||||||
}
|
|
||||||
async encryptData(data, password) {
|
|
||||||
// Placeholder - would use crypto module
|
|
||||||
return data; // For now, no encryption
|
|
||||||
}
|
|
||||||
async decryptData(data, password) {
|
|
||||||
// Placeholder - would use crypto module
|
|
||||||
return data; // For now, no decryption
|
|
||||||
}
|
|
||||||
async verifyBackup(backupPath, options) {
|
|
||||||
// Placeholder - would verify backup integrity
|
|
||||||
}
|
|
||||||
async verifyRestoreData(data, manifest) {
|
|
||||||
const actualChecksum = await this.calculateChecksum(JSON.stringify(data));
|
|
||||||
if (actualChecksum !== manifest.checksum) {
|
|
||||||
throw new Error('Data integrity check failed - backup may be corrupted');
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async executeRestore(data, manifest) {
|
|
||||||
// Placeholder restore implementation
|
|
||||||
console.log(this.colors.warning('Note: Restore system is in beta - limited functionality'));
|
|
||||||
// Phase 1: Validate data structure
|
|
||||||
if (!data.entities || !Array.isArray(data.entities)) {
|
|
||||||
throw new Error('Invalid backup data structure');
|
|
||||||
}
|
|
||||||
// Phase 2: Restore entities (placeholder)
|
|
||||||
console.log(this.colors.info(`Would restore ${data.entities.length} entities`));
|
|
||||||
// Phase 3: Restore relationships (placeholder)
|
|
||||||
console.log(this.colors.info(`Would restore ${data.relationships.length} relationships`));
|
|
||||||
// Phase 4: Restore metadata (placeholder)
|
|
||||||
if (data.metadata) {
|
|
||||||
await this.restoreMetadata(data.metadata);
|
|
||||||
}
|
|
||||||
// Phase 5: Simulate successful restore
|
|
||||||
console.log(this.colors.success('Backup structure validated - restore would be successful'));
|
|
||||||
}
|
|
||||||
async collectMetadata() {
|
|
||||||
// Collect global metadata
|
|
||||||
return {};
|
|
||||||
}
|
|
||||||
async restoreMetadata(metadata) {
|
|
||||||
// Restore global metadata
|
|
||||||
}
|
|
||||||
async calculateChecksum(data) {
|
|
||||||
// Placeholder - would calculate SHA-256 hash
|
|
||||||
return 'checksum-placeholder';
|
|
||||||
}
|
|
||||||
formatFileSize(bytes) {
|
|
||||||
const units = ['B', 'KB', 'MB', 'GB'];
|
|
||||||
let size = bytes;
|
|
||||||
let unitIndex = 0;
|
|
||||||
while (size >= 1024 && unitIndex < units.length - 1) {
|
|
||||||
size /= 1024;
|
|
||||||
unitIndex++;
|
|
||||||
}
|
|
||||||
return `${size.toFixed(1)} ${units[unitIndex]}`;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=backupRestore.js.map
|
|
||||||
1
dist/cortex/backupRestore.js.map
vendored
1
dist/cortex/backupRestore.js.map
vendored
File diff suppressed because one or more lines are too long
85
dist/cortex/healthCheck.d.ts
vendored
85
dist/cortex/healthCheck.d.ts
vendored
|
|
@ -1,85 +0,0 @@
|
||||||
/**
|
|
||||||
* Health Check System - Atomic Age Diagnostic Engine
|
|
||||||
*
|
|
||||||
* 🧠 Comprehensive health diagnostics for vector + graph operations
|
|
||||||
* ⚛️ Auto-repair capabilities with 1950s retro sci-fi aesthetics
|
|
||||||
* 🚀 Scalable health monitoring for high-performance databases
|
|
||||||
*/
|
|
||||||
import { BrainyData } from '../brainyData.js';
|
|
||||||
export interface HealthCheckResult {
|
|
||||||
component: string;
|
|
||||||
status: 'healthy' | 'warning' | 'critical' | 'offline';
|
|
||||||
score: number;
|
|
||||||
message: string;
|
|
||||||
details?: string[];
|
|
||||||
autoFixAvailable?: boolean;
|
|
||||||
lastChecked: string;
|
|
||||||
responseTime?: number;
|
|
||||||
}
|
|
||||||
export interface SystemHealth {
|
|
||||||
overall: HealthCheckResult;
|
|
||||||
vector: HealthCheckResult;
|
|
||||||
graph: HealthCheckResult;
|
|
||||||
storage: HealthCheckResult;
|
|
||||||
memory: HealthCheckResult;
|
|
||||||
network: HealthCheckResult;
|
|
||||||
embedding: HealthCheckResult;
|
|
||||||
cache: HealthCheckResult;
|
|
||||||
timestamp: string;
|
|
||||||
recommendations: string[];
|
|
||||||
}
|
|
||||||
export interface RepairAction {
|
|
||||||
id: string;
|
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
severity: 'low' | 'medium' | 'high';
|
|
||||||
automated: boolean;
|
|
||||||
estimatedTime: string;
|
|
||||||
riskLevel: 'safe' | 'moderate' | 'high';
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Comprehensive Health Check and Auto-Repair System
|
|
||||||
*/
|
|
||||||
export declare class HealthCheck {
|
|
||||||
private brainy;
|
|
||||||
private colors;
|
|
||||||
private emojis;
|
|
||||||
constructor(brainy: BrainyData);
|
|
||||||
/**
|
|
||||||
* Run comprehensive system health check
|
|
||||||
*/
|
|
||||||
runHealthCheck(): Promise<SystemHealth>;
|
|
||||||
/**
|
|
||||||
* Display health check results in terminal
|
|
||||||
*/
|
|
||||||
displayHealthReport(health?: SystemHealth): Promise<void>;
|
|
||||||
/**
|
|
||||||
* Get available repair actions
|
|
||||||
*/
|
|
||||||
getRepairActions(): Promise<RepairAction[]>;
|
|
||||||
/**
|
|
||||||
* Execute automated repairs
|
|
||||||
*/
|
|
||||||
executeAutoRepairs(): Promise<{
|
|
||||||
success: string[];
|
|
||||||
failed: string[];
|
|
||||||
}>;
|
|
||||||
/**
|
|
||||||
* Individual health check methods
|
|
||||||
*/
|
|
||||||
private checkVectorOperations;
|
|
||||||
private checkGraphOperations;
|
|
||||||
private checkStorageHealth;
|
|
||||||
private checkMemoryHealth;
|
|
||||||
private checkNetworkHealth;
|
|
||||||
private checkEmbeddingHealth;
|
|
||||||
private checkCacheHealth;
|
|
||||||
/**
|
|
||||||
* Helper methods
|
|
||||||
*/
|
|
||||||
private getOverallMessage;
|
|
||||||
private generateRecommendations;
|
|
||||||
private getHealthIcon;
|
|
||||||
private getStatusColor;
|
|
||||||
private executeRepairAction;
|
|
||||||
}
|
|
||||||
546
dist/cortex/healthCheck.js
vendored
546
dist/cortex/healthCheck.js
vendored
|
|
@ -1,546 +0,0 @@
|
||||||
/**
|
|
||||||
* Health Check System - Atomic Age Diagnostic Engine
|
|
||||||
*
|
|
||||||
* 🧠 Comprehensive health diagnostics for vector + graph operations
|
|
||||||
* ⚛️ Auto-repair capabilities with 1950s retro sci-fi aesthetics
|
|
||||||
* 🚀 Scalable health monitoring for high-performance databases
|
|
||||||
*/
|
|
||||||
// @ts-ignore
|
|
||||||
import chalk from 'chalk';
|
|
||||||
// @ts-ignore
|
|
||||||
import boxen from 'boxen';
|
|
||||||
// @ts-ignore
|
|
||||||
import ora from 'ora';
|
|
||||||
/**
|
|
||||||
* Comprehensive Health Check and Auto-Repair System
|
|
||||||
*/
|
|
||||||
export class HealthCheck {
|
|
||||||
constructor(brainy) {
|
|
||||||
this.colors = {
|
|
||||||
primary: chalk.hex('#3A5F4A'),
|
|
||||||
success: chalk.hex('#2D4A3A'),
|
|
||||||
warning: chalk.hex('#D67441'),
|
|
||||||
error: chalk.hex('#B85C35'),
|
|
||||||
info: chalk.hex('#4A6B5A'),
|
|
||||||
dim: chalk.hex('#8A9B8A'),
|
|
||||||
highlight: chalk.hex('#E88B5A'),
|
|
||||||
accent: chalk.hex('#F5E6D3'),
|
|
||||||
brain: chalk.hex('#E88B5A')
|
|
||||||
};
|
|
||||||
this.emojis = {
|
|
||||||
brain: '🧠',
|
|
||||||
atom: '⚛️',
|
|
||||||
health: '💚',
|
|
||||||
warning: '⚠️',
|
|
||||||
critical: '🔥',
|
|
||||||
offline: '💀',
|
|
||||||
repair: '🔧',
|
|
||||||
shield: '🛡️',
|
|
||||||
rocket: '🚀',
|
|
||||||
gear: '⚙️',
|
|
||||||
check: '✅',
|
|
||||||
cross: '❌',
|
|
||||||
lightning: '⚡',
|
|
||||||
sparkle: '✨'
|
|
||||||
};
|
|
||||||
this.brainy = brainy;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Run comprehensive system health check
|
|
||||||
*/
|
|
||||||
async runHealthCheck() {
|
|
||||||
console.log(boxen(`${this.emojis.shield} ${this.colors.brain('ATOMIC DIAGNOSTIC ENGINE')} ${this.emojis.atom}\n\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Initiating comprehensive system diagnostics')}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Scanning vector + graph database health')}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Auto-repair recommendations included')}`, { padding: 1, borderStyle: 'round', borderColor: '#E88B5A' }));
|
|
||||||
const spinner = ora(`${this.emojis.brain} Running neural diagnostics...`).start();
|
|
||||||
try {
|
|
||||||
// Run all health checks in parallel for speed
|
|
||||||
const [vectorHealth, graphHealth, storageHealth, memoryHealth, networkHealth, embeddingHealth, cacheHealth] = await Promise.all([
|
|
||||||
this.checkVectorOperations(spinner),
|
|
||||||
this.checkGraphOperations(spinner),
|
|
||||||
this.checkStorageHealth(spinner),
|
|
||||||
this.checkMemoryHealth(spinner),
|
|
||||||
this.checkNetworkHealth(spinner),
|
|
||||||
this.checkEmbeddingHealth(spinner),
|
|
||||||
this.checkCacheHealth(spinner)
|
|
||||||
]);
|
|
||||||
// Calculate overall health
|
|
||||||
const components = [vectorHealth, graphHealth, storageHealth, memoryHealth, networkHealth, embeddingHealth, cacheHealth];
|
|
||||||
const averageScore = components.reduce((sum, c) => sum + c.score, 0) / components.length;
|
|
||||||
const criticalIssues = components.filter(c => c.status === 'critical').length;
|
|
||||||
const warnings = components.filter(c => c.status === 'warning').length;
|
|
||||||
const overallStatus = criticalIssues > 0 ? 'critical' :
|
|
||||||
warnings > 2 ? 'warning' :
|
|
||||||
averageScore >= 90 ? 'healthy' : 'warning';
|
|
||||||
const overall = {
|
|
||||||
component: 'System Overall',
|
|
||||||
status: overallStatus,
|
|
||||||
score: Math.floor(averageScore),
|
|
||||||
message: this.getOverallMessage(overallStatus, criticalIssues, warnings),
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
const health = {
|
|
||||||
overall,
|
|
||||||
vector: vectorHealth,
|
|
||||||
graph: graphHealth,
|
|
||||||
storage: storageHealth,
|
|
||||||
memory: memoryHealth,
|
|
||||||
network: networkHealth,
|
|
||||||
embedding: embeddingHealth,
|
|
||||||
cache: cacheHealth,
|
|
||||||
timestamp: new Date().toISOString(),
|
|
||||||
recommendations: this.generateRecommendations(components)
|
|
||||||
};
|
|
||||||
spinner.succeed(this.colors.success(`${this.emojis.sparkle} Health check complete - Neural pathways analyzed`));
|
|
||||||
return health;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
spinner.fail('Health check failed - Diagnostic systems compromised!');
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Display health check results in terminal
|
|
||||||
*/
|
|
||||||
async displayHealthReport(health) {
|
|
||||||
if (!health) {
|
|
||||||
health = await this.runHealthCheck();
|
|
||||||
}
|
|
||||||
console.log('\n' + boxen(`${this.emojis.brain} ${this.colors.brain('SYSTEM HEALTH REPORT')} ${this.emojis.atom}\n` +
|
|
||||||
`${this.colors.dim('Comprehensive Vector + Graph Database Diagnostics')}\n` +
|
|
||||||
`${this.colors.accent('Overall Health:')} ${this.getHealthIcon(health.overall.status)} ${this.colors.primary(health.overall.score + '/100')}`, { padding: 1, borderStyle: 'double', borderColor: '#E88B5A', width: 80 }));
|
|
||||||
// Component Health Status
|
|
||||||
const components = [
|
|
||||||
health.vector,
|
|
||||||
health.graph,
|
|
||||||
health.storage,
|
|
||||||
health.memory,
|
|
||||||
health.network,
|
|
||||||
health.embedding,
|
|
||||||
health.cache
|
|
||||||
];
|
|
||||||
console.log('\n' + this.colors.brain(`${this.emojis.gear} COMPONENT STATUS`));
|
|
||||||
components.forEach(component => {
|
|
||||||
const statusColor = this.getStatusColor(component.status);
|
|
||||||
const icon = this.getHealthIcon(component.status);
|
|
||||||
const timeStr = component.responseTime ? ` (${component.responseTime}ms)` : '';
|
|
||||||
console.log(`${icon} ${statusColor(component.component.padEnd(20))} ` +
|
|
||||||
`${this.colors.primary((component.score + '/100').padEnd(8))} ` +
|
|
||||||
`${this.colors.dim(component.message)}${timeStr}`);
|
|
||||||
if (component.details && component.details.length > 0) {
|
|
||||||
component.details.forEach(detail => {
|
|
||||||
console.log(` ${this.colors.dim('→')} ${this.colors.accent(detail)}`);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
// Auto-repair recommendations
|
|
||||||
if (health.recommendations.length > 0) {
|
|
||||||
console.log('\n' + this.colors.warning(`${this.emojis.repair} AUTO-REPAIR RECOMMENDATIONS`));
|
|
||||||
console.log(boxen(health.recommendations.map((rec, i) => `${this.colors.accent((i + 1) + '.')} ${this.colors.dim(rec)}`).join('\n'), { padding: 1, borderStyle: 'round', borderColor: '#D67441' }));
|
|
||||||
}
|
|
||||||
// Critical issues
|
|
||||||
const criticalComponents = components.filter(c => c.status === 'critical');
|
|
||||||
if (criticalComponents.length > 0) {
|
|
||||||
console.log('\n' + this.colors.error(`${this.emojis.critical} CRITICAL ISSUES REQUIRING ATTENTION`));
|
|
||||||
criticalComponents.forEach(component => {
|
|
||||||
console.log(this.colors.error(` ${this.emojis.cross} ${component.component}: ${component.message}`));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
console.log('\n' + this.colors.dim(`Report generated: ${new Date(health.timestamp).toLocaleString()}`));
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Get available repair actions
|
|
||||||
*/
|
|
||||||
async getRepairActions() {
|
|
||||||
const health = await this.runHealthCheck();
|
|
||||||
const actions = [];
|
|
||||||
// Vector operations repairs
|
|
||||||
if (health.vector.status !== 'healthy') {
|
|
||||||
actions.push({
|
|
||||||
id: 'rebuild-vector-index',
|
|
||||||
name: 'Rebuild Vector Index',
|
|
||||||
description: 'Reconstruct HNSW index for optimal vector search performance',
|
|
||||||
severity: 'medium',
|
|
||||||
automated: true,
|
|
||||||
estimatedTime: '2-5 minutes',
|
|
||||||
riskLevel: 'safe'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
// Graph operations repairs
|
|
||||||
if (health.graph.status !== 'healthy') {
|
|
||||||
actions.push({
|
|
||||||
id: 'optimize-graph-connections',
|
|
||||||
name: 'Optimize Graph Connections',
|
|
||||||
description: 'Clean up orphaned relationships and optimize graph traversal paths',
|
|
||||||
severity: 'medium',
|
|
||||||
automated: true,
|
|
||||||
estimatedTime: '1-3 minutes',
|
|
||||||
riskLevel: 'safe'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
// Memory optimization
|
|
||||||
if (health.memory.score < 70) {
|
|
||||||
actions.push({
|
|
||||||
id: 'optimize-memory-usage',
|
|
||||||
name: 'Optimize Memory Usage',
|
|
||||||
description: 'Clear unused caches and optimize memory allocation',
|
|
||||||
severity: 'low',
|
|
||||||
automated: true,
|
|
||||||
estimatedTime: '30 seconds',
|
|
||||||
riskLevel: 'safe'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
// Cache optimization
|
|
||||||
if (health.cache.score < 80) {
|
|
||||||
actions.push({
|
|
||||||
id: 'rebuild-cache-indexes',
|
|
||||||
name: 'Rebuild Cache Indexes',
|
|
||||||
description: 'Optimize cache data structures for better hit rates',
|
|
||||||
severity: 'low',
|
|
||||||
automated: true,
|
|
||||||
estimatedTime: '1-2 minutes',
|
|
||||||
riskLevel: 'safe'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
// Storage optimization
|
|
||||||
if (health.storage.score < 75) {
|
|
||||||
actions.push({
|
|
||||||
id: 'compress-storage-data',
|
|
||||||
name: 'Compress Storage Data',
|
|
||||||
description: 'Apply compression to reduce storage size and improve I/O',
|
|
||||||
severity: 'medium',
|
|
||||||
automated: false,
|
|
||||||
estimatedTime: '5-15 minutes',
|
|
||||||
riskLevel: 'moderate'
|
|
||||||
});
|
|
||||||
}
|
|
||||||
return actions;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Execute automated repairs
|
|
||||||
*/
|
|
||||||
async executeAutoRepairs() {
|
|
||||||
const actions = await this.getRepairActions();
|
|
||||||
const automatedActions = actions.filter(a => a.automated && a.riskLevel === 'safe');
|
|
||||||
if (automatedActions.length === 0) {
|
|
||||||
console.log(this.colors.info('No safe automated repairs available'));
|
|
||||||
return { success: [], failed: [] };
|
|
||||||
}
|
|
||||||
console.log(boxen(`${this.emojis.repair} ${this.colors.brain('AUTOMATED REPAIR SEQUENCE')} ${this.emojis.atom}\n\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Executing safe automated repairs')}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Actions:')} ${this.colors.highlight(automatedActions.length.toString())}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Risk Level:')} ${this.colors.success('Safe')}`, { padding: 1, borderStyle: 'round', borderColor: '#E88B5A' }));
|
|
||||||
const success = [];
|
|
||||||
const failed = [];
|
|
||||||
for (const action of automatedActions) {
|
|
||||||
const spinner = ora(`${this.emojis.gear} Executing: ${action.name}`).start();
|
|
||||||
try {
|
|
||||||
await this.executeRepairAction(action);
|
|
||||||
spinner.succeed(this.colors.success(`${action.name} completed successfully`));
|
|
||||||
success.push(action.name);
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
spinner.fail(this.colors.error(`${action.name} failed: ${error}`));
|
|
||||||
failed.push(action.name);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (success.length > 0) {
|
|
||||||
console.log(this.colors.success(`\n${this.emojis.sparkle} Auto-repair complete: ${success.length} actions successful`));
|
|
||||||
}
|
|
||||||
if (failed.length > 0) {
|
|
||||||
console.log(this.colors.warning(`${this.emojis.warning} ${failed.length} actions failed - manual intervention required`));
|
|
||||||
}
|
|
||||||
return { success, failed };
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Individual health check methods
|
|
||||||
*/
|
|
||||||
async checkVectorOperations(spinner) {
|
|
||||||
spinner.text = `${this.emojis.lightning} Checking vector operations...`;
|
|
||||||
const startTime = Date.now();
|
|
||||||
try {
|
|
||||||
// Simulate vector health check
|
|
||||||
await new Promise(resolve => setTimeout(resolve, 200 + Math.random() * 300));
|
|
||||||
const responseTime = Date.now() - startTime;
|
|
||||||
const score = Math.floor(85 + Math.random() * 15);
|
|
||||||
const status = score >= 90 ? 'healthy' : score >= 70 ? 'warning' : 'critical';
|
|
||||||
return {
|
|
||||||
component: 'Vector Operations',
|
|
||||||
status,
|
|
||||||
score,
|
|
||||||
message: status === 'healthy' ? 'Optimal vector search performance' :
|
|
||||||
status === 'warning' ? 'Vector search slower than optimal' :
|
|
||||||
'Vector search performance degraded',
|
|
||||||
details: [
|
|
||||||
`HNSW Index: ${score >= 85 ? 'Optimized' : 'Needs rebuilding'}`,
|
|
||||||
`Embedding Cache: ${score >= 80 ? 'Efficient' : 'Cache misses high'}`,
|
|
||||||
`Query Latency: ${responseTime}ms average`
|
|
||||||
],
|
|
||||||
autoFixAvailable: score < 85,
|
|
||||||
lastChecked: new Date().toISOString(),
|
|
||||||
responseTime
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return {
|
|
||||||
component: 'Vector Operations',
|
|
||||||
status: 'critical',
|
|
||||||
score: 0,
|
|
||||||
message: 'Vector operations failed',
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async checkGraphOperations(spinner) {
|
|
||||||
spinner.text = `${this.emojis.gear} Checking graph operations...`;
|
|
||||||
const startTime = Date.now();
|
|
||||||
try {
|
|
||||||
await new Promise(resolve => setTimeout(resolve, 150 + Math.random() * 200));
|
|
||||||
const responseTime = Date.now() - startTime;
|
|
||||||
const score = Math.floor(80 + Math.random() * 20);
|
|
||||||
const status = score >= 90 ? 'healthy' : score >= 70 ? 'warning' : 'critical';
|
|
||||||
return {
|
|
||||||
component: 'Graph Operations',
|
|
||||||
status,
|
|
||||||
score,
|
|
||||||
message: status === 'healthy' ? 'Graph traversal performing optimally' :
|
|
||||||
status === 'warning' ? 'Graph queries slower than expected' :
|
|
||||||
'Graph operations significantly degraded',
|
|
||||||
details: [
|
|
||||||
`Relationship Index: ${score >= 85 ? 'Optimized' : 'Fragmented'}`,
|
|
||||||
`Traversal Cache: ${score >= 75 ? 'Efficient' : 'Low hit rate'}`,
|
|
||||||
`Connection Health: ${score >= 80 ? 'Good' : 'Orphaned connections detected'}`
|
|
||||||
],
|
|
||||||
autoFixAvailable: score < 80,
|
|
||||||
lastChecked: new Date().toISOString(),
|
|
||||||
responseTime
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return {
|
|
||||||
component: 'Graph Operations',
|
|
||||||
status: 'critical',
|
|
||||||
score: 0,
|
|
||||||
message: 'Graph operations failed',
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async checkStorageHealth(spinner) {
|
|
||||||
spinner.text = `${this.emojis.shield} Checking storage systems...`;
|
|
||||||
try {
|
|
||||||
await new Promise(resolve => setTimeout(resolve, 100 + Math.random() * 200));
|
|
||||||
const score = Math.floor(88 + Math.random() * 12);
|
|
||||||
const status = score >= 90 ? 'healthy' : score >= 75 ? 'warning' : 'critical';
|
|
||||||
return {
|
|
||||||
component: 'Storage Systems',
|
|
||||||
status,
|
|
||||||
score,
|
|
||||||
message: status === 'healthy' ? 'Storage operating at peak efficiency' :
|
|
||||||
status === 'warning' ? 'Storage performance below optimal' :
|
|
||||||
'Storage systems experiencing issues',
|
|
||||||
details: [
|
|
||||||
`I/O Performance: ${score >= 85 ? 'Excellent' : 'Needs optimization'}`,
|
|
||||||
`Data Integrity: ${score >= 90 ? 'Verified' : 'Minor inconsistencies'}`,
|
|
||||||
`Compression Ratio: ${score >= 80 ? 'Optimal' : 'Can be improved'}`
|
|
||||||
],
|
|
||||||
autoFixAvailable: score < 85,
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return {
|
|
||||||
component: 'Storage Systems',
|
|
||||||
status: 'offline',
|
|
||||||
score: 0,
|
|
||||||
message: 'Storage systems offline',
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async checkMemoryHealth(spinner) {
|
|
||||||
spinner.text = `${this.emojis.brain} Analyzing memory usage...`;
|
|
||||||
try {
|
|
||||||
const memUsage = process.memoryUsage();
|
|
||||||
const heapUsedMB = memUsage.heapUsed / (1024 * 1024);
|
|
||||||
const heapTotalMB = memUsage.heapTotal / (1024 * 1024);
|
|
||||||
const usage = (heapUsedMB / heapTotalMB) * 100;
|
|
||||||
const score = usage < 70 ? 95 : usage < 85 ? 80 : usage < 95 ? 60 : 30;
|
|
||||||
const status = score >= 80 ? 'healthy' : score >= 60 ? 'warning' : 'critical';
|
|
||||||
return {
|
|
||||||
component: 'Memory Management',
|
|
||||||
status,
|
|
||||||
score,
|
|
||||||
message: status === 'healthy' ? 'Memory usage within optimal range' :
|
|
||||||
status === 'warning' ? 'Memory usage elevated but stable' :
|
|
||||||
'Memory usage critically high',
|
|
||||||
details: [
|
|
||||||
`Heap Usage: ${heapUsedMB.toFixed(1)}MB / ${heapTotalMB.toFixed(1)}MB (${usage.toFixed(1)}%)`,
|
|
||||||
`Memory Efficiency: ${score >= 80 ? 'Excellent' : 'Needs optimization'}`,
|
|
||||||
`GC Pressure: ${usage < 70 ? 'Low' : usage < 85 ? 'Moderate' : 'High'}`
|
|
||||||
],
|
|
||||||
autoFixAvailable: score < 75,
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return {
|
|
||||||
component: 'Memory Management',
|
|
||||||
status: 'critical',
|
|
||||||
score: 0,
|
|
||||||
message: 'Memory analysis failed',
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async checkNetworkHealth(spinner) {
|
|
||||||
spinner.text = `${this.emojis.rocket} Testing network connectivity...`;
|
|
||||||
try {
|
|
||||||
await new Promise(resolve => setTimeout(resolve, 50 + Math.random() * 100));
|
|
||||||
const score = Math.floor(90 + Math.random() * 10);
|
|
||||||
const status = 'healthy'; // Assume healthy for local operations
|
|
||||||
return {
|
|
||||||
component: 'Network/Connectivity',
|
|
||||||
status,
|
|
||||||
score,
|
|
||||||
message: 'Network connectivity optimal',
|
|
||||||
details: [
|
|
||||||
'Local Operations: Excellent',
|
|
||||||
'API Endpoints: Responsive',
|
|
||||||
'Storage Access: Fast'
|
|
||||||
],
|
|
||||||
autoFixAvailable: false,
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return {
|
|
||||||
component: 'Network/Connectivity',
|
|
||||||
status: 'critical',
|
|
||||||
score: 0,
|
|
||||||
message: 'Network connectivity issues',
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async checkEmbeddingHealth(spinner) {
|
|
||||||
spinner.text = `${this.emojis.atom} Verifying embedding system...`;
|
|
||||||
try {
|
|
||||||
await new Promise(resolve => setTimeout(resolve, 300 + Math.random() * 200));
|
|
||||||
const score = Math.floor(85 + Math.random() * 15);
|
|
||||||
const status = score >= 90 ? 'healthy' : score >= 75 ? 'warning' : 'critical';
|
|
||||||
return {
|
|
||||||
component: 'Embedding System',
|
|
||||||
status,
|
|
||||||
score,
|
|
||||||
message: status === 'healthy' ? 'Embedding generation optimal' :
|
|
||||||
status === 'warning' ? 'Embedding performance acceptable' :
|
|
||||||
'Embedding system issues detected',
|
|
||||||
details: [
|
|
||||||
`Model Loading: ${score >= 85 ? 'Cached' : 'Slow to load'}`,
|
|
||||||
`Generation Speed: ${score >= 80 ? 'Fast' : 'Slower than expected'}`,
|
|
||||||
`Quality Score: ${score >= 90 ? 'Excellent' : 'Good'}`
|
|
||||||
],
|
|
||||||
autoFixAvailable: score < 85,
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return {
|
|
||||||
component: 'Embedding System',
|
|
||||||
status: 'critical',
|
|
||||||
score: 0,
|
|
||||||
message: 'Embedding system failed',
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async checkCacheHealth(spinner) {
|
|
||||||
spinner.text = `${this.emojis.lightning} Analyzing cache performance...`;
|
|
||||||
try {
|
|
||||||
await new Promise(resolve => setTimeout(resolve, 100 + Math.random() * 150));
|
|
||||||
const hitRate = 0.75 + Math.random() * 0.2;
|
|
||||||
const score = Math.floor(hitRate * 100);
|
|
||||||
const status = score >= 85 ? 'healthy' : score >= 70 ? 'warning' : 'critical';
|
|
||||||
return {
|
|
||||||
component: 'Cache System',
|
|
||||||
status,
|
|
||||||
score,
|
|
||||||
message: status === 'healthy' ? 'Cache performance excellent' :
|
|
||||||
status === 'warning' ? 'Cache hit rate below optimal' :
|
|
||||||
'Cache system underperforming',
|
|
||||||
details: [
|
|
||||||
`Hit Rate: ${(hitRate * 100).toFixed(1)}%`,
|
|
||||||
`Memory Efficiency: ${score >= 80 ? 'Good' : 'Needs optimization'}`,
|
|
||||||
`Eviction Rate: ${score >= 85 ? 'Low' : 'High'}`
|
|
||||||
],
|
|
||||||
autoFixAvailable: score < 80,
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
return {
|
|
||||||
component: 'Cache System',
|
|
||||||
status: 'critical',
|
|
||||||
score: 0,
|
|
||||||
message: 'Cache system failed',
|
|
||||||
lastChecked: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Helper methods
|
|
||||||
*/
|
|
||||||
getOverallMessage(status, critical, warnings) {
|
|
||||||
if (status === 'critical')
|
|
||||||
return `${critical} critical issue${critical > 1 ? 's' : ''} detected`;
|
|
||||||
if (status === 'warning')
|
|
||||||
return `${warnings} warning${warnings > 1 ? 's' : ''} detected`;
|
|
||||||
return 'All systems operating normally';
|
|
||||||
}
|
|
||||||
generateRecommendations(components) {
|
|
||||||
const recommendations = [];
|
|
||||||
components.forEach(component => {
|
|
||||||
if (component.status === 'critical') {
|
|
||||||
recommendations.push(`Immediate attention required for ${component.component}`);
|
|
||||||
}
|
|
||||||
else if (component.status === 'warning' && component.autoFixAvailable) {
|
|
||||||
recommendations.push(`Run auto-repair for ${component.component} to improve performance`);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
if (recommendations.length === 0) {
|
|
||||||
recommendations.push('All systems healthy - no actions required');
|
|
||||||
}
|
|
||||||
return recommendations;
|
|
||||||
}
|
|
||||||
getHealthIcon(status) {
|
|
||||||
switch (status) {
|
|
||||||
case 'healthy': return this.emojis.health;
|
|
||||||
case 'warning': return this.emojis.warning;
|
|
||||||
case 'critical': return this.emojis.critical;
|
|
||||||
case 'offline': return this.emojis.offline;
|
|
||||||
default: return this.emojis.gear;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
getStatusColor(status) {
|
|
||||||
switch (status) {
|
|
||||||
case 'healthy': return this.colors.success;
|
|
||||||
case 'warning': return this.colors.warning;
|
|
||||||
case 'critical': return this.colors.error;
|
|
||||||
case 'offline': return this.colors.dim;
|
|
||||||
default: return this.colors.info;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async executeRepairAction(action) {
|
|
||||||
// Simulate repair execution
|
|
||||||
const delay = action.estimatedTime.includes('second') ? 1000 :
|
|
||||||
action.estimatedTime.includes('minute') ? 2000 : 3000;
|
|
||||||
await new Promise(resolve => setTimeout(resolve, delay));
|
|
||||||
// Simulate occasional failure
|
|
||||||
if (Math.random() < 0.1) {
|
|
||||||
throw new Error('Repair action failed - manual intervention required');
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=healthCheck.js.map
|
|
||||||
1
dist/cortex/healthCheck.js.map
vendored
1
dist/cortex/healthCheck.js.map
vendored
File diff suppressed because one or more lines are too long
145
dist/cortex/neuralImport.d.ts
vendored
145
dist/cortex/neuralImport.d.ts
vendored
|
|
@ -1,145 +0,0 @@
|
||||||
/**
|
|
||||||
* Neural Import - Atomic Age AI-Powered Data Understanding System
|
|
||||||
*
|
|
||||||
* 🧠 Leveraging the brain-in-jar to understand and automatically structure data
|
|
||||||
* ⚛️ Complete with confidence scoring and relationship weight calculation
|
|
||||||
*/
|
|
||||||
import { BrainyData } from '../brainyData.js';
|
|
||||||
export interface NeuralAnalysisResult {
|
|
||||||
detectedEntities: DetectedEntity[];
|
|
||||||
detectedRelationships: DetectedRelationship[];
|
|
||||||
confidence: number;
|
|
||||||
insights: NeuralInsight[];
|
|
||||||
preview: ProcessedData[];
|
|
||||||
}
|
|
||||||
export interface DetectedEntity {
|
|
||||||
originalData: any;
|
|
||||||
nounType: string;
|
|
||||||
confidence: number;
|
|
||||||
suggestedId: string;
|
|
||||||
reasoning: string;
|
|
||||||
alternativeTypes: Array<{
|
|
||||||
type: string;
|
|
||||||
confidence: number;
|
|
||||||
}>;
|
|
||||||
}
|
|
||||||
export interface DetectedRelationship {
|
|
||||||
sourceId: string;
|
|
||||||
targetId: string;
|
|
||||||
verbType: string;
|
|
||||||
confidence: number;
|
|
||||||
weight: number;
|
|
||||||
reasoning: string;
|
|
||||||
context: string;
|
|
||||||
metadata?: Record<string, any>;
|
|
||||||
}
|
|
||||||
export interface NeuralInsight {
|
|
||||||
type: 'hierarchy' | 'cluster' | 'pattern' | 'anomaly' | 'opportunity';
|
|
||||||
description: string;
|
|
||||||
confidence: number;
|
|
||||||
affectedEntities: string[];
|
|
||||||
recommendation?: string;
|
|
||||||
}
|
|
||||||
export interface ProcessedData {
|
|
||||||
id: string;
|
|
||||||
nounType: string;
|
|
||||||
data: any;
|
|
||||||
relationships: Array<{
|
|
||||||
target: string;
|
|
||||||
verbType: string;
|
|
||||||
weight: number;
|
|
||||||
confidence: number;
|
|
||||||
}>;
|
|
||||||
}
|
|
||||||
export interface NeuralImportOptions {
|
|
||||||
confidenceThreshold: number;
|
|
||||||
autoApply: boolean;
|
|
||||||
enableWeights: boolean;
|
|
||||||
previewOnly: boolean;
|
|
||||||
validateOnly: boolean;
|
|
||||||
categoryFilter?: string[];
|
|
||||||
skipDuplicates: boolean;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Neural Import Engine - The Brain Behind the Analysis
|
|
||||||
*/
|
|
||||||
export declare class NeuralImport {
|
|
||||||
private brainy;
|
|
||||||
private colors;
|
|
||||||
private emojis;
|
|
||||||
constructor(brainy: BrainyData);
|
|
||||||
/**
|
|
||||||
* Main Neural Import Function - The Master Controller
|
|
||||||
*/
|
|
||||||
neuralImport(filePath: string, options?: Partial<NeuralImportOptions>): Promise<NeuralAnalysisResult>;
|
|
||||||
/**
|
|
||||||
* Parse file based on extension
|
|
||||||
*/
|
|
||||||
private parseFile;
|
|
||||||
/**
|
|
||||||
* Basic CSV parser
|
|
||||||
*/
|
|
||||||
private parseCSV;
|
|
||||||
/**
|
|
||||||
* Neural Entity Detection - The Core AI Engine
|
|
||||||
*/
|
|
||||||
private detectEntitiesWithNeuralAnalysis;
|
|
||||||
/**
|
|
||||||
* Calculate entity type confidence using AI
|
|
||||||
*/
|
|
||||||
private calculateEntityTypeConfidence;
|
|
||||||
/**
|
|
||||||
* Field-based confidence calculation
|
|
||||||
*/
|
|
||||||
private calculateFieldBasedConfidence;
|
|
||||||
/**
|
|
||||||
* Pattern-based confidence calculation
|
|
||||||
*/
|
|
||||||
private calculatePatternBasedConfidence;
|
|
||||||
/**
|
|
||||||
* Generate reasoning for entity type selection
|
|
||||||
*/
|
|
||||||
private generateEntityReasoning;
|
|
||||||
/**
|
|
||||||
* Neural Relationship Detection
|
|
||||||
*/
|
|
||||||
private detectRelationshipsWithNeuralAnalysis;
|
|
||||||
/**
|
|
||||||
* Calculate relationship confidence
|
|
||||||
*/
|
|
||||||
private calculateRelationshipConfidence;
|
|
||||||
/**
|
|
||||||
* Calculate relationship weight/strength
|
|
||||||
*/
|
|
||||||
private calculateRelationshipWeight;
|
|
||||||
/**
|
|
||||||
* Generate Neural Insights - The Intelligence Layer
|
|
||||||
*/
|
|
||||||
private generateNeuralInsights;
|
|
||||||
/**
|
|
||||||
* Display Neural Analysis Results
|
|
||||||
*/
|
|
||||||
private displayNeuralAnalysisResults;
|
|
||||||
/**
|
|
||||||
* Helper methods for the neural system
|
|
||||||
*/
|
|
||||||
private extractMainText;
|
|
||||||
private generateSmartId;
|
|
||||||
private extractRelationshipContext;
|
|
||||||
private calculateTypeCompatibility;
|
|
||||||
private getVerbSpecificity;
|
|
||||||
private getRelevantFields;
|
|
||||||
private getMatchedPatterns;
|
|
||||||
private pruneRelationships;
|
|
||||||
private detectHierarchies;
|
|
||||||
private detectClusters;
|
|
||||||
private detectPatterns;
|
|
||||||
private summarizeEntities;
|
|
||||||
private summarizeRelationships;
|
|
||||||
private calculateOverallConfidence;
|
|
||||||
private generatePreview;
|
|
||||||
private confirmNeuralImport;
|
|
||||||
private executeNeuralImport;
|
|
||||||
private generateRelationshipReasoning;
|
|
||||||
private extractRelationshipMetadata;
|
|
||||||
}
|
|
||||||
618
dist/cortex/neuralImport.js
vendored
618
dist/cortex/neuralImport.js
vendored
|
|
@ -1,618 +0,0 @@
|
||||||
/**
|
|
||||||
* Neural Import - Atomic Age AI-Powered Data Understanding System
|
|
||||||
*
|
|
||||||
* 🧠 Leveraging the brain-in-jar to understand and automatically structure data
|
|
||||||
* ⚛️ Complete with confidence scoring and relationship weight calculation
|
|
||||||
*/
|
|
||||||
import { NounType, VerbType } from '../types/graphTypes.js';
|
|
||||||
import * as fs from '../universal/fs.js';
|
|
||||||
import * as path from '../universal/path.js';
|
|
||||||
// @ts-ignore
|
|
||||||
import chalk from 'chalk';
|
|
||||||
// @ts-ignore
|
|
||||||
import ora from 'ora';
|
|
||||||
// @ts-ignore
|
|
||||||
import boxen from 'boxen';
|
|
||||||
// @ts-ignore
|
|
||||||
import Table from 'cli-table3';
|
|
||||||
// @ts-ignore
|
|
||||||
import prompts from 'prompts';
|
|
||||||
/**
|
|
||||||
* Neural Import Engine - The Brain Behind the Analysis
|
|
||||||
*/
|
|
||||||
export class NeuralImport {
|
|
||||||
constructor(brainy) {
|
|
||||||
this.colors = {
|
|
||||||
primary: chalk.hex('#3A5F4A'),
|
|
||||||
success: chalk.hex('#2D4A3A'),
|
|
||||||
warning: chalk.hex('#D67441'),
|
|
||||||
error: chalk.hex('#B85C35'),
|
|
||||||
info: chalk.hex('#4A6B5A'),
|
|
||||||
dim: chalk.hex('#8A9B8A'),
|
|
||||||
highlight: chalk.hex('#E88B5A'),
|
|
||||||
accent: chalk.hex('#F5E6D3'),
|
|
||||||
brain: chalk.hex('#E88B5A')
|
|
||||||
};
|
|
||||||
this.emojis = {
|
|
||||||
brain: '🧠',
|
|
||||||
atom: '⚛️',
|
|
||||||
lab: '🔬',
|
|
||||||
data: '🎛️',
|
|
||||||
magic: '⚡',
|
|
||||||
check: '✅',
|
|
||||||
warning: '⚠️',
|
|
||||||
sparkle: '✨',
|
|
||||||
rocket: '🚀',
|
|
||||||
gear: '⚙️'
|
|
||||||
};
|
|
||||||
this.brainy = brainy;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Main Neural Import Function - The Master Controller
|
|
||||||
*/
|
|
||||||
async neuralImport(filePath, options = {}) {
|
|
||||||
const opts = {
|
|
||||||
confidenceThreshold: 0.7,
|
|
||||||
autoApply: false,
|
|
||||||
enableWeights: true,
|
|
||||||
previewOnly: false,
|
|
||||||
validateOnly: false,
|
|
||||||
skipDuplicates: true,
|
|
||||||
...options
|
|
||||||
};
|
|
||||||
console.log(boxen(`${this.emojis.brain} ${this.colors.brain('NEURAL IMPORT INITIATED')} ${this.emojis.atom}\n\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Activating atomic age AI analysis')}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('File:')} ${this.colors.highlight(filePath)}\n` +
|
|
||||||
`${this.colors.accent('◆')} ${this.colors.dim('Confidence Threshold:')} ${this.colors.highlight(opts.confidenceThreshold.toString())}`, { padding: 1, borderStyle: 'round', borderColor: '#E88B5A' }));
|
|
||||||
const spinner = ora(`${this.emojis.brain} Initializing neural analysis...`).start();
|
|
||||||
try {
|
|
||||||
// Phase 1: Data Parsing
|
|
||||||
spinner.text = `${this.emojis.lab} Parsing data structure...`;
|
|
||||||
const rawData = await this.parseFile(filePath);
|
|
||||||
// Phase 2: Neural Entity Detection
|
|
||||||
spinner.text = `${this.emojis.atom} Analyzing ${Object.keys(NounType).length} entity types...`;
|
|
||||||
const detectedEntities = await this.detectEntitiesWithNeuralAnalysis(rawData, opts);
|
|
||||||
// Phase 3: Neural Relationship Detection
|
|
||||||
spinner.text = `${this.emojis.data} Testing ${Object.keys(VerbType).length} relationship patterns...`;
|
|
||||||
const detectedRelationships = await this.detectRelationshipsWithNeuralAnalysis(detectedEntities, rawData, opts);
|
|
||||||
// Phase 4: Neural Insights Generation
|
|
||||||
spinner.text = `${this.emojis.magic} Computing neural insights...`;
|
|
||||||
const insights = await this.generateNeuralInsights(detectedEntities, detectedRelationships);
|
|
||||||
// Phase 5: Confidence Scoring
|
|
||||||
const overallConfidence = this.calculateOverallConfidence(detectedEntities, detectedRelationships);
|
|
||||||
spinner.stop();
|
|
||||||
const result = {
|
|
||||||
detectedEntities,
|
|
||||||
detectedRelationships,
|
|
||||||
confidence: overallConfidence,
|
|
||||||
insights,
|
|
||||||
preview: await this.generatePreview(detectedEntities, detectedRelationships)
|
|
||||||
};
|
|
||||||
// Display results
|
|
||||||
await this.displayNeuralAnalysisResults(result, opts);
|
|
||||||
// Handle execution based on options
|
|
||||||
if (opts.previewOnly || opts.validateOnly) {
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
if (!opts.autoApply) {
|
|
||||||
const shouldExecute = await this.confirmNeuralImport(result);
|
|
||||||
if (!shouldExecute) {
|
|
||||||
console.log(this.colors.dim('Neural import cancelled'));
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Execute the import
|
|
||||||
await this.executeNeuralImport(result, opts);
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
spinner.fail('Neural analysis failed');
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Parse file based on extension
|
|
||||||
*/
|
|
||||||
async parseFile(filePath) {
|
|
||||||
const ext = path.extname(filePath).toLowerCase();
|
|
||||||
const content = await fs.readFile(filePath, 'utf8');
|
|
||||||
switch (ext) {
|
|
||||||
case '.json':
|
|
||||||
const jsonData = JSON.parse(content);
|
|
||||||
return Array.isArray(jsonData) ? jsonData : [jsonData];
|
|
||||||
case '.csv':
|
|
||||||
return this.parseCSV(content);
|
|
||||||
case '.yaml':
|
|
||||||
case '.yml':
|
|
||||||
// For now, basic YAML support - in full implementation would use yaml parser
|
|
||||||
return JSON.parse(content); // Placeholder
|
|
||||||
default:
|
|
||||||
throw new Error(`Unsupported file format: ${ext}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Basic CSV parser
|
|
||||||
*/
|
|
||||||
parseCSV(content) {
|
|
||||||
const lines = content.split('\n').filter(line => line.trim());
|
|
||||||
if (lines.length < 2)
|
|
||||||
return [];
|
|
||||||
const headers = lines[0].split(',').map(h => h.trim().replace(/"/g, ''));
|
|
||||||
const data = [];
|
|
||||||
for (let i = 1; i < lines.length; i++) {
|
|
||||||
const values = lines[i].split(',').map(v => v.trim().replace(/"/g, ''));
|
|
||||||
const row = {};
|
|
||||||
headers.forEach((header, index) => {
|
|
||||||
row[header] = values[index] || '';
|
|
||||||
});
|
|
||||||
data.push(row);
|
|
||||||
}
|
|
||||||
return data;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Neural Entity Detection - The Core AI Engine
|
|
||||||
*/
|
|
||||||
async detectEntitiesWithNeuralAnalysis(rawData, options) {
|
|
||||||
const entities = [];
|
|
||||||
const nounTypes = Object.values(NounType);
|
|
||||||
for (const [index, dataItem] of rawData.entries()) {
|
|
||||||
const mainText = this.extractMainText(dataItem);
|
|
||||||
const detections = [];
|
|
||||||
// Test against all noun types using semantic similarity
|
|
||||||
for (const nounType of nounTypes) {
|
|
||||||
const confidence = await this.calculateEntityTypeConfidence(mainText, dataItem, nounType);
|
|
||||||
if (confidence >= options.confidenceThreshold - 0.2) { // Allow slightly lower for alternatives
|
|
||||||
const reasoning = await this.generateEntityReasoning(mainText, dataItem, nounType);
|
|
||||||
detections.push({ type: nounType, confidence, reasoning });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (detections.length > 0) {
|
|
||||||
// Sort by confidence
|
|
||||||
detections.sort((a, b) => b.confidence - a.confidence);
|
|
||||||
const primaryType = detections[0];
|
|
||||||
const alternatives = detections.slice(1, 3); // Top 2 alternatives
|
|
||||||
entities.push({
|
|
||||||
originalData: dataItem,
|
|
||||||
nounType: primaryType.type,
|
|
||||||
confidence: primaryType.confidence,
|
|
||||||
suggestedId: this.generateSmartId(dataItem, primaryType.type, index),
|
|
||||||
reasoning: primaryType.reasoning,
|
|
||||||
alternativeTypes: alternatives
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return entities;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Calculate entity type confidence using AI
|
|
||||||
*/
|
|
||||||
async calculateEntityTypeConfidence(text, data, nounType) {
|
|
||||||
// Base semantic similarity using search instead of similarity method
|
|
||||||
const searchResults = await this.brainy.search(text + ' ' + nounType, 1);
|
|
||||||
const textSimilarity = searchResults.length > 0 ? searchResults[0].score : 0.5;
|
|
||||||
// Field-based confidence boost
|
|
||||||
const fieldBoost = this.calculateFieldBasedConfidence(data, nounType);
|
|
||||||
// Pattern-based confidence boost
|
|
||||||
const patternBoost = this.calculatePatternBasedConfidence(text, data, nounType);
|
|
||||||
// Combine confidences with weights
|
|
||||||
const combined = (textSimilarity * 0.5) + (fieldBoost * 0.3) + (patternBoost * 0.2);
|
|
||||||
return Math.min(combined, 1.0);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Field-based confidence calculation
|
|
||||||
*/
|
|
||||||
calculateFieldBasedConfidence(data, nounType) {
|
|
||||||
const fields = Object.keys(data);
|
|
||||||
let boost = 0;
|
|
||||||
// Field patterns that boost confidence for specific noun types
|
|
||||||
const fieldPatterns = {
|
|
||||||
[NounType.Person]: ['name', 'email', 'phone', 'age', 'firstname', 'lastname', 'employee'],
|
|
||||||
[NounType.Organization]: ['company', 'organization', 'corp', 'inc', 'ltd', 'department', 'team'],
|
|
||||||
[NounType.Project]: ['project', 'task', 'deadline', 'status', 'milestone', 'deliverable'],
|
|
||||||
[NounType.Location]: ['address', 'city', 'country', 'state', 'zip', 'location', 'coordinates'],
|
|
||||||
[NounType.Product]: ['product', 'price', 'sku', 'inventory', 'category', 'brand'],
|
|
||||||
[NounType.Event]: ['date', 'time', 'venue', 'event', 'meeting', 'conference', 'schedule']
|
|
||||||
};
|
|
||||||
const relevantPatterns = fieldPatterns[nounType] || [];
|
|
||||||
for (const field of fields) {
|
|
||||||
for (const pattern of relevantPatterns) {
|
|
||||||
if (field.toLowerCase().includes(pattern)) {
|
|
||||||
boost += 0.1;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return Math.min(boost, 0.5);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Pattern-based confidence calculation
|
|
||||||
*/
|
|
||||||
calculatePatternBasedConfidence(text, data, nounType) {
|
|
||||||
let boost = 0;
|
|
||||||
// Content patterns that indicate entity types
|
|
||||||
const patterns = {
|
|
||||||
[NounType.Person]: [
|
|
||||||
/@.*\.com/i, // Email pattern
|
|
||||||
/\b[A-Z][a-z]+ [A-Z][a-z]+\b/, // Name pattern
|
|
||||||
/Mr\.|Mrs\.|Dr\.|Prof\./i // Title pattern
|
|
||||||
],
|
|
||||||
[NounType.Organization]: [
|
|
||||||
/\bInc\.|Corp\.|LLC\.|Ltd\./i, // Corporate suffixes
|
|
||||||
/Company|Corporation|Enterprise/i
|
|
||||||
],
|
|
||||||
[NounType.Location]: [
|
|
||||||
/\b\d{5}(-\d{4})?\b/, // ZIP code
|
|
||||||
/Street|Ave|Road|Blvd/i
|
|
||||||
]
|
|
||||||
};
|
|
||||||
const relevantPatterns = patterns[nounType] || [];
|
|
||||||
for (const pattern of relevantPatterns) {
|
|
||||||
if (pattern.test(text)) {
|
|
||||||
boost += 0.15;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return Math.min(boost, 0.3);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Generate reasoning for entity type selection
|
|
||||||
*/
|
|
||||||
async generateEntityReasoning(text, data, nounType) {
|
|
||||||
const reasons = [];
|
|
||||||
// Semantic similarity reason using search
|
|
||||||
const searchResults = await this.brainy.search(text + ' ' + nounType, 1);
|
|
||||||
const similarity = searchResults.length > 0 ? searchResults[0].score : 0.5;
|
|
||||||
if (similarity > 0.7) {
|
|
||||||
reasons.push(`High semantic similarity (${(similarity * 100).toFixed(1)}%)`);
|
|
||||||
}
|
|
||||||
// Field-based reasons
|
|
||||||
const relevantFields = this.getRelevantFields(data, nounType);
|
|
||||||
if (relevantFields.length > 0) {
|
|
||||||
reasons.push(`Contains ${nounType}-specific fields: ${relevantFields.join(', ')}`);
|
|
||||||
}
|
|
||||||
// Pattern-based reasons
|
|
||||||
const matchedPatterns = this.getMatchedPatterns(text, data, nounType);
|
|
||||||
if (matchedPatterns.length > 0) {
|
|
||||||
reasons.push(`Matches ${nounType} patterns: ${matchedPatterns.join(', ')}`);
|
|
||||||
}
|
|
||||||
return reasons.length > 0 ? reasons.join('; ') : 'General semantic match';
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Neural Relationship Detection
|
|
||||||
*/
|
|
||||||
async detectRelationshipsWithNeuralAnalysis(entities, rawData, options) {
|
|
||||||
const relationships = [];
|
|
||||||
const verbTypes = Object.values(VerbType);
|
|
||||||
// For each pair of entities, test relationship possibilities
|
|
||||||
for (let i = 0; i < entities.length; i++) {
|
|
||||||
for (let j = i + 1; j < entities.length; j++) {
|
|
||||||
const sourceEntity = entities[i];
|
|
||||||
const targetEntity = entities[j];
|
|
||||||
// Extract context for relationship detection
|
|
||||||
const context = this.extractRelationshipContext(sourceEntity.originalData, targetEntity.originalData, rawData);
|
|
||||||
// Test all verb types
|
|
||||||
for (const verbType of verbTypes) {
|
|
||||||
const confidence = await this.calculateRelationshipConfidence(sourceEntity, targetEntity, verbType, context);
|
|
||||||
if (confidence >= options.confidenceThreshold - 0.1) { // Slightly lower threshold for relationships
|
|
||||||
const weight = options.enableWeights ?
|
|
||||||
this.calculateRelationshipWeight(sourceEntity, targetEntity, verbType, context) :
|
|
||||||
0.5;
|
|
||||||
const reasoning = await this.generateRelationshipReasoning(sourceEntity, targetEntity, verbType, context);
|
|
||||||
relationships.push({
|
|
||||||
sourceId: sourceEntity.suggestedId,
|
|
||||||
targetId: targetEntity.suggestedId,
|
|
||||||
verbType,
|
|
||||||
confidence,
|
|
||||||
weight,
|
|
||||||
reasoning,
|
|
||||||
context,
|
|
||||||
metadata: this.extractRelationshipMetadata(sourceEntity.originalData, targetEntity.originalData, verbType)
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Sort by confidence and remove duplicates/conflicts
|
|
||||||
return this.pruneRelationships(relationships);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Calculate relationship confidence
|
|
||||||
*/
|
|
||||||
async calculateRelationshipConfidence(source, target, verbType, context) {
|
|
||||||
// Semantic similarity between entities and verb type using search
|
|
||||||
const relationshipText = `${this.extractMainText(source.originalData)} ${verbType} ${this.extractMainText(target.originalData)}`;
|
|
||||||
const directResults = await this.brainy.search(relationshipText, 1);
|
|
||||||
const directSimilarity = directResults.length > 0 ? directResults[0].score : 0.5;
|
|
||||||
// Context-based similarity using search
|
|
||||||
const contextResults = await this.brainy.search(context + ' ' + verbType, 1);
|
|
||||||
const contextSimilarity = contextResults.length > 0 ? contextResults[0].score : 0.5;
|
|
||||||
// Entity type compatibility
|
|
||||||
const typeCompatibility = this.calculateTypeCompatibility(source.nounType, target.nounType, verbType);
|
|
||||||
// Combine with weights
|
|
||||||
return (directSimilarity * 0.4) + (contextSimilarity * 0.4) + (typeCompatibility * 0.2);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Calculate relationship weight/strength
|
|
||||||
*/
|
|
||||||
calculateRelationshipWeight(source, target, verbType, context) {
|
|
||||||
let weight = 0.5; // Base weight
|
|
||||||
// Context richness (more descriptive = stronger)
|
|
||||||
const contextWords = context.split(' ').length;
|
|
||||||
weight += Math.min(contextWords / 20, 0.2);
|
|
||||||
// Entity importance (higher confidence entities = stronger relationships)
|
|
||||||
const avgEntityConfidence = (source.confidence + target.confidence) / 2;
|
|
||||||
weight += avgEntityConfidence * 0.2;
|
|
||||||
// Verb type specificity (more specific verbs = stronger)
|
|
||||||
const verbSpecificity = this.getVerbSpecificity(verbType);
|
|
||||||
weight += verbSpecificity * 0.1;
|
|
||||||
return Math.min(weight, 1.0);
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Generate Neural Insights - The Intelligence Layer
|
|
||||||
*/
|
|
||||||
async generateNeuralInsights(entities, relationships) {
|
|
||||||
const insights = [];
|
|
||||||
// Detect hierarchies
|
|
||||||
const hierarchies = this.detectHierarchies(relationships);
|
|
||||||
hierarchies.forEach(hierarchy => {
|
|
||||||
insights.push({
|
|
||||||
type: 'hierarchy',
|
|
||||||
description: `Detected ${hierarchy.type} hierarchy with ${hierarchy.levels} levels`,
|
|
||||||
confidence: hierarchy.confidence,
|
|
||||||
affectedEntities: hierarchy.entities,
|
|
||||||
recommendation: `Consider visualizing the ${hierarchy.type} structure`
|
|
||||||
});
|
|
||||||
});
|
|
||||||
// Detect clusters
|
|
||||||
const clusters = this.detectClusters(entities, relationships);
|
|
||||||
clusters.forEach(cluster => {
|
|
||||||
insights.push({
|
|
||||||
type: 'cluster',
|
|
||||||
description: `Found cluster of ${cluster.size} ${cluster.primaryType} entities`,
|
|
||||||
confidence: cluster.confidence,
|
|
||||||
affectedEntities: cluster.entities,
|
|
||||||
recommendation: `These ${cluster.primaryType}s might form a natural grouping`
|
|
||||||
});
|
|
||||||
});
|
|
||||||
// Detect patterns
|
|
||||||
const patterns = this.detectPatterns(relationships);
|
|
||||||
patterns.forEach(pattern => {
|
|
||||||
insights.push({
|
|
||||||
type: 'pattern',
|
|
||||||
description: `Common relationship pattern: ${pattern.description}`,
|
|
||||||
confidence: pattern.confidence,
|
|
||||||
affectedEntities: pattern.entities,
|
|
||||||
recommendation: pattern.recommendation
|
|
||||||
});
|
|
||||||
});
|
|
||||||
return insights;
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Display Neural Analysis Results
|
|
||||||
*/
|
|
||||||
async displayNeuralAnalysisResults(result, options) {
|
|
||||||
// Entity summary
|
|
||||||
const entityTable = new Table({
|
|
||||||
head: [this.colors.brain('Entity Type'), this.colors.brain('Count'), this.colors.brain('Avg Confidence')],
|
|
||||||
colWidths: [20, 10, 15]
|
|
||||||
});
|
|
||||||
const entitySummary = this.summarizeEntities(result.detectedEntities);
|
|
||||||
Object.entries(entitySummary).forEach(([type, stats]) => {
|
|
||||||
entityTable.push([
|
|
||||||
this.colors.highlight(type),
|
|
||||||
this.colors.primary(stats.count.toString()),
|
|
||||||
this.colors.success(`${(stats.avgConfidence * 100).toFixed(1)}%`)
|
|
||||||
]);
|
|
||||||
});
|
|
||||||
// Relationship summary
|
|
||||||
const relationshipTable = new Table({
|
|
||||||
head: [this.colors.brain('Relationship Type'), this.colors.brain('Count'), this.colors.brain('Avg Weight'), this.colors.brain('Avg Confidence')],
|
|
||||||
colWidths: [20, 10, 12, 15]
|
|
||||||
});
|
|
||||||
const relationshipSummary = this.summarizeRelationships(result.detectedRelationships);
|
|
||||||
Object.entries(relationshipSummary).forEach(([type, stats]) => {
|
|
||||||
relationshipTable.push([
|
|
||||||
this.colors.highlight(type),
|
|
||||||
this.colors.primary(stats.count.toString()),
|
|
||||||
this.colors.warning(`${stats.avgWeight.toFixed(2)}`),
|
|
||||||
this.colors.success(`${(stats.avgConfidence * 100).toFixed(1)}%`)
|
|
||||||
]);
|
|
||||||
});
|
|
||||||
console.log(boxen(`${this.emojis.atom} ${this.colors.brain('NEURAL CLASSIFICATION RESULTS')}\n\n` +
|
|
||||||
entityTable.toString(), { padding: 1, borderStyle: 'round', borderColor: '#D67441' }));
|
|
||||||
console.log(boxen(`${this.emojis.data} ${this.colors.brain('NEURAL RELATIONSHIP MAPPING')}\n\n` +
|
|
||||||
relationshipTable.toString(), { padding: 1, borderStyle: 'round', borderColor: '#D67441' }));
|
|
||||||
// Display insights
|
|
||||||
if (result.insights.length > 0) {
|
|
||||||
const insightsText = result.insights.map(insight => `${this.colors.accent('◆')} ${insight.description} (${(insight.confidence * 100).toFixed(1)}% confidence)`).join('\n');
|
|
||||||
console.log(boxen(`${this.emojis.magic} ${this.colors.brain('NEURAL INSIGHTS')}\n\n` +
|
|
||||||
insightsText, { padding: 1, borderStyle: 'round', borderColor: '#E88B5A' }));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Helper methods for the neural system
|
|
||||||
*/
|
|
||||||
extractMainText(data) {
|
|
||||||
// Extract the most relevant text from a data object
|
|
||||||
const textFields = ['name', 'title', 'description', 'content', 'text', 'label'];
|
|
||||||
for (const field of textFields) {
|
|
||||||
if (data[field] && typeof data[field] === 'string') {
|
|
||||||
return data[field];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Fallback: concatenate all string values
|
|
||||||
return Object.values(data)
|
|
||||||
.filter(v => typeof v === 'string')
|
|
||||||
.join(' ')
|
|
||||||
.substring(0, 200); // Limit length
|
|
||||||
}
|
|
||||||
generateSmartId(data, nounType, index) {
|
|
||||||
const mainText = this.extractMainText(data);
|
|
||||||
const cleanText = mainText.toLowerCase().replace(/[^a-z0-9]/g, '_').substring(0, 20);
|
|
||||||
return `${nounType}_${cleanText}_${index}`;
|
|
||||||
}
|
|
||||||
extractRelationshipContext(source, target, allData) {
|
|
||||||
// Extract context for relationship detection
|
|
||||||
return [
|
|
||||||
this.extractMainText(source),
|
|
||||||
this.extractMainText(target),
|
|
||||||
// Add more contextual information
|
|
||||||
].join(' ');
|
|
||||||
}
|
|
||||||
calculateTypeCompatibility(sourceType, targetType, verbType) {
|
|
||||||
// Define type compatibility matrix for relationships
|
|
||||||
const compatibilityMatrix = {
|
|
||||||
[NounType.Person]: {
|
|
||||||
[NounType.Organization]: [VerbType.MemberOf, VerbType.WorksWith],
|
|
||||||
[NounType.Project]: [VerbType.WorksWith, VerbType.Creates],
|
|
||||||
[NounType.Person]: [VerbType.WorksWith, VerbType.Mentors, VerbType.ReportsTo]
|
|
||||||
}
|
|
||||||
// Add more compatibility rules
|
|
||||||
};
|
|
||||||
const sourceCompatibility = compatibilityMatrix[sourceType];
|
|
||||||
if (sourceCompatibility && sourceCompatibility[targetType]) {
|
|
||||||
return sourceCompatibility[targetType].includes(verbType) ? 1.0 : 0.3;
|
|
||||||
}
|
|
||||||
return 0.5; // Default compatibility
|
|
||||||
}
|
|
||||||
getVerbSpecificity(verbType) {
|
|
||||||
// More specific verbs get higher scores
|
|
||||||
const specificityScores = {
|
|
||||||
[VerbType.RelatedTo]: 0.1, // Very generic
|
|
||||||
[VerbType.WorksWith]: 0.7, // Specific
|
|
||||||
[VerbType.Mentors]: 0.9, // Very specific
|
|
||||||
[VerbType.ReportsTo]: 0.9, // Very specific
|
|
||||||
[VerbType.Supervises]: 0.9 // Very specific
|
|
||||||
};
|
|
||||||
return specificityScores[verbType] || 0.5;
|
|
||||||
}
|
|
||||||
getRelevantFields(data, nounType) {
|
|
||||||
// Implementation for finding relevant fields
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
getMatchedPatterns(text, data, nounType) {
|
|
||||||
// Implementation for finding matched patterns
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
pruneRelationships(relationships) {
|
|
||||||
// Remove duplicates and low-confidence relationships
|
|
||||||
return relationships
|
|
||||||
.sort((a, b) => b.confidence - a.confidence)
|
|
||||||
.slice(0, 1000); // Limit to top 1000 relationships
|
|
||||||
}
|
|
||||||
detectHierarchies(relationships) {
|
|
||||||
// Detect hierarchical structures
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
detectClusters(entities, relationships) {
|
|
||||||
// Detect entity clusters
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
detectPatterns(relationships) {
|
|
||||||
// Detect relationship patterns
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
summarizeEntities(entities) {
|
|
||||||
const summary = {};
|
|
||||||
entities.forEach(entity => {
|
|
||||||
if (!summary[entity.nounType]) {
|
|
||||||
summary[entity.nounType] = { count: 0, totalConfidence: 0 };
|
|
||||||
}
|
|
||||||
summary[entity.nounType].count++;
|
|
||||||
summary[entity.nounType].totalConfidence += entity.confidence;
|
|
||||||
});
|
|
||||||
Object.keys(summary).forEach(type => {
|
|
||||||
summary[type].avgConfidence = summary[type].totalConfidence / summary[type].count;
|
|
||||||
});
|
|
||||||
return summary;
|
|
||||||
}
|
|
||||||
summarizeRelationships(relationships) {
|
|
||||||
const summary = {};
|
|
||||||
relationships.forEach(rel => {
|
|
||||||
if (!summary[rel.verbType]) {
|
|
||||||
summary[rel.verbType] = { count: 0, totalWeight: 0, totalConfidence: 0 };
|
|
||||||
}
|
|
||||||
summary[rel.verbType].count++;
|
|
||||||
summary[rel.verbType].totalWeight += rel.weight;
|
|
||||||
summary[rel.verbType].totalConfidence += rel.confidence;
|
|
||||||
});
|
|
||||||
Object.keys(summary).forEach(type => {
|
|
||||||
const stats = summary[type];
|
|
||||||
stats.avgWeight = stats.totalWeight / stats.count;
|
|
||||||
stats.avgConfidence = stats.totalConfidence / stats.count;
|
|
||||||
});
|
|
||||||
return summary;
|
|
||||||
}
|
|
||||||
calculateOverallConfidence(entities, relationships) {
|
|
||||||
const entityConfidence = entities.reduce((sum, e) => sum + e.confidence, 0) / entities.length;
|
|
||||||
const relationshipConfidence = relationships.reduce((sum, r) => sum + r.confidence, 0) / relationships.length;
|
|
||||||
return (entityConfidence + relationshipConfidence) / 2;
|
|
||||||
}
|
|
||||||
async generatePreview(entities, relationships) {
|
|
||||||
return entities.slice(0, 5).map(entity => ({
|
|
||||||
id: entity.suggestedId,
|
|
||||||
nounType: entity.nounType,
|
|
||||||
data: entity.originalData,
|
|
||||||
relationships: relationships
|
|
||||||
.filter(r => r.sourceId === entity.suggestedId)
|
|
||||||
.slice(0, 3)
|
|
||||||
.map(r => ({
|
|
||||||
target: r.targetId,
|
|
||||||
verbType: r.verbType,
|
|
||||||
weight: r.weight,
|
|
||||||
confidence: r.confidence
|
|
||||||
}))
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
async confirmNeuralImport(result) {
|
|
||||||
const { confirm } = await prompts({
|
|
||||||
type: 'confirm',
|
|
||||||
name: 'confirm',
|
|
||||||
message: `${this.emojis.rocket} Execute neural import?`,
|
|
||||||
initial: true
|
|
||||||
});
|
|
||||||
return confirm;
|
|
||||||
}
|
|
||||||
async executeNeuralImport(result, options) {
|
|
||||||
const spinner = ora(`${this.emojis.gear} Executing neural import...`).start();
|
|
||||||
try {
|
|
||||||
// Add entities to Brainy
|
|
||||||
for (const entity of result.detectedEntities) {
|
|
||||||
await this.brainy.add(this.extractMainText(entity.originalData), {
|
|
||||||
...entity.originalData,
|
|
||||||
nounType: entity.nounType,
|
|
||||||
confidence: entity.confidence,
|
|
||||||
id: entity.suggestedId
|
|
||||||
});
|
|
||||||
}
|
|
||||||
// Add relationships to Brainy
|
|
||||||
for (const relationship of result.detectedRelationships) {
|
|
||||||
await this.brainy.addVerb(relationship.sourceId, relationship.targetId, relationship.verbType, {
|
|
||||||
weight: relationship.weight,
|
|
||||||
metadata: {
|
|
||||||
confidence: relationship.confidence,
|
|
||||||
context: relationship.context,
|
|
||||||
...relationship.metadata
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
spinner.succeed(this.colors.success(`${this.emojis.check} Neural import complete! ` +
|
|
||||||
`${result.detectedEntities.length} entities and ` +
|
|
||||||
`${result.detectedRelationships.length} relationships imported.`));
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
spinner.fail('Neural import failed');
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
async generateRelationshipReasoning(source, target, verbType, context) {
|
|
||||||
return `Neural analysis detected ${verbType} relationship based on semantic context`;
|
|
||||||
}
|
|
||||||
extractRelationshipMetadata(sourceData, targetData, verbType) {
|
|
||||||
return {
|
|
||||||
sourceType: typeof sourceData,
|
|
||||||
targetType: typeof targetData,
|
|
||||||
detectedBy: 'neural-import',
|
|
||||||
timestamp: new Date().toISOString()
|
|
||||||
};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
//# sourceMappingURL=neuralImport.js.map
|
|
||||||
1
dist/cortex/neuralImport.js.map
vendored
1
dist/cortex/neuralImport.js.map
vendored
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue