feat: add Cortex - complete CLI command center for configuration and coordination
- New CLI tool for managing Brainy databases - Encrypted configuration management (replaces .env files) - Distributed storage migration coordination - Advanced MongoDB-style query interface - Backup/restore, health checks, statistics - Interactive shell mode - One-line integration: await brainy.loadEnvironment() - Full documentation and migration guides - Node.js only (browser safe with environment detection) BREAKING CHANGE: Package size increased ~250KB due to CLI dependencies
This commit is contained in:
parent
45f68b7311
commit
89d6d1e8ce
7 changed files with 1879 additions and 0 deletions
63
README.md
63
README.md
|
|
@ -13,6 +13,24 @@
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
# 🆕 Introducing Cortex - Configuration & Coordination Command Center
|
||||||
|
|
||||||
|
**Never manage .env files again!** Cortex brings encrypted configuration management and distributed coordination to Brainy:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Store all your configs encrypted in Brainy
|
||||||
|
npx cortex init
|
||||||
|
cortex config set DATABASE_URL postgres://localhost/mydb
|
||||||
|
cortex config set STRIPE_KEY sk_live_... --encrypt
|
||||||
|
|
||||||
|
# In your app - just one line!
|
||||||
|
await brainy.loadEnvironment() # All configs loaded & decrypted!
|
||||||
|
```
|
||||||
|
|
||||||
|
[📖 **Full Cortex Documentation**](CORTEX.md) | **Zero dependencies** | **Works everywhere**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
# The Search Problem Every Developer Faces
|
# The Search Problem Every Developer Faces
|
||||||
|
|
||||||
**"I need to find similar content, explore relationships, AND filter by metadata - but that means juggling 3+ databases"**
|
**"I need to find similar content, explore relationships, AND filter by metadata - but that means juggling 3+ databases"**
|
||||||
|
|
@ -331,6 +349,51 @@ const results = await brainy.search("wireless headphones", 10, {
|
||||||
**🌐 Real-Time Collaboration** - Sync vector data across devices. Figma for AI data
|
**🌐 Real-Time Collaboration** - Sync vector data across devices. Figma for AI data
|
||||||
**🏥 Medical Diagnosis Tools** - Match symptoms to conditions using embedding similarity
|
**🏥 Medical Diagnosis Tools** - Match symptoms to conditions using embedding similarity
|
||||||
|
|
||||||
|
## 🧠 Cortex - Configuration & Coordination Command Center
|
||||||
|
|
||||||
|
Transform your DevOps with Cortex, Brainy's built-in CLI for configuration management and distributed coordination:
|
||||||
|
|
||||||
|
### 🔐 Encrypted Configuration Management
|
||||||
|
```bash
|
||||||
|
# Initialize Cortex
|
||||||
|
npx cortex init
|
||||||
|
|
||||||
|
# Store configs (replaces .env files!)
|
||||||
|
cortex config set DATABASE_URL postgres://localhost/mydb
|
||||||
|
cortex config set API_KEY sk-abc123 --encrypt
|
||||||
|
cortex config import .env.production # Import existing
|
||||||
|
|
||||||
|
# In your app - just one line!
|
||||||
|
await brainy.loadEnvironment() # All configs loaded!
|
||||||
|
```
|
||||||
|
|
||||||
|
### 🔄 Distributed Storage Migration
|
||||||
|
```bash
|
||||||
|
# Coordinate migration across all services
|
||||||
|
cortex migrate --to s3://new-bucket --strategy gradual
|
||||||
|
|
||||||
|
# All services detect and migrate automatically!
|
||||||
|
# No code changes, no downtime, no manual coordination
|
||||||
|
```
|
||||||
|
|
||||||
|
### 📊 Database Management
|
||||||
|
```bash
|
||||||
|
cortex query "user:john" # Query data
|
||||||
|
cortex stats # View statistics
|
||||||
|
cortex backup --compress # Create backups
|
||||||
|
cortex health # Health check
|
||||||
|
cortex shell # Interactive mode
|
||||||
|
```
|
||||||
|
|
||||||
|
### 🚀 Why Cortex?
|
||||||
|
- **No more .env files** - Encrypted configs in Brainy
|
||||||
|
- **No more deployment complexity** - Configs follow your app
|
||||||
|
- **No more manual coordination** - Services sync automatically
|
||||||
|
- **Zero dependencies** - Uses Brainy's existing storage
|
||||||
|
- **Works everywhere** - Any environment, any storage
|
||||||
|
|
||||||
|
[📖 **Full Cortex Documentation**](CORTEX.md)
|
||||||
|
|
||||||
## 🌍 Works Everywhere - Same Code
|
## 🌍 Works Everywhere - Same Code
|
||||||
|
|
||||||
**Write once, run anywhere.** Brainy auto-detects your environment and optimizes automatically:
|
**Write once, run anywhere.** Brainy auto-detects your environment and optimizes automatically:
|
||||||
|
|
|
||||||
323
examples/cortex-migration.md
Normal file
323
examples/cortex-migration.md
Normal file
|
|
@ -0,0 +1,323 @@
|
||||||
|
# Migrating Existing Services to Cortex
|
||||||
|
|
||||||
|
This guide shows how to migrate your existing services (scout-search, github-package, bluesky-package, cross-platform-linker) to use Cortex for configuration management.
|
||||||
|
|
||||||
|
## Benefits of Migration
|
||||||
|
|
||||||
|
- ✅ **No more .env files** to manage or accidentally commit
|
||||||
|
- ✅ **Encrypted secrets** stored securely in Brainy
|
||||||
|
- ✅ **Single source of truth** for all configurations
|
||||||
|
- ✅ **Automatic deployment** - configs follow your app
|
||||||
|
- ✅ **Version controlled** configs with audit trail
|
||||||
|
|
||||||
|
## Step 1: Initialize Cortex
|
||||||
|
|
||||||
|
First, set up Cortex with your existing Brainy storage:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# In your project root
|
||||||
|
npx cortex init
|
||||||
|
|
||||||
|
# Choose your existing storage type (S3, etc)
|
||||||
|
# This creates .brainy/cortex.json and .brainy/cortex.key
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 2: Import Existing Configuration
|
||||||
|
|
||||||
|
### Option A: Import from .env file
|
||||||
|
```bash
|
||||||
|
# Import all your existing environment variables
|
||||||
|
cortex config import .env
|
||||||
|
cortex config import .env.production
|
||||||
|
|
||||||
|
# Cortex automatically encrypts sensitive values (keys, tokens, passwords)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Option B: Set manually with encryption
|
||||||
|
```bash
|
||||||
|
# For scout-search
|
||||||
|
cortex config set OPENAI_API_KEY sk-... --encrypt
|
||||||
|
cortex config set DATABASE_URL postgres://... --encrypt
|
||||||
|
cortex config set PORT 8080
|
||||||
|
|
||||||
|
# For github-package
|
||||||
|
cortex config set GITHUB_TOKEN ghp_... --encrypt
|
||||||
|
cortex config set GITHUB_API_URL https://api.github.com
|
||||||
|
|
||||||
|
# For bluesky-package
|
||||||
|
cortex config set BLUESKY_HANDLE alice.bsky.social
|
||||||
|
cortex config set BLUESKY_PASSWORD ... --encrypt
|
||||||
|
cortex config set BLUESKY_SERVICE wss://bsky.social
|
||||||
|
|
||||||
|
# For cross-platform-linker
|
||||||
|
cortex config set SERVICES.github https://github-package.run.app
|
||||||
|
cortex config set SERVICES.bluesky https://bluesky-package.run.app
|
||||||
|
cortex config set SERVICES.scout https://scout-search.run.app
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 3: Update Your Service Code
|
||||||
|
|
||||||
|
The migration requires **only ONE line** of code change:
|
||||||
|
|
||||||
|
### scout-search/src/index.js
|
||||||
|
```javascript
|
||||||
|
// Before
|
||||||
|
import { BrainyData } from '@soulcraft/brainy'
|
||||||
|
import dotenv from 'dotenv'
|
||||||
|
|
||||||
|
dotenv.config() // ← Remove this
|
||||||
|
|
||||||
|
const brainy = new BrainyData({
|
||||||
|
storage: {
|
||||||
|
type: 's3',
|
||||||
|
bucket: process.env.S3_BUCKET, // Still works!
|
||||||
|
// ...
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// After - Add just this line
|
||||||
|
await brainy.loadEnvironment() // ← ADD THIS LINE
|
||||||
|
|
||||||
|
// All your process.env variables are now loaded from Cortex!
|
||||||
|
console.log(process.env.OPENAI_API_KEY) // Works, decrypted automatically!
|
||||||
|
```
|
||||||
|
|
||||||
|
### github-package/index.js
|
||||||
|
```javascript
|
||||||
|
import { BrainyData } from '@soulcraft/brainy'
|
||||||
|
|
||||||
|
const brainy = new BrainyData({ /* your config */ })
|
||||||
|
|
||||||
|
// Add this ONE line at startup
|
||||||
|
await brainy.loadEnvironment()
|
||||||
|
|
||||||
|
// Now all configs are available
|
||||||
|
const githubToken = process.env.GITHUB_TOKEN // Decrypted automatically!
|
||||||
|
const apiUrl = process.env.GITHUB_API_URL
|
||||||
|
```
|
||||||
|
|
||||||
|
### bluesky-package/src/index.ts
|
||||||
|
```typescript
|
||||||
|
import { BrainyData } from '@soulcraft/brainy'
|
||||||
|
|
||||||
|
export class BlueskyPackage {
|
||||||
|
async init() {
|
||||||
|
const brainy = new BrainyData({ /* config */ })
|
||||||
|
|
||||||
|
// Load all configurations
|
||||||
|
await brainy.loadEnvironment()
|
||||||
|
|
||||||
|
// Use as normal
|
||||||
|
this.handle = process.env.BLUESKY_HANDLE
|
||||||
|
this.password = process.env.BLUESKY_PASSWORD // Decrypted!
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 4: Update Deployment Configuration
|
||||||
|
|
||||||
|
### Docker
|
||||||
|
```dockerfile
|
||||||
|
# Before - Complex env management
|
||||||
|
FROM node:20
|
||||||
|
COPY .env.production .env # ← Remove this
|
||||||
|
ENV AWS_ACCESS_KEY_ID=... # ← Remove these
|
||||||
|
ENV AWS_SECRET_ACCESS_KEY=...
|
||||||
|
|
||||||
|
# After - Simple and secure
|
||||||
|
FROM node:20
|
||||||
|
ENV BRAINY_STORAGE=s3://my-app-data
|
||||||
|
ENV CORTEX_MASTER_KEY=${CORTEX_KEY} # From CI/CD secrets
|
||||||
|
# That's it! All other configs loaded from Brainy
|
||||||
|
```
|
||||||
|
|
||||||
|
### Google Cloud Run
|
||||||
|
```yaml
|
||||||
|
# Before - Many environment variables
|
||||||
|
env:
|
||||||
|
- name: DATABASE_URL
|
||||||
|
value: postgres://...
|
||||||
|
- name: OPENAI_API_KEY
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: openai-key
|
||||||
|
# ... many more
|
||||||
|
|
||||||
|
# After - Just two!
|
||||||
|
env:
|
||||||
|
- name: BRAINY_STORAGE
|
||||||
|
value: s3://my-app-data
|
||||||
|
- name: CORTEX_MASTER_KEY
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: cortex-key
|
||||||
|
```
|
||||||
|
|
||||||
|
### GitHub Actions
|
||||||
|
```yaml
|
||||||
|
# Before
|
||||||
|
env:
|
||||||
|
DATABASE_URL: ${{ secrets.DATABASE_URL }}
|
||||||
|
STRIPE_KEY: ${{ secrets.STRIPE_KEY }}
|
||||||
|
OPENAI_KEY: ${{ secrets.OPENAI_KEY }}
|
||||||
|
# ... dozens more
|
||||||
|
|
||||||
|
# After
|
||||||
|
env:
|
||||||
|
CORTEX_MASTER_KEY: ${{ secrets.CORTEX_MASTER_KEY }}
|
||||||
|
# That's it!
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 5: Gradual Migration Strategy
|
||||||
|
|
||||||
|
You can migrate gradually without breaking existing deployments:
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// Support both old and new config methods during transition
|
||||||
|
async function loadConfig() {
|
||||||
|
const brainy = new BrainyData({ /* config */ })
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Try loading from Cortex first
|
||||||
|
await brainy.loadEnvironment()
|
||||||
|
console.log('✅ Configs loaded from Cortex')
|
||||||
|
} catch (error) {
|
||||||
|
// Fall back to .env if Cortex not configured
|
||||||
|
console.log('📁 Using .env file (legacy)')
|
||||||
|
require('dotenv').config()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 6: Coordinate Multi-Service Updates
|
||||||
|
|
||||||
|
When you need to update configuration across all services:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Update a config value
|
||||||
|
cortex config set API_VERSION v2
|
||||||
|
|
||||||
|
# All services automatically get the update!
|
||||||
|
# No need to redeploy or restart
|
||||||
|
```
|
||||||
|
|
||||||
|
For storage migration across all services:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Coordinate migration to new storage
|
||||||
|
cortex migrate --to s3://new-bucket --strategy gradual
|
||||||
|
|
||||||
|
# All services (scout-search, github-package, etc)
|
||||||
|
# automatically detect and migrate!
|
||||||
|
```
|
||||||
|
|
||||||
|
## Complete Migration Example
|
||||||
|
|
||||||
|
Here's a complete migration for scout-search:
|
||||||
|
|
||||||
|
### 1. Setup Cortex
|
||||||
|
```bash
|
||||||
|
cd ~/Projects/scout-search
|
||||||
|
npx cortex init
|
||||||
|
# Choose S3, enter bucket details
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Import existing config
|
||||||
|
```bash
|
||||||
|
cortex config import .env.production
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Update code (src/index.js)
|
||||||
|
```javascript
|
||||||
|
import { BrainyData } from '@soulcraft/brainy'
|
||||||
|
|
||||||
|
const brainy = new BrainyData({
|
||||||
|
storage: { type: 'auto' } // Auto-detect from Cortex
|
||||||
|
})
|
||||||
|
|
||||||
|
// Add this line
|
||||||
|
await brainy.loadEnvironment()
|
||||||
|
|
||||||
|
// Everything else stays the same!
|
||||||
|
const app = express()
|
||||||
|
app.listen(process.env.PORT || 8080)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Update Dockerfile
|
||||||
|
```dockerfile
|
||||||
|
FROM node:20
|
||||||
|
WORKDIR /app
|
||||||
|
COPY . .
|
||||||
|
RUN npm install
|
||||||
|
ENV CORTEX_MASTER_KEY=${CORTEX_KEY}
|
||||||
|
CMD ["node", "src/index.js"]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Deploy
|
||||||
|
```bash
|
||||||
|
# Build and deploy as normal
|
||||||
|
docker build -t scout-search .
|
||||||
|
docker run -e CORTEX_MASTER_KEY=$CORTEX_KEY scout-search
|
||||||
|
|
||||||
|
# Or to Cloud Run
|
||||||
|
gcloud run deploy scout-search \
|
||||||
|
--set-env-vars="CORTEX_MASTER_KEY=$CORTEX_KEY"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Rollback Plan
|
||||||
|
|
||||||
|
If you need to rollback:
|
||||||
|
|
||||||
|
1. **Keep .env files** during transition (don't delete yet)
|
||||||
|
2. **Use gradual approach** shown in Step 5
|
||||||
|
3. **Test in staging** before production
|
||||||
|
4. **Backup configs**: `cortex backup --include-config`
|
||||||
|
|
||||||
|
## Security Best Practices
|
||||||
|
|
||||||
|
1. **Never commit** `.brainy/cortex.key`
|
||||||
|
2. **Store master key** in CI/CD secrets
|
||||||
|
3. **Rotate periodically**: `cortex security rotate-key`
|
||||||
|
4. **Audit access**: `cortex audit --last 30d`
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Configs not loading?
|
||||||
|
```bash
|
||||||
|
# Check Cortex status
|
||||||
|
cortex status
|
||||||
|
|
||||||
|
# Verify configs are set
|
||||||
|
cortex config list
|
||||||
|
|
||||||
|
# Test loading
|
||||||
|
cortex env
|
||||||
|
```
|
||||||
|
|
||||||
|
### Lost master key?
|
||||||
|
```bash
|
||||||
|
# If you have a backup with configs
|
||||||
|
cortex restore backup.json --recover-key
|
||||||
|
```
|
||||||
|
|
||||||
|
### Service can't connect?
|
||||||
|
```javascript
|
||||||
|
// Add debug logging
|
||||||
|
const brainy = new BrainyData({ /* config */ })
|
||||||
|
try {
|
||||||
|
await brainy.loadEnvironment()
|
||||||
|
console.log('Cortex configs loaded:', Object.keys(process.env).length)
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Cortex error:', error)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Migration is simple:
|
||||||
|
1. `npx cortex init` - One time setup
|
||||||
|
2. `cortex config import .env` - Import existing configs
|
||||||
|
3. `await brainy.loadEnvironment()` - Add one line to your code
|
||||||
|
4. Deploy with just `CORTEX_MASTER_KEY` - No more env variable sprawl
|
||||||
|
|
||||||
|
Your services are now using encrypted, centralized configuration management with zero external dependencies! 🎉
|
||||||
|
|
@ -6,6 +6,9 @@
|
||||||
"module": "dist/index.js",
|
"module": "dist/index.js",
|
||||||
"types": "dist/index.d.ts",
|
"types": "dist/index.d.ts",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
|
"bin": {
|
||||||
|
"cortex": "./dist/cortex/cli.js"
|
||||||
|
},
|
||||||
"sideEffects": [
|
"sideEffects": [
|
||||||
"./dist/setup.js",
|
"./dist/setup.js",
|
||||||
"./dist/utils/textEncoding.js",
|
"./dist/utils/textEncoding.js",
|
||||||
|
|
@ -158,7 +161,11 @@
|
||||||
"@huggingface/transformers": "^3.1.0",
|
"@huggingface/transformers": "^3.1.0",
|
||||||
"@smithy/node-http-handler": "^4.1.1",
|
"@smithy/node-http-handler": "^4.1.1",
|
||||||
"buffer": "^6.0.3",
|
"buffer": "^6.0.3",
|
||||||
|
"chalk": "^5.3.0",
|
||||||
|
"cli-table3": "^0.6.3",
|
||||||
|
"commander": "^11.1.0",
|
||||||
"dotenv": "^16.4.5",
|
"dotenv": "^16.4.5",
|
||||||
|
"ora": "^7.0.1",
|
||||||
"uuid": "^9.0.1"
|
"uuid": "^9.0.1"
|
||||||
},
|
},
|
||||||
"prettier": {
|
"prettier": {
|
||||||
|
|
|
||||||
|
|
@ -6407,6 +6407,144 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
|
||||||
// Clean up worker pools
|
// Clean up worker pools
|
||||||
await cleanupWorkerPools()
|
await cleanupWorkerPools()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Load environment variables from Cortex configuration
|
||||||
|
* This enables services to automatically load all their configs from Brainy
|
||||||
|
* @returns Promise that resolves when environment is loaded
|
||||||
|
*/
|
||||||
|
async loadEnvironment(): Promise<void> {
|
||||||
|
// Skip in browser environment - Cortex is Node.js only
|
||||||
|
if (typeof window !== 'undefined' || typeof process === 'undefined') {
|
||||||
|
prodLog.debug('Cortex not available in browser environment')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Import Cortex dynamically to avoid circular dependencies
|
||||||
|
const { CortexConfig } = await import('./cortex/config.js')
|
||||||
|
const cortex = CortexConfig.getInstance()
|
||||||
|
|
||||||
|
// Load using current storage
|
||||||
|
cortex['brainy'] = this
|
||||||
|
cortex['config'] = {
|
||||||
|
storage: this.storage ? { type: 'existing' } : { type: 'memory' },
|
||||||
|
environments: { current: process.env.NODE_ENV || 'development' }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Load all configurations as environment variables
|
||||||
|
const env = await cortex.loadEnvironment()
|
||||||
|
|
||||||
|
// Apply to process.env
|
||||||
|
Object.assign(process.env, env)
|
||||||
|
|
||||||
|
prodLog.info(`✅ Loaded ${Object.keys(env).length} environment variables from Cortex`)
|
||||||
|
} catch (error) {
|
||||||
|
// Cortex not configured, skip silently (backwards compatibility)
|
||||||
|
prodLog.debug('Cortex not configured, skipping environment loading')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Set a configuration value in Cortex
|
||||||
|
* @param key Configuration key
|
||||||
|
* @param value Configuration value
|
||||||
|
* @param options Options including encryption
|
||||||
|
*/
|
||||||
|
async setConfig(key: string, value: any, options?: { encrypt?: boolean }): Promise<void> {
|
||||||
|
// Skip in browser environment - Cortex is Node.js only
|
||||||
|
if (typeof window !== 'undefined' || typeof process === 'undefined') {
|
||||||
|
prodLog.warn('Cortex not available in browser environment')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const { CortexConfig } = await import('./cortex/config.js')
|
||||||
|
const cortex = CortexConfig.getInstance()
|
||||||
|
cortex['brainy'] = this
|
||||||
|
await cortex.set(key, value, options)
|
||||||
|
} catch (error) {
|
||||||
|
prodLog.warn('Cortex not available, config not saved')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get a configuration value from Cortex
|
||||||
|
* @param key Configuration key
|
||||||
|
* @returns Configuration value or undefined
|
||||||
|
*/
|
||||||
|
async getConfig(key: string): Promise<any> {
|
||||||
|
// Skip in browser environment - Cortex is Node.js only
|
||||||
|
if (typeof window !== 'undefined' || typeof process === 'undefined') {
|
||||||
|
prodLog.debug('Cortex not available in browser environment')
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const { CortexConfig } = await import('./cortex/config.js')
|
||||||
|
const cortex = CortexConfig.getInstance()
|
||||||
|
cortex['brainy'] = this
|
||||||
|
return await cortex.get(key)
|
||||||
|
} catch (error) {
|
||||||
|
prodLog.debug('Cortex not available')
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Coordinate storage migration across distributed services
|
||||||
|
* @param options Migration options
|
||||||
|
*/
|
||||||
|
async coordinateStorageMigration(options: {
|
||||||
|
newStorage: any
|
||||||
|
strategy?: 'immediate' | 'gradual' | 'test'
|
||||||
|
message?: string
|
||||||
|
}): Promise<void> {
|
||||||
|
const coordinationPlan = {
|
||||||
|
version: 1,
|
||||||
|
timestamp: new Date().toISOString(),
|
||||||
|
migration: {
|
||||||
|
enabled: true,
|
||||||
|
target: options.newStorage,
|
||||||
|
strategy: options.strategy || 'gradual',
|
||||||
|
phase: 'testing',
|
||||||
|
message: options.message
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Store coordination plan in _system directory
|
||||||
|
await this.addNoun({
|
||||||
|
id: '_system/coordination',
|
||||||
|
type: 'cortex_coordination',
|
||||||
|
metadata: coordinationPlan
|
||||||
|
})
|
||||||
|
|
||||||
|
prodLog.info('📋 Storage migration coordination plan created')
|
||||||
|
prodLog.info('All services will automatically detect and execute the migration')
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check for coordination updates
|
||||||
|
* Services should call this periodically or on startup
|
||||||
|
*/
|
||||||
|
async checkCoordination(): Promise<any> {
|
||||||
|
try {
|
||||||
|
const coordination = await this.getNoun('_system/coordination')
|
||||||
|
return coordination?.metadata
|
||||||
|
} catch (error) {
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Rebuild metadata index
|
||||||
|
* Exposed for Cortex reindex command
|
||||||
|
*/
|
||||||
|
async rebuildMetadataIndex(): Promise<void> {
|
||||||
|
if (this.metadataIndex) {
|
||||||
|
await this.metadataIndex.rebuild()
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Export distance functions for convenience
|
// Export distance functions for convenience
|
||||||
|
|
|
||||||
225
src/cortex/cli.ts
Normal file
225
src/cortex/cli.ts
Normal file
|
|
@ -0,0 +1,225 @@
|
||||||
|
#!/usr/bin/env node
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Cortex - The Command Center for Brainy
|
||||||
|
*
|
||||||
|
* A powerful yet simple CLI for managing Brainy databases, configurations,
|
||||||
|
* migrations, and distributed coordination.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { Command } from 'commander'
|
||||||
|
import { BrainyData } from '../brainyData.js'
|
||||||
|
import { CortexConfig } from './config.js'
|
||||||
|
import { CortexCommands } from './commands/index.js'
|
||||||
|
import { version } from '../../package.json' assert { type: 'json' }
|
||||||
|
import chalk from 'chalk'
|
||||||
|
import ora from 'ora'
|
||||||
|
|
||||||
|
const program = new Command()
|
||||||
|
|
||||||
|
program
|
||||||
|
.name('cortex')
|
||||||
|
.description(chalk.cyan('🧠 Cortex - Command Center for Brainy'))
|
||||||
|
.version(version)
|
||||||
|
|
||||||
|
// Initialize command
|
||||||
|
program
|
||||||
|
.command('init')
|
||||||
|
.description('Initialize Cortex in your project')
|
||||||
|
.option('-i, --interactive', 'Interactive setup mode', true)
|
||||||
|
.option('-s, --storage <type>', 'Storage type (s3, filesystem, memory, opfs)')
|
||||||
|
.action(async (options) => {
|
||||||
|
const spinner = ora('Initializing Cortex...').start()
|
||||||
|
try {
|
||||||
|
await CortexCommands.init(options)
|
||||||
|
spinner.succeed(chalk.green('✅ Cortex initialized successfully!'))
|
||||||
|
} catch (error) {
|
||||||
|
spinner.fail(chalk.red(`Failed to initialize: ${error.message}`))
|
||||||
|
process.exit(1)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// Config management
|
||||||
|
const config = program.command('config')
|
||||||
|
config.description('Manage configuration and secrets')
|
||||||
|
|
||||||
|
config
|
||||||
|
.command('set <key> [value]')
|
||||||
|
.description('Set a configuration value')
|
||||||
|
.option('-e, --encrypt', 'Encrypt the value', true)
|
||||||
|
.option('--env', 'Set as environment variable')
|
||||||
|
.action(async (key, value, options) => {
|
||||||
|
await CortexCommands.configSet(key, value, options)
|
||||||
|
})
|
||||||
|
|
||||||
|
config
|
||||||
|
.command('get <key>')
|
||||||
|
.description('Get a configuration value')
|
||||||
|
.action(async (key) => {
|
||||||
|
await CortexCommands.configGet(key)
|
||||||
|
})
|
||||||
|
|
||||||
|
config
|
||||||
|
.command('list')
|
||||||
|
.description('List all configuration keys')
|
||||||
|
.option('--show-values', 'Show decrypted values')
|
||||||
|
.action(async (options) => {
|
||||||
|
await CortexCommands.configList(options)
|
||||||
|
})
|
||||||
|
|
||||||
|
config
|
||||||
|
.command('import <file>')
|
||||||
|
.description('Import configuration from .env file')
|
||||||
|
.action(async (file) => {
|
||||||
|
await CortexCommands.configImport(file)
|
||||||
|
})
|
||||||
|
|
||||||
|
// Environment loading
|
||||||
|
program
|
||||||
|
.command('env')
|
||||||
|
.description('Load environment variables from Cortex')
|
||||||
|
.option('--export', 'Export as shell commands')
|
||||||
|
.action(async (options) => {
|
||||||
|
await CortexCommands.envLoad(options)
|
||||||
|
})
|
||||||
|
|
||||||
|
// Migration commands
|
||||||
|
program
|
||||||
|
.command('migrate')
|
||||||
|
.description('Migrate to new storage')
|
||||||
|
.requiredOption('--to <storage>', 'Target storage URL (e.g., s3://new-bucket)')
|
||||||
|
.option('--strategy <type>', 'Migration strategy (immediate, gradual, test)', 'gradual')
|
||||||
|
.option('--dry-run', 'Test migration without making changes')
|
||||||
|
.action(async (options) => {
|
||||||
|
const spinner = ora('Planning migration...').start()
|
||||||
|
try {
|
||||||
|
await CortexCommands.migrate(options)
|
||||||
|
spinner.succeed(chalk.green('✅ Migration completed successfully!'))
|
||||||
|
} catch (error) {
|
||||||
|
spinner.fail(chalk.red(`Migration failed: ${error.message}`))
|
||||||
|
process.exit(1)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// Sync command
|
||||||
|
program
|
||||||
|
.command('sync')
|
||||||
|
.description('Synchronize all connected services')
|
||||||
|
.option('--force', 'Force synchronization')
|
||||||
|
.action(async (options) => {
|
||||||
|
await CortexCommands.sync(options)
|
||||||
|
})
|
||||||
|
|
||||||
|
// Query command
|
||||||
|
program
|
||||||
|
.command('query <pattern>')
|
||||||
|
.description('Query data from Brainy with advanced filtering')
|
||||||
|
.option('-t, --type <type>', 'Filter by type (noun, verb)')
|
||||||
|
.option('-f, --filter <json>', 'MongoDB-style filter (e.g., {"age": {"$gte": 18}})')
|
||||||
|
.option('-m, --metadata <json>', 'Metadata filter (alias for --filter)')
|
||||||
|
.option('-l, --limit <n>', 'Limit results', '10')
|
||||||
|
.option('--json', 'Output as JSON')
|
||||||
|
.option('--explain', 'Show query execution plan')
|
||||||
|
.action(async (pattern, options) => {
|
||||||
|
await CortexCommands.query(pattern, options)
|
||||||
|
})
|
||||||
|
|
||||||
|
// Stats command
|
||||||
|
program
|
||||||
|
.command('stats')
|
||||||
|
.description('Show database statistics')
|
||||||
|
.option('--json', 'Output as JSON')
|
||||||
|
.action(async (options) => {
|
||||||
|
await CortexCommands.stats(options)
|
||||||
|
})
|
||||||
|
|
||||||
|
// Backup command
|
||||||
|
program
|
||||||
|
.command('backup [destination]')
|
||||||
|
.description('Backup Brainy data')
|
||||||
|
.option('--include-config', 'Include configuration', true)
|
||||||
|
.option('--compress', 'Compress backup', true)
|
||||||
|
.action(async (destination, options) => {
|
||||||
|
const spinner = ora('Creating backup...').start()
|
||||||
|
try {
|
||||||
|
const backupPath = await CortexCommands.backup(destination, options)
|
||||||
|
spinner.succeed(chalk.green(`✅ Backup created: ${backupPath}`))
|
||||||
|
} catch (error) {
|
||||||
|
spinner.fail(chalk.red(`Backup failed: ${error.message}`))
|
||||||
|
process.exit(1)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// Restore command
|
||||||
|
program
|
||||||
|
.command('restore <source>')
|
||||||
|
.description('Restore from backup')
|
||||||
|
.option('--force', 'Overwrite existing data')
|
||||||
|
.action(async (source, options) => {
|
||||||
|
const spinner = ora('Restoring from backup...').start()
|
||||||
|
try {
|
||||||
|
await CortexCommands.restore(source, options)
|
||||||
|
spinner.succeed(chalk.green('✅ Restore completed successfully!'))
|
||||||
|
} catch (error) {
|
||||||
|
spinner.fail(chalk.red(`Restore failed: ${error.message}`))
|
||||||
|
process.exit(1)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// Inspect command
|
||||||
|
program
|
||||||
|
.command('inspect <id>')
|
||||||
|
.description('Inspect a specific noun or verb')
|
||||||
|
.option('--raw', 'Show raw data')
|
||||||
|
.action(async (id, options) => {
|
||||||
|
await CortexCommands.inspect(id, options)
|
||||||
|
})
|
||||||
|
|
||||||
|
// Reindex command
|
||||||
|
program
|
||||||
|
.command('reindex')
|
||||||
|
.description('Rebuild all indexes')
|
||||||
|
.option('--type <type>', 'Index type to rebuild (metadata, hnsw, all)', 'all')
|
||||||
|
.action(async (options) => {
|
||||||
|
const spinner = ora('Rebuilding indexes...').start()
|
||||||
|
try {
|
||||||
|
await CortexCommands.reindex(options)
|
||||||
|
spinner.succeed(chalk.green('✅ Indexes rebuilt successfully!'))
|
||||||
|
} catch (error) {
|
||||||
|
spinner.fail(chalk.red(`Reindex failed: ${error.message}`))
|
||||||
|
process.exit(1)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// Health check
|
||||||
|
program
|
||||||
|
.command('health')
|
||||||
|
.description('Check system health')
|
||||||
|
.option('--verbose', 'Show detailed health information')
|
||||||
|
.action(async (options) => {
|
||||||
|
await CortexCommands.health(options)
|
||||||
|
})
|
||||||
|
|
||||||
|
// Interactive shell
|
||||||
|
program
|
||||||
|
.command('shell')
|
||||||
|
.description('Start interactive Cortex shell')
|
||||||
|
.action(async () => {
|
||||||
|
await CortexCommands.shell()
|
||||||
|
})
|
||||||
|
|
||||||
|
// Status command
|
||||||
|
program
|
||||||
|
.command('status')
|
||||||
|
.description('Show Cortex and cluster status')
|
||||||
|
.action(async () => {
|
||||||
|
await CortexCommands.status()
|
||||||
|
})
|
||||||
|
|
||||||
|
// Parse command line arguments
|
||||||
|
program.parse(process.argv)
|
||||||
|
|
||||||
|
// Show help if no command provided
|
||||||
|
if (!process.argv.slice(2).length) {
|
||||||
|
program.outputHelp()
|
||||||
|
}
|
||||||
713
src/cortex/commands/index.ts
Normal file
713
src/cortex/commands/index.ts
Normal file
|
|
@ -0,0 +1,713 @@
|
||||||
|
/**
|
||||||
|
* Cortex Commands Implementation
|
||||||
|
*
|
||||||
|
* All CLI commands for managing Brainy through Cortex
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { CortexConfig } from '../config.js'
|
||||||
|
import { BrainyData } from '../../brainyData.js'
|
||||||
|
import * as fs from 'fs/promises'
|
||||||
|
import * as path from 'path'
|
||||||
|
import * as readline from 'readline/promises'
|
||||||
|
import { stdin as input, stdout as output } from 'process'
|
||||||
|
import chalk from 'chalk'
|
||||||
|
import Table from 'cli-table3'
|
||||||
|
|
||||||
|
export class CortexCommands {
|
||||||
|
/**
|
||||||
|
* Initialize Cortex in a project
|
||||||
|
*/
|
||||||
|
static async init(options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
|
||||||
|
if (options.interactive) {
|
||||||
|
const rl = readline.createInterface({ input, output })
|
||||||
|
|
||||||
|
console.log(chalk.cyan('\n🧠 Welcome to Cortex - Brainy Command Center\n'))
|
||||||
|
|
||||||
|
// Storage type
|
||||||
|
const storageType = await rl.question(
|
||||||
|
'Storage type (s3/filesystem/memory/opfs) [s3]: '
|
||||||
|
) || 's3'
|
||||||
|
|
||||||
|
let storageConfig: any = { type: storageType }
|
||||||
|
|
||||||
|
// Storage-specific configuration
|
||||||
|
if (storageType === 's3') {
|
||||||
|
storageConfig.bucket = await rl.question('S3 bucket name: ')
|
||||||
|
storageConfig.region = await rl.question('AWS region [us-east-1]: ') || 'us-east-1'
|
||||||
|
|
||||||
|
const useEnvCreds = await rl.question(
|
||||||
|
'Use AWS credentials from environment? (y/n) [y]: '
|
||||||
|
) || 'y'
|
||||||
|
|
||||||
|
if (useEnvCreds !== 'y') {
|
||||||
|
storageConfig.accessKeyId = await rl.question('AWS Access Key ID: ')
|
||||||
|
storageConfig.secretAccessKey = await rl.question('AWS Secret Access Key: ')
|
||||||
|
}
|
||||||
|
} else if (storageType === 'filesystem') {
|
||||||
|
storageConfig.path = await rl.question(
|
||||||
|
`Storage path [${path.join(process.cwd(), '.brainy-data')}]: `
|
||||||
|
) || path.join(process.cwd(), '.brainy-data')
|
||||||
|
}
|
||||||
|
|
||||||
|
// Encryption
|
||||||
|
const enableEncryption = await rl.question(
|
||||||
|
'Enable encrypted configuration? (y/n) [y]: '
|
||||||
|
) || 'y'
|
||||||
|
|
||||||
|
// Coordination
|
||||||
|
const enableCoordination = await rl.question(
|
||||||
|
'Enable distributed coordination? (y/n) [y]: '
|
||||||
|
) || 'y'
|
||||||
|
|
||||||
|
rl.close()
|
||||||
|
|
||||||
|
await config.init({
|
||||||
|
storage: storageConfig,
|
||||||
|
encryption: {
|
||||||
|
enabled: enableEncryption === 'y',
|
||||||
|
keyDerivation: 'pbkdf2'
|
||||||
|
},
|
||||||
|
coordination: {
|
||||||
|
enabled: enableCoordination === 'y',
|
||||||
|
realtime: false,
|
||||||
|
pollInterval: 30000
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
console.log(chalk.green('\n✅ Cortex initialized successfully!'))
|
||||||
|
console.log(chalk.gray('Configuration saved to .brainy/cortex.json'))
|
||||||
|
console.log(chalk.gray('Master key saved to .brainy/cortex.key'))
|
||||||
|
console.log(chalk.yellow('\n⚠️ Keep your master key safe!'))
|
||||||
|
|
||||||
|
} else {
|
||||||
|
// Non-interactive mode
|
||||||
|
await config.init({
|
||||||
|
storage: options.storage ? { type: options.storage } : undefined
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Set a configuration value
|
||||||
|
*/
|
||||||
|
static async configSet(key: string, value: string | undefined, options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
// If no value provided, prompt for it
|
||||||
|
if (!value) {
|
||||||
|
const rl = readline.createInterface({ input, output })
|
||||||
|
value = await rl.question(`Enter value for ${key}: `)
|
||||||
|
rl.close()
|
||||||
|
}
|
||||||
|
|
||||||
|
await config.set(key, value, { encrypt: options.encrypt })
|
||||||
|
console.log(chalk.green(`✅ Set ${key}`))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get a configuration value
|
||||||
|
*/
|
||||||
|
static async configGet(key: string): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
const value = await config.get(key)
|
||||||
|
|
||||||
|
if (value === undefined) {
|
||||||
|
console.log(chalk.yellow(`Configuration key '${key}' not found`))
|
||||||
|
} else {
|
||||||
|
console.log(value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* List all configuration keys
|
||||||
|
*/
|
||||||
|
static async configList(options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
const configs = await config.list()
|
||||||
|
|
||||||
|
if (configs.length === 0) {
|
||||||
|
console.log(chalk.yellow('No configurations found'))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
const table = new Table({
|
||||||
|
head: ['Key', 'Encrypted', 'Environment'],
|
||||||
|
style: { head: ['cyan'] }
|
||||||
|
})
|
||||||
|
|
||||||
|
for (const cfg of configs) {
|
||||||
|
const row = [
|
||||||
|
cfg.key,
|
||||||
|
cfg.encrypted ? chalk.green('✓') : chalk.gray('-'),
|
||||||
|
cfg.environment
|
||||||
|
]
|
||||||
|
|
||||||
|
if (options.showValues) {
|
||||||
|
const value = await config.get(cfg.key)
|
||||||
|
row.push(value ? String(value).substring(0, 50) : '')
|
||||||
|
}
|
||||||
|
|
||||||
|
table.push(row)
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(table.toString())
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Import configuration from .env file
|
||||||
|
*/
|
||||||
|
static async configImport(file: string): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
await config.importEnv(file)
|
||||||
|
console.log(chalk.green(`✅ Imported configuration from ${file}`))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Load environment variables
|
||||||
|
*/
|
||||||
|
static async envLoad(options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
const env = await config.loadEnvironment()
|
||||||
|
|
||||||
|
if (options.export) {
|
||||||
|
// Output as shell export commands
|
||||||
|
for (const [key, value] of Object.entries(env)) {
|
||||||
|
console.log(`export ${key}="${value}"`)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Set in current process
|
||||||
|
Object.assign(process.env, env)
|
||||||
|
console.log(chalk.green(`✅ Loaded ${Object.keys(env).length} environment variables`))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Migrate to new storage
|
||||||
|
*/
|
||||||
|
static async migrate(options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
const brainy = config.getBrainy()
|
||||||
|
|
||||||
|
// Parse target storage URL
|
||||||
|
const targetUrl = new URL(options.to)
|
||||||
|
let targetConfig: any = {}
|
||||||
|
|
||||||
|
if (targetUrl.protocol === 's3:') {
|
||||||
|
targetConfig = {
|
||||||
|
type: 's3',
|
||||||
|
bucket: targetUrl.hostname,
|
||||||
|
region: targetUrl.searchParams.get('region') || 'us-east-1'
|
||||||
|
}
|
||||||
|
} else if (targetUrl.protocol === 'file:') {
|
||||||
|
targetConfig = {
|
||||||
|
type: 'filesystem',
|
||||||
|
path: targetUrl.pathname
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create coordination plan
|
||||||
|
const coordinationPlan = {
|
||||||
|
version: 1,
|
||||||
|
timestamp: new Date().toISOString(),
|
||||||
|
migration: {
|
||||||
|
enabled: true,
|
||||||
|
target: targetConfig,
|
||||||
|
strategy: options.strategy,
|
||||||
|
dryRun: options.dryRun || false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Store coordination plan in current storage
|
||||||
|
await brainy.addNoun({
|
||||||
|
id: '_system/coordination',
|
||||||
|
type: 'cortex_coordination',
|
||||||
|
metadata: coordinationPlan
|
||||||
|
})
|
||||||
|
|
||||||
|
console.log(chalk.cyan('📋 Migration plan created:'))
|
||||||
|
console.log(JSON.stringify(coordinationPlan, null, 2))
|
||||||
|
|
||||||
|
if (!options.dryRun) {
|
||||||
|
console.log(chalk.yellow('\n⏳ Migration will begin shortly...'))
|
||||||
|
console.log(chalk.gray('All services will automatically detect and execute the migration'))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Synchronize services
|
||||||
|
*/
|
||||||
|
static async sync(options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
const brainy = config.getBrainy()
|
||||||
|
|
||||||
|
// Trigger sync by updating coordination timestamp
|
||||||
|
await brainy.addNoun({
|
||||||
|
id: '_system/sync',
|
||||||
|
type: 'cortex_sync',
|
||||||
|
metadata: {
|
||||||
|
timestamp: new Date().toISOString(),
|
||||||
|
force: options.force || false
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
console.log(chalk.green('✅ Sync signal sent to all services'))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Query data with advanced filtering
|
||||||
|
*/
|
||||||
|
static async query(pattern: string, options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
const brainy = config.getBrainy()
|
||||||
|
|
||||||
|
// Parse metadata filter if provided
|
||||||
|
let metadataFilter = null
|
||||||
|
if (options.filter || options.metadata) {
|
||||||
|
try {
|
||||||
|
metadataFilter = JSON.parse(options.filter || options.metadata)
|
||||||
|
} catch (error) {
|
||||||
|
console.log(chalk.red(`Invalid filter JSON: ${error.message}`))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Show query plan if requested
|
||||||
|
if (options.explain) {
|
||||||
|
console.log(chalk.cyan('\n📋 Query Plan:'))
|
||||||
|
console.log(`Pattern: "${pattern}"`)
|
||||||
|
if (metadataFilter) {
|
||||||
|
console.log(`Filter: ${JSON.stringify(metadataFilter, null, 2)}`)
|
||||||
|
}
|
||||||
|
console.log(`Limit: ${options.limit || 10}\n`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Perform search with metadata filtering
|
||||||
|
const results = await brainy.searchNouns(pattern, {
|
||||||
|
limit: parseInt(options.limit) || 10,
|
||||||
|
metadata: metadataFilter
|
||||||
|
})
|
||||||
|
|
||||||
|
if (options.json) {
|
||||||
|
console.log(JSON.stringify(results, null, 2))
|
||||||
|
} else {
|
||||||
|
if (results.length === 0) {
|
||||||
|
console.log(chalk.yellow('No results found'))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
const table = new Table({
|
||||||
|
head: ['ID', 'Type', 'Score', 'Sample Metadata'],
|
||||||
|
style: { head: ['cyan'] },
|
||||||
|
colWidths: [30, 15, 10, 40]
|
||||||
|
})
|
||||||
|
|
||||||
|
for (const result of results) {
|
||||||
|
// Show a sample of metadata
|
||||||
|
let metaSample = ''
|
||||||
|
if (result.metadata) {
|
||||||
|
const keys = Object.keys(result.metadata).slice(0, 3)
|
||||||
|
metaSample = keys.map(k => `${k}: ${JSON.stringify(result.metadata[k])}`).join(', ')
|
||||||
|
if (Object.keys(result.metadata).length > 3) {
|
||||||
|
metaSample += ', ...'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
table.push([
|
||||||
|
result.id.substring(0, 28),
|
||||||
|
result.type || '-',
|
||||||
|
result.score?.toFixed(3) || '-',
|
||||||
|
metaSample.substring(0, 38)
|
||||||
|
])
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(table.toString())
|
||||||
|
console.log(chalk.gray(`\nFound ${results.length} results`))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Show statistics
|
||||||
|
*/
|
||||||
|
static async stats(options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
const brainy = config.getBrainy()
|
||||||
|
const stats = await brainy.getStatistics()
|
||||||
|
|
||||||
|
if (options.json) {
|
||||||
|
console.log(JSON.stringify(stats, null, 2))
|
||||||
|
} else {
|
||||||
|
console.log(chalk.cyan('\n📊 Brainy Statistics\n'))
|
||||||
|
console.log(`Nouns: ${chalk.green(stats.totalNouns)}`)
|
||||||
|
console.log(`Verbs: ${chalk.green(stats.totalVerbs)}`)
|
||||||
|
console.log(`Storage Used: ${chalk.green(this.formatBytes(stats.storageUsed || 0))}`)
|
||||||
|
console.log(`Index Size: ${chalk.green(this.formatBytes(stats.indexSize || 0))}`)
|
||||||
|
|
||||||
|
if (stats.metadata) {
|
||||||
|
console.log(`\nMetadata:`)
|
||||||
|
for (const [key, value] of Object.entries(stats.metadata)) {
|
||||||
|
console.log(` ${key}: ${chalk.gray(JSON.stringify(value))}`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Create backup
|
||||||
|
*/
|
||||||
|
static async backup(destination: string | undefined, options: any): Promise<string> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
const timestamp = new Date().toISOString().replace(/[:.]/g, '-')
|
||||||
|
const backupName = `brainy-backup-${timestamp}`
|
||||||
|
const backupPath = destination || path.join(process.cwd(), `${backupName}.json`)
|
||||||
|
|
||||||
|
const brainy = config.getBrainy()
|
||||||
|
|
||||||
|
// Export all data
|
||||||
|
const backup: any = {
|
||||||
|
version: 1,
|
||||||
|
timestamp: new Date().toISOString(),
|
||||||
|
data: {
|
||||||
|
nouns: [],
|
||||||
|
verbs: []
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Export nouns with pagination
|
||||||
|
let offset = 0
|
||||||
|
let hasMore = true
|
||||||
|
while (hasMore) {
|
||||||
|
const result = await brainy.getNouns({
|
||||||
|
pagination: { offset, limit: 100 }
|
||||||
|
})
|
||||||
|
backup.data.nouns.push(...result.items)
|
||||||
|
hasMore = result.hasMore
|
||||||
|
offset += 100
|
||||||
|
}
|
||||||
|
|
||||||
|
// Export verbs with pagination
|
||||||
|
offset = 0
|
||||||
|
hasMore = true
|
||||||
|
while (hasMore) {
|
||||||
|
const result = await brainy.getVerbs({
|
||||||
|
pagination: { offset, limit: 100 }
|
||||||
|
})
|
||||||
|
backup.data.verbs.push(...result.items)
|
||||||
|
hasMore = result.hasMore
|
||||||
|
offset += 100
|
||||||
|
}
|
||||||
|
|
||||||
|
// Include configuration if requested
|
||||||
|
if (options.includeConfig) {
|
||||||
|
backup.config = await config.list()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Save backup
|
||||||
|
const backupContent = JSON.stringify(backup, null, 2)
|
||||||
|
|
||||||
|
if (options.compress) {
|
||||||
|
const zlib = await import('zlib')
|
||||||
|
const compressed = await new Promise<Buffer>((resolve, reject) => {
|
||||||
|
zlib.gzip(Buffer.from(backupContent), (err, result) => {
|
||||||
|
if (err) reject(err)
|
||||||
|
else resolve(result)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
await fs.writeFile(`${backupPath}.gz`, compressed)
|
||||||
|
return `${backupPath}.gz`
|
||||||
|
} else {
|
||||||
|
await fs.writeFile(backupPath, backupContent)
|
||||||
|
return backupPath
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Restore from backup
|
||||||
|
*/
|
||||||
|
static async restore(source: string, options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
let backupContent: string
|
||||||
|
|
||||||
|
if (source.endsWith('.gz')) {
|
||||||
|
const zlib = await import('zlib')
|
||||||
|
const compressed = await fs.readFile(source)
|
||||||
|
backupContent = await new Promise<string>((resolve, reject) => {
|
||||||
|
zlib.gunzip(compressed, (err, result) => {
|
||||||
|
if (err) reject(err)
|
||||||
|
else resolve(result.toString())
|
||||||
|
})
|
||||||
|
})
|
||||||
|
} else {
|
||||||
|
backupContent = await fs.readFile(source, 'utf-8')
|
||||||
|
}
|
||||||
|
|
||||||
|
const backup = JSON.parse(backupContent)
|
||||||
|
const brainy = config.getBrainy()
|
||||||
|
|
||||||
|
if (!options.force) {
|
||||||
|
const rl = readline.createInterface({ input, output })
|
||||||
|
const confirm = await rl.question(
|
||||||
|
chalk.yellow('⚠️ This will overwrite existing data. Continue? (y/n): ')
|
||||||
|
)
|
||||||
|
rl.close()
|
||||||
|
|
||||||
|
if (confirm !== 'y') {
|
||||||
|
console.log('Restore cancelled')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Restore nouns
|
||||||
|
for (const noun of backup.data.nouns) {
|
||||||
|
await brainy.addNoun(noun)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Restore verbs
|
||||||
|
for (const verb of backup.data.verbs) {
|
||||||
|
await brainy.addVerb(verb)
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(chalk.green(`✅ Restored ${backup.data.nouns.length} nouns and ${backup.data.verbs.length} verbs`))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Inspect an item
|
||||||
|
*/
|
||||||
|
static async inspect(id: string, options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
const brainy = config.getBrainy()
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Try as noun first
|
||||||
|
const noun = await brainy.getNoun(id)
|
||||||
|
if (noun) {
|
||||||
|
if (options.raw) {
|
||||||
|
console.log(JSON.stringify(noun, null, 2))
|
||||||
|
} else {
|
||||||
|
console.log(chalk.cyan(`\n📦 Noun: ${id}\n`))
|
||||||
|
console.log(`Type: ${chalk.green(noun.type || 'unknown')}`)
|
||||||
|
if (noun.metadata) {
|
||||||
|
console.log(`Metadata:`)
|
||||||
|
console.log(JSON.stringify(noun.metadata, null, 2))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
// Not a noun, try verb
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Try as verb
|
||||||
|
const verb = await brainy.getVerb(id)
|
||||||
|
if (verb) {
|
||||||
|
if (options.raw) {
|
||||||
|
console.log(JSON.stringify(verb, null, 2))
|
||||||
|
} else {
|
||||||
|
console.log(chalk.cyan(`\n🔗 Verb: ${id}\n`))
|
||||||
|
console.log(`Type: ${chalk.green(verb.type)}`)
|
||||||
|
console.log(`Source: ${chalk.blue(verb.source)}`)
|
||||||
|
console.log(`Target: ${chalk.blue(verb.target)}`)
|
||||||
|
if (verb.metadata) {
|
||||||
|
console.log(`Metadata:`)
|
||||||
|
console.log(JSON.stringify(verb.metadata, null, 2))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
// Not found
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(chalk.red(`Item '${id}' not found`))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Reindex database
|
||||||
|
*/
|
||||||
|
static async reindex(options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
const brainy = config.getBrainy()
|
||||||
|
|
||||||
|
if (options.type === 'metadata' || options.type === 'all') {
|
||||||
|
await brainy.rebuildMetadataIndex()
|
||||||
|
}
|
||||||
|
|
||||||
|
if (options.type === 'hnsw' || options.type === 'all') {
|
||||||
|
// HNSW reindex would go here
|
||||||
|
console.log(chalk.yellow('HNSW reindexing not yet implemented'))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Health check
|
||||||
|
*/
|
||||||
|
static async health(options: any): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
|
||||||
|
try {
|
||||||
|
await config.load()
|
||||||
|
const brainy = config.getBrainy()
|
||||||
|
const stats = await brainy.getStatistics()
|
||||||
|
|
||||||
|
console.log(chalk.green('✅ Cortex: Healthy'))
|
||||||
|
console.log(chalk.green('✅ Brainy: Connected'))
|
||||||
|
console.log(chalk.green(`✅ Storage: Active (${stats.totalNouns} nouns, ${stats.totalVerbs} verbs)`))
|
||||||
|
|
||||||
|
if (options.verbose) {
|
||||||
|
console.log('\nDetailed Health:')
|
||||||
|
console.log(`- Config Path: ${path.join(process.cwd(), '.brainy/cortex.json')}`)
|
||||||
|
console.log(`- Storage Type: ${config.getStorageConfig()?.type}`)
|
||||||
|
console.log(`- Environment: ${config.getCurrentEnvironment()}`)
|
||||||
|
}
|
||||||
|
} catch (error: any) {
|
||||||
|
console.log(chalk.red('❌ Cortex: Error'))
|
||||||
|
console.log(chalk.red(` ${error.message}`))
|
||||||
|
process.exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Interactive shell
|
||||||
|
*/
|
||||||
|
static async shell(): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
await config.load()
|
||||||
|
|
||||||
|
console.log(chalk.cyan('\n🧠 Cortex Interactive Shell\n'))
|
||||||
|
console.log(chalk.gray('Type "help" for commands, "exit" to quit\n'))
|
||||||
|
|
||||||
|
const rl = readline.createInterface({ input, output })
|
||||||
|
|
||||||
|
while (true) {
|
||||||
|
const command = await rl.question(chalk.cyan('cortex> '))
|
||||||
|
|
||||||
|
if (command === 'exit' || command === 'quit') {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
|
||||||
|
if (command === 'help') {
|
||||||
|
console.log(`
|
||||||
|
Available commands:
|
||||||
|
query <pattern> - Search for data
|
||||||
|
get <key> - Get config value
|
||||||
|
set <key> <value> - Set config value
|
||||||
|
stats - Show statistics
|
||||||
|
health - Check health
|
||||||
|
exit - Exit shell
|
||||||
|
`)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse and execute command
|
||||||
|
const [cmd, ...args] = command.split(' ')
|
||||||
|
|
||||||
|
try {
|
||||||
|
switch (cmd) {
|
||||||
|
case 'query':
|
||||||
|
await this.query(args.join(' '), { limit: 5 })
|
||||||
|
break
|
||||||
|
case 'get':
|
||||||
|
await this.configGet(args[0])
|
||||||
|
break
|
||||||
|
case 'set':
|
||||||
|
await this.configSet(args[0], args.slice(1).join(' '), { encrypt: true })
|
||||||
|
break
|
||||||
|
case 'stats':
|
||||||
|
await this.stats({})
|
||||||
|
break
|
||||||
|
case 'health':
|
||||||
|
await this.health({})
|
||||||
|
break
|
||||||
|
default:
|
||||||
|
console.log(chalk.yellow(`Unknown command: ${cmd}`))
|
||||||
|
}
|
||||||
|
} catch (error: any) {
|
||||||
|
console.log(chalk.red(`Error: ${error.message}`))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
rl.close()
|
||||||
|
console.log(chalk.gray('\nGoodbye!\n'))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Show status
|
||||||
|
*/
|
||||||
|
static async status(): Promise<void> {
|
||||||
|
const config = CortexConfig.getInstance()
|
||||||
|
|
||||||
|
try {
|
||||||
|
await config.load()
|
||||||
|
const brainy = config.getBrainy()
|
||||||
|
const stats = await brainy.getStatistics()
|
||||||
|
|
||||||
|
console.log(chalk.cyan('\n🧠 Cortex Status\n'))
|
||||||
|
|
||||||
|
const table = new Table({
|
||||||
|
style: { head: ['cyan'] }
|
||||||
|
})
|
||||||
|
|
||||||
|
table.push(
|
||||||
|
{ 'Configuration': chalk.green('✓ Loaded') },
|
||||||
|
{ 'Storage': `${config.getStorageConfig()?.type}` },
|
||||||
|
{ 'Environment': config.getCurrentEnvironment() },
|
||||||
|
{ 'Encryption': chalk.green('✓ Enabled') },
|
||||||
|
{ 'Total Nouns': stats.totalNouns.toString() },
|
||||||
|
{ 'Total Verbs': stats.totalVerbs.toString() },
|
||||||
|
{ 'Storage Used': this.formatBytes(stats.storageUsed || 0) }
|
||||||
|
)
|
||||||
|
|
||||||
|
console.log(table.toString())
|
||||||
|
|
||||||
|
// Check for active migrations
|
||||||
|
const coordinationNoun = await brainy.getNoun('_system/coordination')
|
||||||
|
if (coordinationNoun?.metadata?.migration?.enabled) {
|
||||||
|
console.log(chalk.yellow('\n⚠️ Active Migration:'))
|
||||||
|
console.log(` Target: ${coordinationNoun.metadata.migration.target.type}`)
|
||||||
|
console.log(` Strategy: ${coordinationNoun.metadata.migration.strategy}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
} catch (error: any) {
|
||||||
|
console.log(chalk.red('❌ Cortex not initialized'))
|
||||||
|
console.log(chalk.gray('Run "cortex init" to get started'))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Format bytes to human readable
|
||||||
|
*/
|
||||||
|
private static formatBytes(bytes: number): string {
|
||||||
|
if (bytes === 0) return '0 B'
|
||||||
|
const k = 1024
|
||||||
|
const sizes = ['B', 'KB', 'MB', 'GB', 'TB']
|
||||||
|
const i = Math.floor(Math.log(bytes) / Math.log(k))
|
||||||
|
return `${(bytes / Math.pow(k, i)).toFixed(2)} ${sizes[i]}`
|
||||||
|
}
|
||||||
|
}
|
||||||
410
src/cortex/config.ts
Normal file
410
src/cortex/config.ts
Normal file
|
|
@ -0,0 +1,410 @@
|
||||||
|
/**
|
||||||
|
* Cortex Configuration Management
|
||||||
|
*
|
||||||
|
* Handles encrypted configuration storage, environment loading,
|
||||||
|
* and distributed coordination through Brainy.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { BrainyData } from '../brainyData.js'
|
||||||
|
import { StorageConfig } from '../coreTypes.js'
|
||||||
|
import * as crypto from 'crypto'
|
||||||
|
import * as fs from 'fs/promises'
|
||||||
|
import * as path from 'path'
|
||||||
|
import { fileURLToPath } from 'url'
|
||||||
|
import { dirname } from 'path'
|
||||||
|
|
||||||
|
const __filename = fileURLToPath(import.meta.url)
|
||||||
|
const __dirname = dirname(__filename)
|
||||||
|
|
||||||
|
export interface CortexConfigData {
|
||||||
|
version: number
|
||||||
|
storage: StorageConfig
|
||||||
|
encryption?: {
|
||||||
|
enabled: boolean
|
||||||
|
keyDerivation: 'pbkdf2' | 'scrypt'
|
||||||
|
iterations?: number
|
||||||
|
}
|
||||||
|
coordination?: {
|
||||||
|
enabled: boolean
|
||||||
|
realtime: boolean
|
||||||
|
pollInterval: number
|
||||||
|
}
|
||||||
|
environments?: {
|
||||||
|
current: string
|
||||||
|
available: string[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface EncryptedValue {
|
||||||
|
encrypted: true
|
||||||
|
algorithm: string
|
||||||
|
iv: string
|
||||||
|
authTag: string
|
||||||
|
data: string
|
||||||
|
}
|
||||||
|
|
||||||
|
export class CortexConfig {
|
||||||
|
private static instance: CortexConfig
|
||||||
|
private brainy?: BrainyData
|
||||||
|
private config?: CortexConfigData
|
||||||
|
private configPath: string
|
||||||
|
private masterKey?: Buffer
|
||||||
|
|
||||||
|
private constructor() {
|
||||||
|
this.configPath = path.join(process.cwd(), '.brainy', 'cortex.json')
|
||||||
|
}
|
||||||
|
|
||||||
|
static getInstance(): CortexConfig {
|
||||||
|
if (!CortexConfig.instance) {
|
||||||
|
CortexConfig.instance = new CortexConfig()
|
||||||
|
}
|
||||||
|
return CortexConfig.instance
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Initialize Cortex configuration
|
||||||
|
*/
|
||||||
|
async init(options: Partial<CortexConfigData> = {}): Promise<void> {
|
||||||
|
// Create .brainy directory if it doesn't exist
|
||||||
|
const brainyDir = path.dirname(this.configPath)
|
||||||
|
await fs.mkdir(brainyDir, { recursive: true })
|
||||||
|
|
||||||
|
// Default configuration
|
||||||
|
const defaultConfig: CortexConfigData = {
|
||||||
|
version: 1,
|
||||||
|
storage: options.storage || { type: 'memory' },
|
||||||
|
encryption: {
|
||||||
|
enabled: true,
|
||||||
|
keyDerivation: 'pbkdf2',
|
||||||
|
iterations: 100000
|
||||||
|
},
|
||||||
|
coordination: {
|
||||||
|
enabled: true,
|
||||||
|
realtime: false,
|
||||||
|
pollInterval: 30000 // 30 seconds
|
||||||
|
},
|
||||||
|
environments: {
|
||||||
|
current: process.env.NODE_ENV || 'development',
|
||||||
|
available: ['development', 'staging', 'production']
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
this.config = { ...defaultConfig, ...options }
|
||||||
|
|
||||||
|
// Save configuration
|
||||||
|
await this.saveConfig()
|
||||||
|
|
||||||
|
// Initialize Brainy with the configuration
|
||||||
|
await this.initBrainy()
|
||||||
|
|
||||||
|
// Generate or load master key
|
||||||
|
await this.initMasterKey()
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Load existing configuration
|
||||||
|
*/
|
||||||
|
async load(): Promise<void> {
|
||||||
|
try {
|
||||||
|
const configData = await fs.readFile(this.configPath, 'utf-8')
|
||||||
|
this.config = JSON.parse(configData)
|
||||||
|
await this.initBrainy()
|
||||||
|
await this.initMasterKey()
|
||||||
|
} catch (error) {
|
||||||
|
throw new Error(`No Cortex configuration found. Run 'cortex init' first.`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Save configuration to disk
|
||||||
|
*/
|
||||||
|
private async saveConfig(): Promise<void> {
|
||||||
|
await fs.writeFile(
|
||||||
|
this.configPath,
|
||||||
|
JSON.stringify(this.config, null, 2),
|
||||||
|
'utf-8'
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Initialize Brainy instance
|
||||||
|
*/
|
||||||
|
private async initBrainy(): Promise<void> {
|
||||||
|
if (!this.config) {
|
||||||
|
throw new Error('Configuration not loaded')
|
||||||
|
}
|
||||||
|
|
||||||
|
this.brainy = new BrainyData({
|
||||||
|
storage: this.config.storage,
|
||||||
|
writeOnlyMode: false,
|
||||||
|
enableMetadataIndexing: true
|
||||||
|
})
|
||||||
|
|
||||||
|
await this.brainy.init()
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Initialize or load master encryption key
|
||||||
|
*/
|
||||||
|
private async initMasterKey(): Promise<void> {
|
||||||
|
const keyPath = path.join(process.cwd(), '.brainy', 'cortex.key')
|
||||||
|
|
||||||
|
// Try to load from environment first
|
||||||
|
if (process.env.CORTEX_MASTER_KEY) {
|
||||||
|
this.masterKey = Buffer.from(process.env.CORTEX_MASTER_KEY, 'base64')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Try to load from file
|
||||||
|
try {
|
||||||
|
const keyData = await fs.readFile(keyPath, 'utf-8')
|
||||||
|
this.masterKey = Buffer.from(keyData, 'base64')
|
||||||
|
} catch (error) {
|
||||||
|
// Generate new key
|
||||||
|
this.masterKey = crypto.randomBytes(32)
|
||||||
|
await fs.writeFile(keyPath, this.masterKey.toString('base64'), 'utf-8')
|
||||||
|
|
||||||
|
// Set restrictive permissions (Unix-like systems)
|
||||||
|
try {
|
||||||
|
await fs.chmod(keyPath, 0o600)
|
||||||
|
} catch (e) {
|
||||||
|
// Windows doesn't support chmod, ignore
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('🔐 Generated new master key at .brainy/cortex.key')
|
||||||
|
console.log('⚠️ Keep this key safe! You\'ll need it to decrypt your configs.')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get Brainy instance
|
||||||
|
*/
|
||||||
|
getBrainy(): BrainyData {
|
||||||
|
if (!this.brainy) {
|
||||||
|
throw new Error('Brainy not initialized. Run load() first.')
|
||||||
|
}
|
||||||
|
return this.brainy
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Encrypt a value
|
||||||
|
*/
|
||||||
|
encrypt(value: string): EncryptedValue {
|
||||||
|
if (!this.masterKey) {
|
||||||
|
throw new Error('Master key not initialized')
|
||||||
|
}
|
||||||
|
|
||||||
|
const algorithm = 'aes-256-gcm'
|
||||||
|
const iv = crypto.randomBytes(16)
|
||||||
|
const cipher = crypto.createCipheriv(algorithm, this.masterKey, iv)
|
||||||
|
|
||||||
|
let encrypted = cipher.update(value, 'utf8', 'hex')
|
||||||
|
encrypted += cipher.final('hex')
|
||||||
|
|
||||||
|
const authTag = cipher.getAuthTag()
|
||||||
|
|
||||||
|
return {
|
||||||
|
encrypted: true,
|
||||||
|
algorithm,
|
||||||
|
iv: iv.toString('hex'),
|
||||||
|
authTag: authTag.toString('hex'),
|
||||||
|
data: encrypted
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Decrypt a value
|
||||||
|
*/
|
||||||
|
decrypt(encryptedValue: EncryptedValue): string {
|
||||||
|
if (!this.masterKey) {
|
||||||
|
throw new Error('Master key not initialized')
|
||||||
|
}
|
||||||
|
|
||||||
|
const decipher = crypto.createDecipheriv(
|
||||||
|
encryptedValue.algorithm,
|
||||||
|
this.masterKey,
|
||||||
|
Buffer.from(encryptedValue.iv, 'hex')
|
||||||
|
)
|
||||||
|
|
||||||
|
decipher.setAuthTag(Buffer.from(encryptedValue.authTag, 'hex'))
|
||||||
|
|
||||||
|
let decrypted = decipher.update(encryptedValue.data, 'hex', 'utf8')
|
||||||
|
decrypted += decipher.final('utf8')
|
||||||
|
|
||||||
|
return decrypted
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Set a configuration value in Brainy
|
||||||
|
*/
|
||||||
|
async set(key: string, value: any, options: { encrypt?: boolean } = {}): Promise<void> {
|
||||||
|
if (!this.brainy) {
|
||||||
|
await this.load()
|
||||||
|
}
|
||||||
|
|
||||||
|
const configKey = `_cortex/config/${key}`
|
||||||
|
const configValue = options.encrypt && typeof value === 'string'
|
||||||
|
? this.encrypt(value)
|
||||||
|
: value
|
||||||
|
|
||||||
|
await this.brainy!.addNoun({
|
||||||
|
id: configKey,
|
||||||
|
type: 'cortex_config',
|
||||||
|
metadata: {
|
||||||
|
key,
|
||||||
|
value: configValue,
|
||||||
|
encrypted: options.encrypt || false,
|
||||||
|
environment: this.config?.environments?.current,
|
||||||
|
updatedAt: new Date().toISOString()
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get a configuration value from Brainy
|
||||||
|
*/
|
||||||
|
async get(key: string): Promise<any> {
|
||||||
|
if (!this.brainy) {
|
||||||
|
await this.load()
|
||||||
|
}
|
||||||
|
|
||||||
|
const configKey = `_cortex/config/${key}`
|
||||||
|
|
||||||
|
try {
|
||||||
|
const noun = await this.brainy!.getNoun(configKey)
|
||||||
|
if (!noun?.metadata?.value) {
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
|
||||||
|
const value = noun.metadata.value
|
||||||
|
|
||||||
|
// Decrypt if needed
|
||||||
|
if (value.encrypted === true) {
|
||||||
|
return this.decrypt(value as EncryptedValue)
|
||||||
|
}
|
||||||
|
|
||||||
|
return value
|
||||||
|
} catch (error) {
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* List all configuration keys
|
||||||
|
*/
|
||||||
|
async list(): Promise<Array<{ key: string, encrypted: boolean, environment: string }>> {
|
||||||
|
if (!this.brainy) {
|
||||||
|
await this.load()
|
||||||
|
}
|
||||||
|
|
||||||
|
const result = await this.brainy!.getNouns({
|
||||||
|
filter: { type: 'cortex_config' },
|
||||||
|
pagination: { limit: 1000 }
|
||||||
|
})
|
||||||
|
|
||||||
|
return result.items.map(noun => ({
|
||||||
|
key: noun.metadata?.key || noun.id.replace('_cortex/config/', ''),
|
||||||
|
encrypted: noun.metadata?.encrypted || false,
|
||||||
|
environment: noun.metadata?.environment || 'default'
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Load all configurations as environment variables
|
||||||
|
*/
|
||||||
|
async loadEnvironment(): Promise<Record<string, string>> {
|
||||||
|
if (!this.brainy) {
|
||||||
|
await this.load()
|
||||||
|
}
|
||||||
|
|
||||||
|
const configs = await this.brainy!.getNouns({
|
||||||
|
filter: {
|
||||||
|
type: 'cortex_config',
|
||||||
|
'metadata.environment': this.config?.environments?.current
|
||||||
|
},
|
||||||
|
pagination: { limit: 1000 }
|
||||||
|
})
|
||||||
|
|
||||||
|
const env: Record<string, string> = {}
|
||||||
|
|
||||||
|
for (const config of configs.items) {
|
||||||
|
const key = config.metadata?.key || config.id.replace('_cortex/config/', '')
|
||||||
|
let value = config.metadata?.value
|
||||||
|
|
||||||
|
// Decrypt if needed
|
||||||
|
if (value?.encrypted === true) {
|
||||||
|
value = this.decrypt(value as EncryptedValue)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Convert to string if needed
|
||||||
|
if (typeof value !== 'string') {
|
||||||
|
value = JSON.stringify(value)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Use the key as-is (could be nested like 'database.url')
|
||||||
|
// Or convert to UPPER_SNAKE_CASE
|
||||||
|
const envKey = key.toUpperCase().replace(/\./g, '_').replace(/-/g, '_')
|
||||||
|
env[envKey] = value
|
||||||
|
}
|
||||||
|
|
||||||
|
return env
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Import configuration from .env file
|
||||||
|
*/
|
||||||
|
async importEnv(filePath: string): Promise<void> {
|
||||||
|
const content = await fs.readFile(filePath, 'utf-8')
|
||||||
|
const lines = content.split('\n')
|
||||||
|
|
||||||
|
for (const line of lines) {
|
||||||
|
// Skip comments and empty lines
|
||||||
|
if (!line.trim() || line.startsWith('#')) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
const [key, ...valueParts] = line.split('=')
|
||||||
|
const value = valueParts.join('=').trim()
|
||||||
|
|
||||||
|
if (key && value) {
|
||||||
|
// Detect if it looks like a secret
|
||||||
|
const isSecret = key.includes('KEY') ||
|
||||||
|
key.includes('SECRET') ||
|
||||||
|
key.includes('PASSWORD') ||
|
||||||
|
key.includes('TOKEN')
|
||||||
|
|
||||||
|
await this.set(key.trim(), value, { encrypt: isSecret })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get storage configuration
|
||||||
|
*/
|
||||||
|
getStorageConfig(): StorageConfig | undefined {
|
||||||
|
return this.config?.storage
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get current environment
|
||||||
|
*/
|
||||||
|
getCurrentEnvironment(): string {
|
||||||
|
return this.config?.environments?.current || 'development'
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Switch environment
|
||||||
|
*/
|
||||||
|
async switchEnvironment(environment: string): Promise<void> {
|
||||||
|
if (!this.config) {
|
||||||
|
await this.load()
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!this.config!.environments?.available.includes(environment)) {
|
||||||
|
throw new Error(`Unknown environment: ${environment}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
this.config!.environments!.current = environment
|
||||||
|
await this.saveConfig()
|
||||||
|
}
|
||||||
|
}
|
||||||
Loading…
Add table
Add a link
Reference in a new issue