refactor: streamline core API surface

This commit is contained in:
David Snelling 2025-10-04 08:51:49 -07:00
parent 0d54da1471
commit 75ae282861
12 changed files with 4 additions and 3402 deletions

View file

@ -1,440 +0,0 @@
# MCP Integration - Claude Code Setup
**One command. Infinite memory.** This guide shows you how to give Claude Code infinite context and conversation history using Brainy's Model Control Protocol (MCP) integration.
## Quick Setup
### One-Time Configuration
```bash
# Install Brainy globally (if not already installed)
npm install -g @soulcraft/brainy
# Set up MCP server for Claude Code
brainy conversation setup
```
**That's it!** Claude Code now has infinite memory.
### What This Does
The setup command:
1. Creates `~/.brainy-memory/` directory
2. Initializes Brainy database with filesystem storage
3. Creates MCP server script
4. Registers server with Claude Code
5. Ready to use - no configuration needed
## How It Works
### Automatic Integration
Once set up, Claude Code **automatically**:
**On Every Message:**
- Saves your message with semantic embeddings
- Saves Claude's response with metadata
- Links code artifacts to conversations
- Tracks problem-solving phase and confidence
- Indexes everything for instant retrieval
**On Conversation Start:**
- Retrieves relevant past context
- Finds similar previous conversations
- Loads linked code artifacts
- Presents context to Claude seamlessly
**Result:** Claude never loses context or momentum, even across completely separate conversations.
### User Experience
**Before:**
```
You: "How do I implement JWT auth?"
Claude: [Implements authentication]
[Days later, new conversation]
You: "Can you fix the JWT token validation?"
Claude: "I don't have context about your JWT implementation..."
```
**After:**
```
You: "How do I implement JWT auth?"
Claude: [Implements authentication, saves to Brainy]
[Days later, new conversation]
You: "Can you fix the JWT token validation?"
Claude: "I found 3 related past conversations about JWT...
Here's the implementation from /auth/middleware.ts..."
[Full context automatically retrieved]
```
## MCP Tools
The MCP server exposes 6 conversation tools that Claude Code uses automatically:
### 1. conversation_save_message
Saves a message to conversation history.
**Used by:** Claude Code automatically after each message
**Parameters:**
- `content`: Message text
- `role`: 'user' | 'assistant' | 'system' | 'tool'
- `conversationId`: Conversation identifier
- `phase`: Problem-solving phase
- `confidence`: Confidence score
- `artifacts`: Array of artifact IDs
- `toolsUsed`: Tools used in this message
### 2. conversation_get_context
Retrieves relevant past context.
**Used by:** Claude Code at conversation start or when context is needed
**Parameters:**
- `query`: What to retrieve context for
- `limit`: Max messages (default: 10)
- `maxTokens`: Token budget (default: 50000)
- `relevanceThreshold`: Min similarity (default: 0.7)
- `includeArtifacts`: Include code/files
- `includeSimilarConversations`: Include similar past conversations
**Returns:**
- Ranked messages with relevance scores
- Linked artifacts
- Similar past conversations
- Metadata (query time, tokens, etc.)
### 3. conversation_search
Searches all conversations semantically.
**Used by:** When Claude needs to find specific past information
**Parameters:**
- `query`: Search query
- `role`: Filter by role
- `conversationId`: Filter by conversation
- `timeRange`: Time range filter
### 4. conversation_get_thread
Gets complete conversation thread.
**Used by:** When Claude needs full conversation history
**Parameters:**
- `conversationId`: Conversation ID
- `includeArtifacts`: Include linked artifacts
### 5. conversation_save_artifact
Saves code/file artifacts.
**Used by:** When Claude creates files
**Parameters:**
- `path`: File path
- `content`: File content
- `conversationId`: Conversation ID
- `messageId`: Message that created it
- `type`: 'code' | 'config' | 'data' | 'document'
- `language`: Programming language
### 6. conversation_find_similar
Finds similar past conversations.
**Used by:** For discovering related work
**Parameters:**
- `conversationId`: Conversation to find similar to
- `limit`: Max results
- `threshold`: Similarity threshold
## MCP Server Architecture
### Server Location
```
~/.brainy-memory/
├── mcp-server.js # MCP server script
└── data/ # Brainy database
├── nouns/ # Messages
├── verbs/ # Relationships
└── index/ # Search indexes
```
### Server Script
The generated server script (`~/.brainy-memory/mcp-server.js`):
```javascript
import { Brainy } from '@soulcraft/brainy'
import { BrainyMCPService } from '@soulcraft/brainy'
import { MCPConversationToolset } from '@soulcraft/brainy'
// Initialize Brainy
const brain = new Brainy({
storage: {
type: 'filesystem',
path: '~/.brainy-memory/data'
},
silent: true
})
await brain.init()
// Create MCP service
const mcpService = new BrainyMCPService(brain)
const conversationTools = new MCPConversationToolset(brain)
// Handle MCP requests via stdio
process.stdin.on('data', async (data) => {
const request = JSON.parse(data.toString())
let response
if (request.toolName?.startsWith('conversation_')) {
response = await conversationTools.handleRequest(request)
} else {
response = await mcpService.handleRequest(request)
}
process.stdout.write(JSON.stringify(response) + '\\n')
})
```
### Claude Code Configuration
Located at `~/.config/claude-code/mcp-servers.json`:
```json
{
"brainy-memory": {
"command": "node",
"args": ["~/.brainy-memory/mcp-server.js"],
"env": {
"NODE_ENV": "production"
}
}
}
```
## Advanced Configuration
### Custom Storage Location
Edit `~/.brainy-memory/mcp-server.js`:
```javascript
const brain = new Brainy({
storage: {
type: 'filesystem',
path: '/path/to/custom/location'
}
})
```
### Cloud Storage (Multi-Device Sync)
Use S3-compatible storage for sync across machines:
```javascript
const brain = new Brainy({
storage: {
type: 's3',
bucket: 'my-brainy-memory',
region: 'us-east-1'
}
})
```
**Requirements:** Set AWS credentials in environment:
```bash
export AWS_ACCESS_KEY_ID=xxx
export AWS_SECRET_ACCESS_KEY=yyy
```
### Memory Storage (Testing)
For testing or temporary use:
```javascript
const brain = new Brainy({
storage: { type: 'memory' }
})
```
**Note:** Memory storage is lost on server restart.
### Context Retrieval Options
Customize context retrieval behavior:
```javascript
const context = await conversationManager.getRelevantContext(query, {
limit: 15, // More messages
maxTokens: 80000, // Larger context window
relevanceThreshold: 0.6, // Lower threshold = more results
weights: {
semantic: 0.7, // Adjust relevance weights
temporal: 0.6,
graph: 0.4
}
})
```
## Troubleshooting
### Server Not Starting
**Check logs:**
```bash
# Server writes to stderr
tail -f ~/.brainy-memory/server.log
```
**Common issues:**
1. **Node version**: Requires Node.js 22 LTS
2. **Permissions**: Ensure ~/.brainy-memory is writable
3. **Port conflicts**: MCP uses stdio, no ports needed
### Claude Code Not Using Memory
**Verify setup:**
```bash
# Check MCP server is registered
cat ~/.config/claude-code/mcp-servers.json
# Test server manually
echo '{"type":"system_info","infoType":"version","requestId":"test","version":"1.0.0"}' | node ~/.brainy-memory/mcp-server.js
```
**Expected output:**
```json
{"success":true,"requestId":"test","version":"1.0.0","data":{"version":"1.0.0"}}
```
### Memory Not Persisting
**Check storage:**
```bash
# Verify data directory exists
ls -la ~/.brainy-memory/data/
# Check database files
du -sh ~/.brainy-memory/data/
```
**If empty:** Server may be using memory storage. Check `mcp-server.js` configuration.
### Performance Issues
**Optimize database:**
```bash
# Rebuild indexes
brainy conversation stats # This triggers index optimization
```
**Check size:**
```bash
# Show storage usage
du -sh ~/.brainy-memory/
```
**If too large:** Consider cloud storage or compaction strategies.
## CLI Commands
Manage conversations via CLI:
### View Statistics
```bash
brainy conversation stats
```
Output:
```
📊 Conversation Statistics
Overall:
Conversations: 42
Messages: 1,337
Total Tokens: 567,890
Avg Messages/Conversation: 31.8
Avg Tokens/Message: 425.1
By Role:
user: 650
assistant: 687
By Phase:
implementation: 423
planning: 201
testing: 98
```
### Search Messages
```bash
brainy conversation search -q "authentication" -l 10
```
### Get Context
```bash
brainy conversation context -q "JWT token validation" -l 15
```
### View Thread
```bash
brainy conversation thread -c conv_abc123
```
### Export Conversation
```bash
brainy conversation export -c conv_abc123 -o backup.json
```
### Import Conversation
```bash
brainy conversation import -o backup.json
```
## Security & Privacy
### Local-First by Default
- All data stored locally in `~/.brainy-memory/`
- No external services or APIs
- Complete privacy and control
### Data Encryption
For sensitive conversations, use encrypted filesystem:
```bash
# Create encrypted volume
hdiutil create -size 1g -encryption AES-256 -volname BrainyMemory ~/brainy-secure.dmg
# Mount and use
hdiutil attach ~/brainy-secure.dmg
brainy conversation setup --path /Volumes/BrainyMemory
```
### Access Control
MCP server runs with your user permissions. No additional authentication needed for local use.
## Next Steps
- [API Reference](./API_REFERENCE.md) - Complete API documentation
- [Examples](./EXAMPLES.md) - Usage examples and patterns
- [Advanced Features](./ADVANCED.md) - Advanced configuration and optimization
## Support
Issues or questions:
- GitHub: [soulcraftlabs/brainy/issues](https://github.com/soulcraftlabs/brainy/issues)
- Documentation: [docs.brainy.ai](https://docs.brainy.ai)

View file

@ -1,453 +0,0 @@
# Infinite Agent Memory - Conversation API
**Never lose context again.** Brainy's Conversation API provides infinite memory and context management for AI agents like Claude Code, enabling truly continuous conversations with semantic search, smart context retrieval, and automatic knowledge preservation.
## Overview
The Conversation API turns your agent interactions into a living knowledge graph where:
- **Every message is preserved** with semantic embeddings for instant retrieval
- **Context is automatically retrieved** using Triple Intelligence (vector + graph + metadata)
- **Similar conversations are discovered** through neural clustering
- **Code artifacts are linked** to the conversations that created them
- **Memory scales infinitely** to millions of messages with <100ms retrieval
## Quick Start
### Zero-Config Usage
```typescript
import { Brainy } from '@soulcraft/brainy'
const brain = new Brainy() // Zero configuration!
await brain.init()
// Access conversation manager (lazy-loaded)
const conv = brain.conversation
// Save messages with automatic embedding and indexing
const messageId = await conv.saveMessage(
"How do I implement JWT authentication?",
"user",
{ conversationId: "conv_123" }
)
// Get relevant context with semantic search
const context = await conv.getRelevantContext("authentication implementation", {
limit: 10,
includeArtifacts: true
})
// Context includes:
// - Semantically similar messages
// - Recent related conversations
// - Linked code artifacts
// - Relevance scores and explanations
```
### Claude Code Integration (MCP)
One-time setup:
```bash
brainy conversation setup
```
That's it! Claude Code automatically:
- Saves every message with embeddings
- Retrieves relevant past context
- Links code artifacts to conversations
- Never loses context or momentum
## Core Concepts
### 1. Messages
Every message is a **semantic entity** with:
- **Content**: The actual message text
- **Role**: user, assistant, system, or tool
- **Embeddings**: Automatic vector representation
- **Metadata**: Timestamps, conversation ID, phase, confidence, etc.
- **Relationships**: Temporal links to previous/next messages
```typescript
const messageId = await conv.saveMessage(
"Implement user authentication",
"user",
{
conversationId: "conv_123",
phase: "planning",
confidence: 0.95,
tags: ["authentication", "security"]
}
)
```
### 2. Conversations
A conversation is a **collection of related messages**:
- Tracked by `conversationId`
- Contains temporal message sequence
- Stores aggregate metadata (tokens, duration, participants)
- Can span multiple sessions
```typescript
const thread = await conv.getConversationThread("conv_123", {
includeArtifacts: true
})
console.log(`${thread.messages.length} messages, ${thread.metadata.totalTokens} tokens`)
```
### 3. Context Retrieval
Smart context retrieval uses **Triple Intelligence**:
- **Semantic**: Vector similarity to find related messages
- **Temporal**: Recency decay favors recent context
- **Graph**: Relationship traversal finds connected knowledge
```typescript
const context = await conv.getRelevantContext("how to validate JWT tokens", {
limit: 10,
maxTokens: 50000,
relevanceThreshold: 0.7,
weights: {
semantic: 1.0, // Prioritize meaning
temporal: 0.5, // Recent is relevant
graph: 0.3 // Connected knowledge
}
})
```
### 4. Artifacts
Code and files created during conversations are **first-class citizens**:
- Stored in Brainy's Virtual Filesystem
- Linked to messages via graph relationships
- Searchable by content and metadata
- Retrieved with context
```typescript
const artifactId = await conv.saveArtifact(
'/auth/middleware.ts',
codeContent,
{
conversationId: "conv_123",
messageId: messageId,
type: 'code',
language: 'typescript'
}
)
```
## API Reference
### ConversationManager
#### `saveMessage(content, role, options)`
Save a message with automatic embedding.
**Parameters:**
- `content` (string): Message content
- `role` (MessageRole): 'user' | 'assistant' | 'system' | 'tool'
- `options` (SaveMessageOptions):
- `conversationId?`: Conversation ID (auto-generated if not provided)
- `sessionId?`: Session ID
- `phase?`: Problem-solving phase
- `confidence?`: Confidence score (0-1)
- `artifacts?`: Array of artifact IDs
- `toolsUsed?`: Array of tool names
- `tags?`: Array of tags
- `linkToPrevious?`: ID of previous message
**Returns:** `Promise<string>` - Message ID
**Example:**
```typescript
const id = await conv.saveMessage(
"Implement authentication middleware",
"assistant",
{
conversationId: "conv_123",
phase: "implementation",
confidence: 0.92,
artifacts: ["middleware-id"],
toolsUsed: ["write", "edit"],
tags: ["authentication", "middleware"]
}
)
```
#### `getRelevantContext(query, options)`
Retrieve relevant context with smart ranking.
**Parameters:**
- `query` (string | ContextRetrievalOptions): Query or full options
- `options?` (ContextRetrievalOptions):
- `limit?`: Max messages (default: 10)
- `maxTokens?`: Token budget (default: 50000)
- `relevanceThreshold?`: Min score (default: 0.7)
- `role?`: Filter by role
- `phase?`: Filter by phase
- `tags?`: Filter by tags
- `timeRange?`: Time range filter
- `weights?`: Scoring weights
- `includeArtifacts?`: Include linked artifacts
- `includeSimilarConversations?`: Include similar conversations
**Returns:** `Promise<ConversationContext>`
**Example:**
```typescript
const context = await conv.getRelevantContext({
query: "JWT token validation",
limit: 15,
maxTokens: 60000,
role: "assistant",
phase: ["implementation", "testing"],
tags: ["authentication"],
includeArtifacts: true,
includeSimilarConversations: true
})
console.log(`Found ${context.messages.length} relevant messages`)
console.log(`Total tokens: ${context.totalTokens}`)
console.log(`Query time: ${context.metadata.queryTime}ms`)
```
#### `searchMessages(options)`
Search messages with semantic similarity.
**Parameters:**
- `options` (ConversationSearchOptions):
- `query`: Search query (required)
- `limit?`: Max results (default: 10)
- `role?`: Filter by role
- `conversationId?`: Filter by conversation
- `sessionId?`: Filter by session
- `timeRange?`: Time range filter
**Returns:** `Promise<ConversationSearchResult[]>`
**Example:**
```typescript
const results = await conv.searchMessages({
query: "authentication errors",
limit: 20,
role: "assistant",
timeRange: {
start: Date.now() - 7*24*60*60*1000 // Last 7 days
}
})
for (const result of results) {
console.log(`${result.message.role}: ${result.snippet}`)
console.log(`Score: ${result.score}, Conv: ${result.conversationId}`)
}
```
#### `getConversationThread(conversationId, options)`
Get complete conversation thread.
**Parameters:**
- `conversationId` (string): Conversation ID
- `options?`:
- `includeArtifacts?`: Include linked artifacts
**Returns:** `Promise<ConversationThread>`
**Example:**
```typescript
const thread = await conv.getConversationThread("conv_123", {
includeArtifacts: true
})
console.log(`Conversation: ${thread.id}`)
console.log(`Messages: ${thread.metadata.messageCount}`)
console.log(`Duration: ${new Date(thread.metadata.endTime) - new Date(thread.metadata.startTime)}ms`)
for (const msg of thread.messages) {
console.log(`[${msg.role}] ${msg.content}`)
}
```
#### `findSimilarConversations(conversationId, limit, threshold)`
Find similar past conversations.
**Parameters:**
- `conversationId` (string): Conversation to find similar to
- `limit?` (number): Max results (default: 5)
- `threshold?` (number): Min similarity (default: 0.7)
**Returns:** `Promise<Array<{id, relevance, metadata}>>`
**Example:**
```typescript
const similar = await conv.findSimilarConversations("conv_123", 5, 0.75)
for (const s of similar) {
console.log(`Similar conversation: ${s.id}`)
console.log(`Relevance: ${s.relevance.toFixed(2)}`)
}
```
#### `getConversationThemes(conversationId)`
Discover themes via clustering.
**Parameters:**
- `conversationId` (string): Conversation ID
**Returns:** `Promise<ConversationTheme[]>`
**Example:**
```typescript
const themes = await conv.getConversationThemes("conv_123")
for (const theme of themes) {
console.log(`Theme: ${theme.label}`)
console.log(`Messages: ${theme.messages.length}`)
console.log(`Coherence: ${theme.coherence}`)
}
```
#### `saveArtifact(path, content, options)`
Save code/file artifact.
**Parameters:**
- `path` (string): VFS path
- `content` (string | Buffer): File content
- `options` (ArtifactOptions):
- `conversationId`: Conversation ID (required)
- `messageId?`: Message ID to link
- `type?`: 'code' | 'config' | 'data' | 'document' | 'other'
- `language?`: Programming language
- `description?`: Artifact description
**Returns:** `Promise<string>` - Artifact ID
**Example:**
```typescript
const artifactId = await conv.saveArtifact(
'/auth/jwt-middleware.ts',
middlewareCode,
{
conversationId: "conv_123",
messageId: messageId,
type: 'code',
language: 'typescript',
description: 'JWT authentication middleware'
}
)
```
#### `getConversationStats(conversationId?)`
Get conversation statistics.
**Parameters:**
- `conversationId?` (string): Optional conversation to filter
**Returns:** `Promise<ConversationStats>`
**Example:**
```typescript
const stats = await conv.getConversationStats()
console.log(`Total conversations: ${stats.totalConversations}`)
console.log(`Total messages: ${stats.totalMessages}`)
console.log(`Total tokens: ${stats.totalTokens}`)
console.log(`By role:`, stats.roles)
console.log(`By phase:`, stats.phases)
```
#### `exportConversation(conversationId)`
Export conversation to JSON.
**Returns:** `Promise<any>` - Serializable conversation object
#### `importConversation(data)`
Import conversation from JSON.
**Returns:** `Promise<string>` - New conversation ID
## Advanced Usage
### Custom Relevance Weights
Fine-tune context retrieval:
```typescript
const context = await conv.getRelevantContext(query, {
weights: {
semantic: 0.6, // Less weight on exact matching
temporal: 0.8, // More weight on recent messages
graph: 0.4 // Moderate weight on relationships
}
})
```
### Phase-Based Filtering
Track problem-solving progress:
```typescript
const planningMessages = await conv.searchMessages({
query: "authentication approach",
role: "assistant",
phase: ["planning", "analysis"]
})
```
### Time-Based Analysis
Analyze conversation evolution:
```typescript
const recentContext = await conv.getRelevantContext(query, {
timeRange: {
start: Date.now() - 24*60*60*1000 // Last 24 hours
}
})
```
### Multi-Session Conversations
Track across sessions:
```typescript
// Session 1
await conv.saveMessage(content1, "user", {
conversationId: "conv_123",
sessionId: "session_1"
})
// Session 2 (different day)
await conv.saveMessage(content2, "user", {
conversationId: "conv_123", // Same conversation!
sessionId: "session_2"
})
```
## Performance
- **Message save**: <50ms (with embedding)
- **Context retrieval**: <100ms (10 messages)
- **Search**: <150ms (1000s of messages)
- **Theme clustering**: <500ms
- **Scales to**: 10M+ messages
- **Storage**: Efficient vector + graph storage
## Next Steps
- [MCP Integration Guide](./MCP_INTEGRATION.md) - Claude Code setup
- [API Reference](./API_REFERENCE.md) - Complete API documentation
- [Examples](./EXAMPLES.md) - Code examples and patterns
- [Advanced Features](./ADVANCED.md) - Advanced usage and optimization