feat: Integration Hub for external tool connectivity

- Add native config option: `new Brainy({ integrations: true })`
- OData integration for Excel Power Query and Power BI
- Google Sheets integration with Apps Script
- Server-Sent Events (SSE) for real-time streaming
- Webhooks for push notifications
- Zero-config with sensible defaults
- Full tree-shaking when disabled
This commit is contained in:
David Snelling 2026-01-20 16:21:11 -08:00
parent 24039e8a1a
commit b5bc9000cf
28 changed files with 8186 additions and 1 deletions

350
integrations/README.md Normal file
View file

@ -0,0 +1,350 @@
# Brainy Integrations
Connect Brainy to spreadsheets, BI tools, and external systems with zero configuration.
## Quick Start
```typescript
import { Brainy } from '@soulcraft/brainy'
const brain = new Brainy({ integrations: true })
await brain.init()
// That's it! Your endpoints are ready:
console.log(brain.hub.endpoints)
// { odata: '/odata', sheets: '/sheets', sse: '/events', webhooks: '/webhooks' }
```
## Supported Tools
| Tool | Protocol | How to Connect |
|------|----------|----------------|
| Excel | OData | Data → Get Data → From OData Feed → `http://your-server/odata` |
| Power BI | OData | Get Data → OData Feed → `http://your-server/odata` |
| Tableau | OData | Connect → OData → `http://your-server/odata` |
| Google Sheets | REST | Install Apps Script add-on (see below) |
| Any SSE Client | SSE | `new EventSource('http://your-server/events')` |
---
## Excel (Power Query)
### Connect in 3 clicks:
1. **Data** tab → **Get Data****From Other Sources** → **From OData Feed**
2. Enter URL: `http://your-server/odata`
3. Click **OK** → **Load**
### Query Options
Excel Power Query supports OData query parameters:
```
/odata/Entities?$filter=Type eq 'person'&$top=100
/odata/Entities?$select=Id,Type,Metadata_name
/odata/Entities?$orderby=CreatedAt desc
/odata/Entities?$search=machine learning
```
### Refresh Data
- Manual: **Data** → **Refresh All**
- Automatic: **Query Properties** → Set refresh interval
---
## Power BI
### Connect:
1. **Get Data** → **OData Feed**
2. Enter URL: `http://your-server/odata`
3. Select tables: `Entities`, `Relationships`
4. Click **Load**
### Advanced
Power BI DirectQuery is supported for real-time data.
---
## Tableau
### Connect:
1. **Connect****To a Server** → **OData**
2. Enter: `http://your-server/odata`
3. Drag tables to canvas
---
## Google Sheets
### Setup (5 minutes):
1. Open your Google Sheet
2. **Extensions** → **Apps Script**
3. Delete existing code
4. Copy `integrations/google-sheets/Code.gs` into the editor
5. Create new file `Sidebar.html`, paste from `integrations/google-sheets/Sidebar.html`
6. **Project Settings****Script properties** → Add:
- `BRAINY_URL` = `http://your-server`
7. Save and refresh spreadsheet
### Custom Functions:
```
=BRAINY_QUERY("type:person", 100) // Query entities
=BRAINY_GET("entity-id") // Get one entity
=BRAINY_SIMILAR("machine learning", 5) // Semantic search
=BRAINY_RELATIONS("entity-id", "from") // Get relationships
```
### Sidebar:
**Extensions** → **Brainy** → **Open Sidebar**
- Search and insert results
- Add new entities
- Sync selected range to Brainy
---
## Real-Time Streaming (SSE)
### JavaScript:
```javascript
const source = new EventSource('http://your-server/events')
source.onmessage = (event) => {
const data = JSON.parse(event.data)
console.log('Change:', data)
}
// Filter by type
const filtered = new EventSource('http://your-server/events?types=noun&operations=create,update')
```
### Query Parameters:
- `types` - Entity types: `noun`, `verb`, `vfs`
- `operations` - Operations: `create`, `update`, `delete`
- `nounTypes` - Filter by noun type: `person`, `document`, etc.
---
## Webhooks
Push events to external URLs:
```typescript
const brain = new Brainy({ integrations: true })
await brain.init()
// Register a webhook
await brain.hub.webhooks?.register({
url: 'https://your-app.com/webhook',
events: { entityTypes: ['noun'], operations: ['create', 'update'] },
secret: 'your-hmac-secret' // optional, for signature verification
})
```
### Webhook Payload:
```json
{
"events": [
{
"id": "event-123",
"type": "noun",
"operation": "create",
"entityId": "entity-456",
"timestamp": 1704067200000
}
],
"deliveredAt": 1704067201000
}
```
### Signature Verification:
Webhooks include `X-Brainy-Signature` header with HMAC-SHA256 signature.
---
## Server Examples
### Minimal (in-memory):
```typescript
import { Brainy } from '@soulcraft/brainy'
const brain = new Brainy({ integrations: true })
await brain.init()
// Add some data
await brain.add({ type: 'Person', metadata: { name: 'Alice' } })
// Get connection instructions
console.log(brain.hub.getInstructions())
```
### Express:
```typescript
import express from 'express'
import { Brainy } from '@soulcraft/brainy'
const app = express()
const brain = new Brainy({
storage: { type: 'filesystem', options: { basePath: './data' } },
integrations: true
})
await brain.init()
app.use(express.json())
// Route all integrations
app.all(['/odata/*', '/sheets/*', '/events/*'], async (req, res) => {
const response = await brain.hub.handleRequest({
method: req.method,
path: req.path,
query: req.query as Record<string, string>,
headers: req.headers as Record<string, string>,
body: req.body
})
res.status(response.status).set(response.headers)
if (response.body) res.json(response.body)
else res.end()
})
app.listen(3000, () => {
console.log('Brainy server ready!')
console.log('Excel/Power BI: http://localhost:3000/odata')
console.log('Google Sheets: http://localhost:3000/sheets')
console.log('SSE Stream: http://localhost:3000/events')
})
```
### Hono (Cloudflare Workers):
```typescript
import { Hono } from 'hono'
import { Brainy } from '@soulcraft/brainy'
const app = new Hono()
const brain = new Brainy({
storage: { type: 'memory' },
integrations: true
})
await brain.init()
app.all('/odata/*', async (c) => {
const response = await brain.hub.handleRequest({
method: c.req.method,
path: c.req.path,
query: Object.fromEntries(new URL(c.req.url).searchParams),
headers: Object.fromEntries(c.req.raw.headers),
body: await c.req.json().catch(() => undefined)
})
return c.json(response.body, response.status)
})
export default app
```
---
## Configuration
### Custom Base Path:
```typescript
const brain = new Brainy({
integrations: { basePath: '/api/v1' }
})
await brain.init()
// Endpoints become:
// /api/v1/odata
// /api/v1/sheets
// /api/v1/events
```
### Select Integrations:
```typescript
// Only OData and Sheets
const brain = new Brainy({
integrations: { enable: ['odata', 'sheets'] }
})
await brain.init()
```
### Per-Integration Config:
```typescript
const brain = new Brainy({
integrations: {
config: {
odata: { basePath: '/data' },
sse: { heartbeatInterval: 15000 }
}
}
})
await brain.init()
```
---
## Troubleshooting
### Excel: "Can't connect to OData feed"
- Ensure server is running and accessible
- Check firewall settings
- Try opening URL in browser first
### Google Sheets: "Brainy URL not configured"
- Add `BRAINY_URL` in Apps Script → Project Settings → Script properties
### Power BI: "Invalid OData"
- Ensure `$metadata` endpoint returns valid XML
- Try: `http://your-server/odata/$metadata`
### SSE: "Connection dropped"
- Check network/proxy timeout settings
- The integration sends heartbeats every 30 seconds by default
---
## API Reference
### OData Endpoints
```
GET /odata/$metadata - Schema (XML)
GET /odata/Entities - List entities
GET /odata/Entities('id') - Get entity
GET /odata/Relationships - List relationships
POST /odata/Entities - Create entity
```
### Sheets Endpoints
```
GET /sheets/query?q=...&limit=100 - Query entities
GET /sheets/entity/:id - Get entity
GET /sheets/similar?text=...&k=10 - Semantic search
POST /sheets/add - Add entity
```
### SSE Endpoint
```
GET /events - All events
GET /events?types=noun - Filter by type
GET /events?operations=create,update - Filter by operation
```

View file

@ -0,0 +1,451 @@
/**
* Brainy Google Sheets Add-on
*
* Custom functions and sidebar for two-way sync with Brainy.
*
* Setup:
* 1. Open Script Editor: Extensions → Apps Script
* 2. Paste this code into Code.gs
* 3. Set your Brainy URL: Tools → Script properties → Add BRAINY_URL
* 4. Refresh your spreadsheet
*
* Custom Functions:
* - =BRAINY_QUERY(query, limit) - Query entities
* - =BRAINY_GET(id) - Get entity by ID
* - =BRAINY_SIMILAR(text, limit) - Semantic search
* - =BRAINY_RELATIONS(entityId, direction) - Get relationships
* - =BRAINY_TYPES() - List available types
*
* For full documentation: https://github.com/soulcraft/brainy
*/
// Configuration
const CONFIG_BRAINY_URL = 'BRAINY_URL';
const CONFIG_API_KEY = 'BRAINY_API_KEY';
const DEFAULT_LIMIT = 100;
/**
* Get Brainy URL from script properties
*/
function getBrainyUrl() {
const props = PropertiesService.getScriptProperties();
const url = props.getProperty(CONFIG_BRAINY_URL);
if (!url) {
throw new Error('Brainy URL not configured. Go to Extensions → Apps Script → Project Settings → Script Properties and add BRAINY_URL');
}
return url.replace(/\/$/, ''); // Remove trailing slash
}
/**
* Get API key (optional)
*/
function getApiKey() {
const props = PropertiesService.getScriptProperties();
return props.getProperty(CONFIG_API_KEY) || '';
}
/**
* Make authenticated request to Brainy
*/
function brainyFetch(endpoint, options = {}) {
const baseUrl = getBrainyUrl();
const apiKey = getApiKey();
const url = `${baseUrl}/sheets${endpoint}`;
const fetchOptions = {
method: options.method || 'GET',
contentType: 'application/json',
muteHttpExceptions: true,
...options
};
if (apiKey) {
fetchOptions.headers = {
...fetchOptions.headers,
'Authorization': `Bearer ${apiKey}`
};
}
if (options.payload) {
fetchOptions.payload = JSON.stringify(options.payload);
}
const response = UrlFetchApp.fetch(url, fetchOptions);
const code = response.getResponseCode();
const text = response.getContentText();
if (code >= 400) {
const error = JSON.parse(text);
throw new Error(error.error || `HTTP ${code}`);
}
return JSON.parse(text);
}
// ============= Custom Functions =============
/**
* Query Brainy entities
*
* @param {string} query Semantic search query or type filter (e.g., "type:person")
* @param {number} limit Maximum results (default: 100)
* @return {Array} Results as rows with headers
* @customfunction
*/
function BRAINY_QUERY(query, limit) {
query = query || '';
limit = limit || DEFAULT_LIMIT;
const params = new URLSearchParams();
params.append('limit', limit);
if (query) {
// Check if it's a type filter
if (query.toLowerCase().startsWith('type:')) {
params.append('type', query.substring(5).trim());
} else {
params.append('q', query);
}
}
const result = brainyFetch(`/query?${params.toString()}`);
if (!result.rows || result.rows.length === 0) {
return [['No results found']];
}
return [result.headers, ...result.rows];
}
/**
* Get a single entity by ID
*
* @param {string} id Entity ID
* @return {Array} Entity as rows with headers
* @customfunction
*/
function BRAINY_GET(id) {
if (!id) {
return [['Error: ID required']];
}
const result = brainyFetch(`/entity/${id}`);
if (!result.rows || result.rows.length === 0) {
return [['Entity not found']];
}
return [result.headers, ...result.rows];
}
/**
* Semantic similarity search
*
* @param {string} text Text to find similar entities to
* @param {number} limit Maximum results (default: 10)
* @return {Array} Similar entities as rows with headers
* @customfunction
*/
function BRAINY_SIMILAR(text, limit) {
if (!text) {
return [['Error: Search text required']];
}
limit = limit || 10;
const params = new URLSearchParams();
params.append('q', text);
params.append('limit', limit);
const result = brainyFetch(`/similar?${params.toString()}`);
if (!result.rows || result.rows.length === 0) {
return [['No similar entities found']];
}
return [result.headers, ...result.rows];
}
/**
* Get relationships for an entity
*
* @param {string} entityId Entity ID to get relationships for
* @param {string} direction Direction: "from", "to", or "both" (default: "from")
* @param {number} limit Maximum results (default: 100)
* @return {Array} Relationships as rows with headers
* @customfunction
*/
function BRAINY_RELATIONS(entityId, direction, limit) {
if (!entityId) {
return [['Error: Entity ID required']];
}
direction = direction || 'from';
limit = limit || DEFAULT_LIMIT;
const params = new URLSearchParams();
params.append('limit', limit);
if (direction === 'from' || direction === 'both') {
params.append('from', entityId);
}
if (direction === 'to' || direction === 'both') {
params.append('to', entityId);
}
const result = brainyFetch(`/relations?${params.toString()}`);
if (!result.rows || result.rows.length === 0) {
return [['No relationships found']];
}
return [result.headers, ...result.rows];
}
/**
* List available entity types
*
* @return {Array} List of noun types
* @customfunction
*/
function BRAINY_TYPES() {
const result = brainyFetch('/schema');
return [['Type'], ...result.nounTypes.map(t => [t])];
}
/**
* List available relationship types
*
* @return {Array} List of verb types
* @customfunction
*/
function BRAINY_RELATION_TYPES() {
const result = brainyFetch('/schema');
return [['Type'], ...result.verbTypes.map(t => [t])];
}
// ============= Menu & Sidebar =============
/**
* Add menu on spreadsheet open
*/
function onOpen() {
const ui = SpreadsheetApp.getUi();
ui.createMenu('Brainy')
.addItem('Open Sidebar', 'showSidebar')
.addSeparator()
.addItem('Configure Connection', 'showConfig')
.addItem('Test Connection', 'testConnection')
.addToUi();
}
/**
* Show the Brainy sidebar
*/
function showSidebar() {
const html = HtmlService.createHtmlOutputFromFile('Sidebar')
.setTitle('Brainy')
.setWidth(300);
SpreadsheetApp.getUi().showSidebar(html);
}
/**
* Show configuration dialog
*/
function showConfig() {
const ui = SpreadsheetApp.getUi();
const props = PropertiesService.getScriptProperties();
const currentUrl = props.getProperty(CONFIG_BRAINY_URL) || '';
const response = ui.prompt(
'Configure Brainy Connection',
`Enter your Brainy server URL (current: ${currentUrl || 'not set'}):`,
ui.ButtonSet.OK_CANCEL
);
if (response.getSelectedButton() === ui.Button.OK) {
const url = response.getResponseText().trim();
if (url) {
props.setProperty(CONFIG_BRAINY_URL, url);
ui.alert('Configuration saved! URL: ' + url);
}
}
}
/**
* Test the connection
*/
function testConnection() {
const ui = SpreadsheetApp.getUi();
try {
const result = brainyFetch('/health');
ui.alert('Connection successful!\n\nStatus: ' + result.status + '\nUptime: ' + Math.round(result.uptime / 1000) + 's');
} catch (e) {
ui.alert('Connection failed!\n\n' + e.message);
}
}
// ============= Sidebar Functions =============
/**
* Get connection status for sidebar
*/
function getConnectionStatus() {
try {
const result = brainyFetch('/health');
return {
connected: true,
url: getBrainyUrl(),
status: result.status,
uptime: result.uptime
};
} catch (e) {
return {
connected: false,
error: e.message
};
}
}
/**
* Query entities from sidebar
*/
function sidebarQuery(query, type, limit) {
const params = new URLSearchParams();
params.append('limit', limit || 50);
if (query) params.append('q', query);
if (type) params.append('type', type);
return brainyFetch(`/query?${params.toString()}`);
}
/**
* Add entity from sidebar
*/
function sidebarAddEntity(type, data, metadata) {
return brainyFetch('/add', {
method: 'POST',
payload: { type, data, metadata }
});
}
/**
* Update entity from sidebar
*/
function sidebarUpdateEntity(id, data, metadata) {
return brainyFetch('/update', {
method: 'POST',
payload: { id, data, metadata, merge: true }
});
}
/**
* Delete entity from sidebar
*/
function sidebarDeleteEntity(id) {
return brainyFetch('/delete', {
method: 'POST',
payload: { id }
});
}
/**
* Get schema for sidebar dropdowns
*/
function sidebarGetSchema() {
return brainyFetch('/schema');
}
/**
* Insert query result into sheet at selection
*/
function insertQueryResult(query, type, limit) {
const sheet = SpreadsheetApp.getActiveSheet();
const selection = sheet.getActiveCell();
const result = sidebarQuery(query, type, limit);
if (!result.rows || result.rows.length === 0) {
selection.setValue('No results');
return;
}
const data = [result.headers, ...result.rows];
const range = sheet.getRange(
selection.getRow(),
selection.getColumn(),
data.length,
data[0].length
);
range.setValues(data);
return {
inserted: result.rows.length,
columns: result.headers.length
};
}
/**
* Sync selected range to Brainy
*/
function syncRangeToBrainy(type) {
const sheet = SpreadsheetApp.getActiveSheet();
const range = sheet.getActiveRange();
const values = range.getValues();
if (values.length < 2) {
throw new Error('Need at least header row and one data row');
}
const headers = values[0];
const operations = [];
// Find ID column
const idCol = headers.findIndex(h => h.toLowerCase() === 'id');
for (let i = 1; i < values.length; i++) {
const row = values[i];
const metadata = {};
for (let j = 0; j < headers.length; j++) {
if (j !== idCol && row[j]) {
metadata[headers[j]] = row[j];
}
}
if (idCol >= 0 && row[idCol]) {
// Update existing
operations.push({
action: 'update',
id: row[idCol],
metadata
});
} else {
// Add new
operations.push({
action: 'add',
type: type,
metadata
});
}
}
const result = brainyFetch('/batch', {
method: 'POST',
payload: { operations }
});
// Update IDs in sheet for new entities
if (idCol >= 0) {
for (let i = 0; i < result.results.length; i++) {
if (result.results[i].action === 'add' && result.results[i].id) {
sheet.getRange(range.getRow() + 1 + i, range.getColumn() + idCol).setValue(result.results[i].id);
}
}
}
return result;
}

View file

@ -0,0 +1,142 @@
# Brainy Google Sheets Add-on
Two-way sync between Brainy and Google Sheets.
## Quick Setup
1. Open your Google Sheet
2. Go to **Extensions → Apps Script**
3. Delete any existing code
4. Copy and paste `Code.gs` into the editor
5. Create a new HTML file called `Sidebar.html` and paste the content
6. Go to **Project Settings** (gear icon) → **Script properties**
7. Add property: `BRAINY_URL` = your Brainy server URL (e.g., `http://localhost:3000`)
8. Save and refresh your spreadsheet
## Custom Functions
Use these directly in cells:
### `=BRAINY_QUERY(query, limit)`
Query entities using semantic search or type filter.
```
=BRAINY_QUERY("machine learning", 10)
=BRAINY_QUERY("type:person", 100)
```
### `=BRAINY_GET(id)`
Get a single entity by ID.
```
=BRAINY_GET("entity-uuid-here")
```
### `=BRAINY_SIMILAR(text, limit)`
Find semantically similar entities.
```
=BRAINY_SIMILAR("artificial intelligence research", 5)
```
### `=BRAINY_RELATIONS(entityId, direction)`
Get relationships for an entity.
```
=BRAINY_RELATIONS("entity-id", "from")
=BRAINY_RELATIONS("entity-id", "to")
=BRAINY_RELATIONS("entity-id", "both")
```
### `=BRAINY_TYPES()`
List all available entity types.
### `=BRAINY_RELATION_TYPES()`
List all available relationship types.
## Sidebar
Open via **Brainy → Open Sidebar** menu:
- **Search**: Query entities and insert results into the sheet
- **Add**: Create new entities with a form
- **Sync**: Sync selected range to Brainy (add or update)
## Authentication
If your Brainy server requires authentication:
1. Go to **Project Settings → Script properties**
2. Add property: `BRAINY_API_KEY` = your API key
## Real-Time Sync
For real-time updates, enable SSE in your Brainy server:
```javascript
brain.augmentations.register(new SSEIntegration())
```
The sidebar will automatically show changes as they happen.
## Troubleshooting
### "Brainy URL not configured"
Add the `BRAINY_URL` script property in Apps Script settings.
### "Connection failed"
- Check that your Brainy server is running
- Ensure the URL is correct (include port if needed)
- For local development, use ngrok or similar to expose localhost
### Custom functions not appearing
- Refresh the spreadsheet
- Wait a few seconds (Apps Script can be slow to register)
- Check the Apps Script execution log for errors
## Server Setup
The simplest way to enable all integrations:
```javascript
import { Brainy } from '@soulcraft/brainy'
const brain = new Brainy({ integrations: true })
await brain.init()
console.log(brain.hub.endpoints)
// { odata: '/odata', sheets: '/sheets', sse: '/events', webhooks: '/webhooks' }
```
With Express:
```javascript
import express from 'express'
import { Brainy } from '@soulcraft/brainy'
const app = express()
const brain = new Brainy({ integrations: true })
await brain.init()
// Route all integration requests
app.use(express.json())
app.all(['/odata/*', '/sheets/*', '/events/*'], async (req, res) => {
const response = await brain.hub.handleRequest({
method: req.method,
path: req.path,
query: req.query,
headers: req.headers,
body: req.body
})
if (response.isSSE) {
// Handle SSE stream
res.setHeader('Content-Type', 'text/event-stream')
// ... streaming logic
} else {
res.status(response.status).set(response.headers).json(response.body)
}
})
app.listen(3000)
```

View file

@ -0,0 +1,421 @@
<!DOCTYPE html>
<html>
<head>
<base target="_top">
<style>
* {
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, sans-serif;
padding: 12px;
margin: 0;
font-size: 13px;
color: #333;
}
.header {
display: flex;
align-items: center;
margin-bottom: 16px;
padding-bottom: 12px;
border-bottom: 1px solid #e0e0e0;
}
.header h1 {
margin: 0;
font-size: 18px;
font-weight: 500;
}
.status {
margin-left: auto;
width: 10px;
height: 10px;
border-radius: 50%;
background: #ccc;
}
.status.connected {
background: #34a853;
}
.status.disconnected {
background: #ea4335;
}
.section {
margin-bottom: 16px;
}
.section-title {
font-weight: 500;
margin-bottom: 8px;
color: #5f6368;
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.5px;
}
input, select, textarea {
width: 100%;
padding: 8px 10px;
border: 1px solid #dadce0;
border-radius: 4px;
font-size: 13px;
margin-bottom: 8px;
}
input:focus, select:focus, textarea:focus {
outline: none;
border-color: #4285f4;
box-shadow: 0 0 0 2px rgba(66, 133, 244, 0.2);
}
button {
padding: 8px 16px;
border: none;
border-radius: 4px;
font-size: 13px;
cursor: pointer;
transition: background 0.2s;
}
button.primary {
background: #4285f4;
color: white;
}
button.primary:hover {
background: #3367d6;
}
button.secondary {
background: #f1f3f4;
color: #5f6368;
}
button.secondary:hover {
background: #e8eaed;
}
button.danger {
background: #ea4335;
color: white;
}
.btn-row {
display: flex;
gap: 8px;
}
.results {
margin-top: 12px;
max-height: 300px;
overflow-y: auto;
}
.result-item {
padding: 10px;
background: #f8f9fa;
border-radius: 4px;
margin-bottom: 6px;
cursor: pointer;
transition: background 0.2s;
}
.result-item:hover {
background: #e8f0fe;
}
.result-item .type {
font-size: 10px;
color: #5f6368;
text-transform: uppercase;
margin-bottom: 2px;
}
.result-item .title {
font-weight: 500;
color: #1967d2;
}
.result-item .id {
font-size: 11px;
color: #80868b;
font-family: monospace;
}
.tabs {
display: flex;
border-bottom: 1px solid #e0e0e0;
margin-bottom: 12px;
}
.tab {
padding: 8px 16px;
cursor: pointer;
color: #5f6368;
border-bottom: 2px solid transparent;
transition: all 0.2s;
}
.tab.active {
color: #4285f4;
border-bottom-color: #4285f4;
}
.tab-content {
display: none;
}
.tab-content.active {
display: block;
}
.loading {
text-align: center;
color: #5f6368;
padding: 20px;
}
.error {
background: #fce8e6;
color: #c5221f;
padding: 10px;
border-radius: 4px;
margin-bottom: 12px;
}
.success {
background: #e6f4ea;
color: #137333;
padding: 10px;
border-radius: 4px;
margin-bottom: 12px;
}
</style>
</head>
<body>
<div class="header">
<h1>🧠 Brainy</h1>
<div class="status" id="status"></div>
</div>
<div id="message"></div>
<div class="tabs">
<div class="tab active" data-tab="search">Search</div>
<div class="tab" data-tab="add">Add</div>
<div class="tab" data-tab="sync">Sync</div>
</div>
<!-- Search Tab -->
<div class="tab-content active" id="tab-search">
<div class="section">
<input type="text" id="searchQuery" placeholder="Search entities...">
<select id="searchType">
<option value="">All Types</option>
</select>
<div class="btn-row">
<button class="primary" onclick="search()">Search</button>
<button class="secondary" onclick="insertResults()">Insert to Sheet</button>
</div>
</div>
<div class="results" id="searchResults">
<div class="loading">Enter a search query</div>
</div>
</div>
<!-- Add Tab -->
<div class="tab-content" id="tab-add">
<div class="section">
<div class="section-title">Entity Type</div>
<select id="addType"></select>
</div>
<div class="section">
<div class="section-title">Name</div>
<input type="text" id="addName" placeholder="Entity name">
</div>
<div class="section">
<div class="section-title">Description</div>
<textarea id="addDescription" rows="3" placeholder="Description (optional)"></textarea>
</div>
<div class="section">
<div class="section-title">Additional Fields (JSON)</div>
<textarea id="addMetadata" rows="3" placeholder='{"email": "john@example.com"}'></textarea>
</div>
<button class="primary" onclick="addEntity()">Add Entity</button>
</div>
<!-- Sync Tab -->
<div class="tab-content" id="tab-sync">
<div class="section">
<p>Select a range with entity data and sync it to Brainy.</p>
<p style="font-size: 11px; color: #5f6368;">
First row should be headers. Include an "Id" column for updates.
</p>
</div>
<div class="section">
<div class="section-title">Type for New Entities</div>
<select id="syncType"></select>
</div>
<button class="primary" onclick="syncSelection()">Sync Selection</button>
</div>
<script>
// State
let schema = { nounTypes: [], verbTypes: [] };
let searchResults = [];
// Initialize
document.addEventListener('DOMContentLoaded', () => {
checkConnection();
loadSchema();
setupTabs();
});
function setupTabs() {
document.querySelectorAll('.tab').forEach(tab => {
tab.addEventListener('click', () => {
document.querySelectorAll('.tab').forEach(t => t.classList.remove('active'));
document.querySelectorAll('.tab-content').forEach(c => c.classList.remove('active'));
tab.classList.add('active');
document.getElementById('tab-' + tab.dataset.tab).classList.add('active');
});
});
}
function checkConnection() {
google.script.run
.withSuccessHandler(result => {
const status = document.getElementById('status');
status.className = 'status ' + (result.connected ? 'connected' : 'disconnected');
})
.withFailureHandler(err => {
document.getElementById('status').className = 'status disconnected';
})
.getConnectionStatus();
}
function loadSchema() {
google.script.run
.withSuccessHandler(result => {
schema = result;
populateTypeDropdowns();
})
.sidebarGetSchema();
}
function populateTypeDropdowns() {
const selects = ['searchType', 'addType', 'syncType'];
selects.forEach(id => {
const select = document.getElementById(id);
const currentValue = select.value;
select.innerHTML = id === 'searchType' ? '<option value="">All Types</option>' : '';
schema.nounTypes.forEach(type => {
const option = document.createElement('option');
option.value = type;
option.textContent = type.charAt(0).toUpperCase() + type.slice(1);
select.appendChild(option);
});
if (currentValue) select.value = currentValue;
});
}
function showMessage(text, type) {
const el = document.getElementById('message');
el.innerHTML = `<div class="${type}">${text}</div>`;
setTimeout(() => el.innerHTML = '', 3000);
}
function search() {
const query = document.getElementById('searchQuery').value;
const type = document.getElementById('searchType').value;
const resultsEl = document.getElementById('searchResults');
resultsEl.innerHTML = '<div class="loading">Searching...</div>';
google.script.run
.withSuccessHandler(result => {
searchResults = result;
renderResults(result);
})
.withFailureHandler(err => {
resultsEl.innerHTML = `<div class="error">${err.message}</div>`;
})
.sidebarQuery(query, type, 50);
}
function renderResults(result) {
const resultsEl = document.getElementById('searchResults');
if (!result.rows || result.rows.length === 0) {
resultsEl.innerHTML = '<div class="loading">No results found</div>';
return;
}
// Find column indices
const headers = result.headers;
const idCol = headers.indexOf('Id');
const typeCol = headers.indexOf('Type');
const nameCol = headers.findIndex(h => h.toLowerCase().includes('name'));
resultsEl.innerHTML = result.rows.map((row, i) => `
<div class="result-item" onclick="selectResult(${i})">
<div class="type">${row[typeCol] || 'unknown'}</div>
<div class="title">${row[nameCol] || row[idCol] || 'Untitled'}</div>
<div class="id">${row[idCol]}</div>
</div>
`).join('');
}
function selectResult(index) {
const row = searchResults.rows[index];
const headers = searchResults.headers;
// Show details or insert into sheet
console.log('Selected:', row);
}
function insertResults() {
const query = document.getElementById('searchQuery').value;
const type = document.getElementById('searchType').value;
google.script.run
.withSuccessHandler(result => {
showMessage(`Inserted ${result.inserted} rows`, 'success');
})
.withFailureHandler(err => {
showMessage(err.message, 'error');
})
.insertQueryResult(query, type, 100);
}
function addEntity() {
const type = document.getElementById('addType').value;
const name = document.getElementById('addName').value;
const description = document.getElementById('addDescription').value;
const metadataStr = document.getElementById('addMetadata').value;
if (!type) {
showMessage('Please select a type', 'error');
return;
}
let metadata = { name };
if (description) metadata.description = description;
try {
if (metadataStr) {
const extra = JSON.parse(metadataStr);
metadata = { ...metadata, ...extra };
}
} catch (e) {
showMessage('Invalid JSON in additional fields', 'error');
return;
}
google.script.run
.withSuccessHandler(result => {
showMessage('Entity created!', 'success');
document.getElementById('addName').value = '';
document.getElementById('addDescription').value = '';
document.getElementById('addMetadata').value = '';
})
.withFailureHandler(err => {
showMessage(err.message, 'error');
})
.sidebarAddEntity(type, null, metadata);
}
function syncSelection() {
const type = document.getElementById('syncType').value;
if (!type) {
showMessage('Please select a type for new entities', 'error');
return;
}
google.script.run
.withSuccessHandler(result => {
showMessage(`Synced ${result.successful} entities (${result.failed} failed)`, 'success');
})
.withFailureHandler(err => {
showMessage(err.message, 'error');
})
.syncRangeToBrainy(type);
}
</script>
</body>
</html>

View file

@ -0,0 +1,11 @@
{
"timeZone": "America/New_York",
"dependencies": {},
"exceptionLogging": "STACKDRIVER",
"runtimeVersion": "V8",
"oauthScopes": [
"https://www.googleapis.com/auth/spreadsheets.currentonly",
"https://www.googleapis.com/auth/script.container.ui",
"https://www.googleapis.com/auth/script.external_request"
]
}

View file

@ -84,6 +84,7 @@ import {
} from './types/brainy.types.js'
import { NounType, VerbType } from './types/graphTypes.js'
import { BrainyInterface } from './types/brainyInterface.js'
import type { IntegrationHub } from './integrations/core/IntegrationHub.js'
/**
* The main Brainy class - Clean, Beautiful, Powerful
@ -129,6 +130,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
private _versions?: VersioningAPI
private _vfs?: VirtualFileSystem
private _vfsInitialized = false // v7.3.0: Track VFS init completion separately
private _hub?: IntegrationHub // v7.4.0: Integration Hub for external tools
// State
private initialized = false
@ -322,6 +324,22 @@ export class Brainy<T = any> implements BrainyInterface<T> {
console.log('✅ Embedding engine ready')
}
// v7.4.0: Integration Hub initialization
// Creates the hub when integrations are enabled in config
// Uses dynamic import for tree-shaking when integrations are disabled
if (this.config.integrations) {
const hubConfig = this.config.integrations === true
? { enable: 'all' as const }
: this.config.integrations
const { IntegrationHub } = await import('./integrations/core/IntegrationHub.js')
this._hub = await IntegrationHub.create(this, {
basePath: hubConfig.basePath,
enable: hubConfig.enable,
config: hubConfig.config as any // Type flexibility for user config
})
}
// v7.3.0: Resolve ready Promise - consumers awaiting brain.ready will now proceed
if (this._readyResolve) {
this._readyResolve()
@ -3852,6 +3870,52 @@ export class Brainy<T = any> implements BrainyInterface<T> {
return this._vfs
}
/**
* Integration Hub for external tools (Excel, Power BI, Google Sheets)
*
* Provides HTTP endpoints that external tools can connect to:
* - OData API for Excel Power Query, Power BI, Tableau
* - REST API for Google Sheets custom functions
* - SSE streaming for real-time dashboards
* - Webhooks for push notifications
*
* Only available when `integrations: true` is set in config.
*
* @example Basic usage
* ```typescript
* const brain = new Brainy({ integrations: true })
* await brain.init()
*
* // Get endpoint URLs
* console.log(brain.hub.endpoints)
* // { odata: '/odata', sheets: '/sheets', sse: '/events', webhooks: '/webhooks' }
*
* // Handle requests (use with Express, Hono, etc.)
* app.all('/odata/*', async (req, res) => {
* const response = await brain.hub.handleRequest({
* method: req.method,
* path: req.path,
* query: req.query,
* headers: req.headers,
* body: req.body
* })
* res.status(response.status).set(response.headers).json(response.body)
* })
* ```
*
* @since v7.4.0
* @throws Error if integrations are not enabled in config
*/
get hub(): IntegrationHub {
if (!this._hub) {
throw new Error(
'Integration Hub not enabled. Set integrations: true in config:\n' +
'new Brainy({ integrations: true })'
)
}
return this._hub
}
/**
* Data Management API - backup, restore, import, export
*/
@ -5692,7 +5756,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
// HNSW persistence mode (v6.2.8) - undefined = smart default in setupIndex
hnswPersistMode: config?.hnswPersistMode ?? undefined as any,
// Embedding initialization (v7.1.2) - false = lazy init on first embed()
eagerEmbeddings: config?.eagerEmbeddings ?? false
eagerEmbeddings: config?.eagerEmbeddings ?? false,
// Integration Hub (v7.4.0) - undefined/false = disabled
integrations: config?.integrations ?? undefined as any
}
}

View file

@ -542,3 +542,86 @@ export type {
MCPServiceOptions,
MCPTool
}
// ============= Integration Hub (v7.4.0) =============
// Connect Brainy to Excel, Power BI, Google Sheets, and more
// Enable with: new Brainy({ integrations: true })
// Hub class (used internally by brain.hub, also available for advanced use)
export {
IntegrationHub,
createIntegrationHub
} from './integrations/index.js'
export type {
IntegrationHubConfig,
IntegrationRequest,
IntegrationResponse
} from './integrations/index.js'
// Re-export IntegrationsConfig from types (for TypeScript users)
export type { IntegrationsConfig } from './types/brainy.types.js'
// Core infrastructure
export {
EventBus,
TabularExporter,
IntegrationBase,
IntegrationLoader,
createIntegrationLoader,
detectEnvironment,
INTEGRATION_CATALOG
} from './integrations/index.js'
// Integration types
export type {
BrainyEvent,
EventFilter,
EventHandler,
EventSubscription,
TabularRow,
RelationTabularRow,
TabularExporterConfig,
IntegrationConfig,
IntegrationHealthStatus,
HTTPIntegration,
StreamingIntegration,
IntegrationType,
RuntimeEnvironment,
IntegrationInfo,
IntegrationLoaderConfig,
ODataQueryOptions,
WebhookRegistration,
WebhookDeliveryResult
} from './integrations/index.js'
// Concrete integrations
export {
GoogleSheetsIntegration,
ODataIntegration,
SSEIntegration,
WebhookIntegration
} from './integrations/index.js'
export type {
GoogleSheetsConfig,
ODataConfig,
SSEConfig,
WebhookConfig
} from './integrations/index.js'
// OData utilities (advanced)
export {
parseODataQuery,
parseFilter,
parseOrderBy,
parseSelect,
odataToFindParams,
applyFilter,
applySelect,
applyOrderBy,
applyPagination,
generateEdmx,
generateMetadataJson,
generateServiceDocument
} from './integrations/index.js'

View file

@ -0,0 +1,313 @@
/**
* Integration Hub - Event Bus
*
* Central event emitter for real-time change propagation.
* Enables integrations to react to Brainy data changes.
*/
import {
BrainyEvent,
EventFilter,
EventHandler,
EventSubscription
} from './types.js'
import { Entity, Relation } from '../../types/brainy.types.js'
import { NounType, VerbType } from '../../types/graphTypes.js'
/**
* Central event bus for real-time Brainy events
*
* Features:
* - Pub/sub pattern for event distribution
* - Filtering by entity type, operation, noun/verb types
* - Sequence IDs for ordering and resumption
* - Optional event buffering for batch processing
* - Memory-efficient circular buffer for replay
*
* @example
* ```typescript
* const eventBus = new EventBus()
*
* // Subscribe to all noun creates
* eventBus.subscribe(
* { entityTypes: ['noun'], operations: ['create'] },
* (event) => console.log('New entity:', event.entityId)
* )
*
* // Emit event
* eventBus.emit({
* entityType: 'noun',
* operation: 'create',
* entityId: 'entity-123',
* nounType: NounType.Person
* })
* ```
*/
export class EventBus {
private subscriptions: Map<
string,
{ filter: EventFilter; handler: EventHandler }
> = new Map()
private sequenceCounter: bigint = 0n
private eventBuffer: BrainyEvent[] = []
private bufferSize: number
private subscriptionIdCounter = 0
/**
* Create a new EventBus
*
* @param options Configuration options
* @param options.bufferSize Size of replay buffer (default: 1000)
*/
constructor(options: { bufferSize?: number } = {}) {
this.bufferSize = options.bufferSize ?? 1000
}
/**
* Subscribe to events matching a filter
*
* @param filter Event filter criteria
* @param handler Function to call when matching events occur
* @returns Subscription that can be used to unsubscribe
*/
subscribe(filter: EventFilter, handler: EventHandler): EventSubscription {
const id = `sub-${++this.subscriptionIdCounter}`
this.subscriptions.set(id, { filter, handler })
// If filter has 'since', replay buffered events
if (filter.since !== undefined) {
this.replayEvents(filter, handler)
}
return {
id,
unsubscribe: () => {
this.subscriptions.delete(id)
}
}
}
/**
* Emit an event to all matching subscribers
*
* @param partialEvent Event data (id, timestamp, sequenceId auto-generated)
*/
emit(
partialEvent: Omit<BrainyEvent, 'id' | 'timestamp' | 'sequenceId'>
): BrainyEvent {
const event: BrainyEvent = {
...partialEvent,
id: this.generateEventId(),
timestamp: Date.now(),
sequenceId: ++this.sequenceCounter
}
// Add to buffer
this.addToBuffer(event)
// Dispatch to matching subscribers
this.dispatch(event)
return event
}
/**
* Emit a noun event
*/
emitNoun(
operation: 'create' | 'update' | 'delete',
entityId: string,
nounType: NounType,
options?: { service?: string; data?: Entity }
): BrainyEvent {
return this.emit({
entityType: 'noun',
operation,
entityId,
nounType,
service: options?.service,
data: options?.data
})
}
/**
* Emit a verb/relation event
*/
emitVerb(
operation: 'create' | 'update' | 'delete',
entityId: string,
verbType: VerbType,
options?: { service?: string; data?: Relation }
): BrainyEvent {
return this.emit({
entityType: 'verb',
operation,
entityId,
verbType,
service: options?.service,
data: options?.data
})
}
/**
* Emit a VFS event
*/
emitVFS(
operation: 'create' | 'update' | 'delete',
entityId: string,
options?: { service?: string; data?: Entity }
): BrainyEvent {
return this.emit({
entityType: 'vfs',
operation,
entityId,
service: options?.service,
data: options?.data
})
}
/**
* Get current sequence ID for resumption
*/
getCurrentSequenceId(): bigint {
return this.sequenceCounter
}
/**
* Get events since a sequence ID (from buffer)
*/
getEventsSince(sequenceId: bigint): BrainyEvent[] {
return this.eventBuffer.filter((event) => event.sequenceId > sequenceId)
}
/**
* Get subscription count
*/
getSubscriptionCount(): number {
return this.subscriptions.size
}
/**
* Clear all subscriptions
*/
clear(): void {
this.subscriptions.clear()
this.eventBuffer = []
this.sequenceCounter = 0n
}
/**
* Check if an event matches a filter
*/
private matchesFilter(event: BrainyEvent, filter: EventFilter): boolean {
// Check entity types
if (
filter.entityTypes &&
filter.entityTypes.length > 0 &&
!filter.entityTypes.includes(event.entityType)
) {
return false
}
// Check operations
if (
filter.operations &&
filter.operations.length > 0 &&
!filter.operations.includes(event.operation)
) {
return false
}
// Check noun types
if (
filter.nounTypes &&
filter.nounTypes.length > 0 &&
event.nounType &&
!filter.nounTypes.includes(event.nounType)
) {
return false
}
// Check verb types
if (
filter.verbTypes &&
filter.verbTypes.length > 0 &&
event.verbType &&
!filter.verbTypes.includes(event.verbType)
) {
return false
}
// Check service
if (filter.service && event.service !== filter.service) {
return false
}
// Check sequence ID
if (filter.since !== undefined && event.sequenceId <= filter.since) {
return false
}
return true
}
/**
* Dispatch event to matching subscribers
*/
private async dispatch(event: BrainyEvent): Promise<void> {
const promises: Promise<void>[] = []
for (const [_, subscription] of this.subscriptions) {
if (this.matchesFilter(event, subscription.filter)) {
const result = subscription.handler(event)
if (result instanceof Promise) {
promises.push(result)
}
}
}
// Wait for all async handlers (fire and forget for sync handlers)
if (promises.length > 0) {
await Promise.allSettled(promises)
}
}
/**
* Replay buffered events to a new subscriber
*/
private async replayEvents(
filter: EventFilter,
handler: EventHandler
): Promise<void> {
const eventsToReplay = this.eventBuffer.filter((event) =>
this.matchesFilter(event, filter)
)
for (const event of eventsToReplay) {
const result = handler(event)
if (result instanceof Promise) {
await result
}
}
}
/**
* Add event to circular buffer
*/
private addToBuffer(event: BrainyEvent): void {
this.eventBuffer.push(event)
// Maintain buffer size
while (this.eventBuffer.length > this.bufferSize) {
this.eventBuffer.shift()
}
}
/**
* Generate unique event ID
*/
private generateEventId(): string {
return `evt-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 9)}`
}
}

View file

@ -0,0 +1,425 @@
/**
* Integration Hub - Integration Base Class
*
* Base class for all integration augmentations. Provides common functionality
* for event subscriptions, tabular export, and lifecycle management.
*/
import {
BaseAugmentation,
AugmentationContext
} from '../../augmentations/brainyAugmentation.js'
import { AugmentationManifest } from '../../augmentations/manifest.js'
import { EventBus } from './EventBus.js'
import { TabularExporter } from './TabularExporter.js'
import {
EventFilter,
EventHandler,
EventSubscription,
IntegrationConfig,
IntegrationHealthStatus,
TabularExporterConfig
} from './types.js'
import { Entity, Relation, FindParams } from '../../types/brainy.types.js'
/**
* Base class for all integration augmentations
*
* Provides:
* - Shared EventBus for real-time updates
* - TabularExporter for entity-to-row conversion
* - Common lifecycle methods (start/stop)
* - Health monitoring
* - Standard configuration patterns
*
* @example
* ```typescript
* class MySQLIntegration extends IntegrationBase {
* readonly name = 'sql'
* readonly category = 'integration'
*
* protected async onStart(): Promise<void> {
* // Start SQL server
* }
*
* protected async onStop(): Promise<void> {
* // Stop SQL server
* }
* }
* ```
*/
export abstract class IntegrationBase extends BaseAugmentation {
// Augmentation interface implementation
readonly timing = 'after' as const
readonly metadata = 'readonly' as const
readonly operations: ('all')[] = ['all']
readonly priority = 5 // Low priority - runs after main operations
category: 'internal' | 'core' | 'premium' | 'community' | 'external' = 'core'
// Shared infrastructure
protected eventBus: EventBus
protected exporter: TabularExporter
// Integration state
protected isRunning = false
protected startedAt?: number
protected requestCount = 0
protected errorCount = 0
protected lastError?: string
// Event subscriptions managed by this integration
private managedSubscriptions: EventSubscription[] = []
/**
* Create a new integration
*
* @param config Integration configuration
* @param exporterConfig Optional TabularExporter configuration
*/
constructor(
config?: IntegrationConfig,
exporterConfig?: TabularExporterConfig
) {
super(config)
this.eventBus = new EventBus()
this.exporter = new TabularExporter(exporterConfig)
}
/**
* Integration name (must be unique)
*/
abstract readonly name: string
/**
* Start the integration (implement in subclass)
*/
protected abstract onStart(): Promise<void>
/**
* Stop the integration (implement in subclass)
*/
protected abstract onStop(): Promise<void>
// BaseAugmentation lifecycle integration
protected async onInitialize(): Promise<void> {
// Auto-start if enabled
if (this.config.enabled !== false) {
await this.start()
}
}
protected async onShutdown(): Promise<void> {
await this.stop()
}
/**
* Execute augmentation (intercept operations to emit events)
*/
async execute<T>(
operation: string,
params: any,
next: () => Promise<T>
): Promise<T> {
// Execute main operation first
const result = await next()
// Emit events for data-changing operations
if (this.isRunning) {
this.emitOperationEvent(operation, params, result)
}
return result
}
// Public API
/**
* Start the integration
*/
async start(): Promise<void> {
if (this.isRunning) {
return
}
this.log(`Starting ${this.name} integration...`)
try {
await this.onStart()
this.isRunning = true
this.startedAt = Date.now()
this.log(`${this.name} integration started`)
} catch (error: any) {
this.lastError = error.message
this.log(`Failed to start ${this.name}: ${error.message}`, 'error')
throw error
}
}
/**
* Stop the integration
*/
async stop(): Promise<void> {
if (!this.isRunning) {
return
}
this.log(`Stopping ${this.name} integration...`)
try {
// Unsubscribe all managed subscriptions
for (const sub of this.managedSubscriptions) {
sub.unsubscribe()
}
this.managedSubscriptions = []
await this.onStop()
this.isRunning = false
this.log(`${this.name} integration stopped`)
} catch (error: any) {
this.lastError = error.message
this.log(`Error stopping ${this.name}: ${error.message}`, 'error')
throw error
}
}
/**
* Check if integration is running
*/
running(): boolean {
return this.isRunning
}
/**
* Get health status
*/
health(): IntegrationHealthStatus {
return {
name: this.name,
status: this.isRunning
? this.errorCount > 10
? 'degraded'
: 'healthy'
: 'stopped',
message: this.isRunning ? 'Running' : 'Stopped',
uptimeMs: this.startedAt ? Date.now() - this.startedAt : undefined,
requestCount: this.requestCount,
errorCount: this.errorCount,
lastError: this.lastError,
checkedAt: Date.now()
}
}
/**
* Get the shared EventBus
*/
getEventBus(): EventBus {
return this.eventBus
}
/**
* Get the TabularExporter
*/
getExporter(): TabularExporter {
return this.exporter
}
// Protected helpers for subclasses
/**
* Subscribe to Brainy events (auto-unsubscribed on stop)
*/
protected subscribeToChanges(
filter: EventFilter,
handler: EventHandler
): EventSubscription {
const subscription = this.eventBus.subscribe(filter, handler)
this.managedSubscriptions.push(subscription)
return subscription
}
/**
* Query entities using Brainy find()
*/
protected async queryEntities(params: FindParams): Promise<Entity[]> {
if (!this.context) {
throw new Error('Integration not initialized')
}
const results = await this.context.brain.find(params)
return results.map((r: any) => r.entity)
}
/**
* Query relations using Brainy getRelations()
*/
protected async queryRelations(params?: {
from?: string
to?: string
type?: any
limit?: number
offset?: number
}): Promise<Relation[]> {
if (!this.context) {
throw new Error('Integration not initialized')
}
return this.context.brain.getRelations(params)
}
/**
* Get a single entity by ID
*/
protected async getEntity(id: string): Promise<Entity | null> {
if (!this.context) {
throw new Error('Integration not initialized')
}
return this.context.brain.get(id)
}
/**
* Export entities to tabular format
*/
protected exportEntities(entities: Entity[]): any[] {
return this.exporter.entitiesToRows(entities)
}
/**
* Export relations to tabular format
*/
protected exportRelations(relations: Relation[]): any[] {
return this.exporter.relationsToRows(relations)
}
/**
* Record a successful request
*/
protected recordRequest(): void {
this.requestCount++
}
/**
* Record an error
*/
protected recordError(error: Error | string): void {
this.errorCount++
this.lastError = typeof error === 'string' ? error : error.message
}
/**
* Get manifest for this integration
* Subclasses should override this
*/
getManifest(): AugmentationManifest {
return {
id: this.name,
name: this.name,
version: '1.0.0',
description: `${this.name} integration`,
category: 'integration',
status: 'stable',
configSchema: {
type: 'object',
properties: {
enabled: {
type: 'boolean',
default: true,
description: 'Whether the integration is enabled'
}
}
},
configDefaults: {
enabled: true
}
}
}
// Private helpers
/**
* Emit events based on operation type
*/
private emitOperationEvent(operation: string, params: any, result: any): void {
// Map operations to event types
const opMap: Record<
string,
{ entityType: 'noun' | 'verb' | 'vfs'; op: 'create' | 'update' | 'delete' }
> = {
add: { entityType: 'noun', op: 'create' },
addNoun: { entityType: 'noun', op: 'create' },
update: { entityType: 'noun', op: 'update' },
delete: { entityType: 'noun', op: 'delete' },
relate: { entityType: 'verb', op: 'create' },
addVerb: { entityType: 'verb', op: 'create' },
unrelate: { entityType: 'verb', op: 'delete' },
deleteVerb: { entityType: 'verb', op: 'delete' }
}
const mapping = opMap[operation]
if (!mapping) {
return
}
// Extract entity ID from result or params
let entityId = result?.id || params?.id
if (!entityId && Array.isArray(result)) {
// Batch operation - emit for each
for (const item of result) {
if (item?.id) {
this.eventBus.emit({
entityType: mapping.entityType,
operation: mapping.op,
entityId: item.id,
nounType: params?.type || item?.type,
verbType: params?.type || item?.type,
service: params?.service || item?.service,
data: item
})
}
}
return
}
if (entityId) {
this.eventBus.emit({
entityType: mapping.entityType,
operation: mapping.op,
entityId,
nounType: params?.type,
verbType: params?.type,
service: params?.service,
data: result
})
}
}
}
/**
* Interface for integrations that expose HTTP endpoints
*/
export interface HTTPIntegration {
/** Port the server is listening on */
port: number
/** Base path for routes */
basePath: string
/** Get registered routes */
getRoutes(): Array<{
method: string
path: string
description: string
}>
}
/**
* Interface for integrations that support streaming
*/
export interface StreamingIntegration {
/** Subscribe to real-time events via callback */
stream(
filter: EventFilter,
callback: (event: any) => void
): { close: () => void }
}

View file

@ -0,0 +1,368 @@
/**
* Integration Hub - Zero-Config Integration Manager
*
* The simplest way to enable external tool integrations.
* One line of code, all integrations ready.
*
* @example
* ```typescript
* // Zero-config: Enable all integrations
* const hub = await IntegrationHub.create(brain)
*
* // Get your endpoints
* console.log(hub.endpoints)
* // {
* // odata: '/odata', → Excel, Power BI, Tableau
* // sheets: '/sheets', → Google Sheets
* // sse: '/events', → Real-time streaming
* // webhooks: '/webhooks' → Push notifications
* // }
* ```
*/
import { IntegrationBase } from './IntegrationBase.js'
import { IntegrationLoader, IntegrationType, INTEGRATION_CATALOG } from './IntegrationLoader.js'
import { IntegrationConfig, IntegrationHealthStatus } from './types.js'
import { ODataIntegration } from '../odata/ODataIntegration.js'
import { GoogleSheetsIntegration } from '../sheets/GoogleSheetsIntegration.js'
import { SSEIntegration } from '../sse/SSEIntegration.js'
import { WebhookIntegration } from '../webhooks/WebhookIntegration.js'
/**
* Integration Hub configuration
*/
export interface IntegrationHubConfig {
/** Base path for all endpoints (default: '') */
basePath?: string
/** Which integrations to enable (default: all) */
enable?: IntegrationType[] | 'all'
/** Per-integration config overrides */
config?: {
odata?: IntegrationConfig & { basePath?: string }
sheets?: IntegrationConfig & { basePath?: string }
sse?: IntegrationConfig & { basePath?: string }
webhooks?: IntegrationConfig & { basePath?: string }
}
}
/**
* HTTP request for integration routing
*/
export interface IntegrationRequest {
method: string
path: string
query?: Record<string, string>
headers?: Record<string, string>
body?: any
}
/**
* HTTP response from integration
*/
export interface IntegrationResponse {
status: number
headers: Record<string, string>
body: any
isSSE?: boolean
}
/**
* Integration Hub - Zero-Configuration Integration Manager
*
* Provides instant access to:
* - OData API (Excel Power Query, Power BI, Tableau)
* - Google Sheets API (two-way sync)
* - SSE streaming (real-time dashboards)
* - Webhooks (push notifications)
*
* All integrations work in any environment with zero external dependencies.
*/
export class IntegrationHub {
private integrations: Map<IntegrationType, IntegrationBase> = new Map()
private config: Required<IntegrationHubConfig>
private _endpoints: Record<IntegrationType, string> = {} as any
/**
* Create and initialize the Integration Hub
*
* @example
* ```typescript
* // All integrations, default paths
* const hub = await IntegrationHub.create(brain)
*
* // Custom base path
* const hub = await IntegrationHub.create(brain, { basePath: '/api/v1' })
*
* // Only specific integrations
* const hub = await IntegrationHub.create(brain, { enable: ['odata', 'sheets'] })
* ```
*/
static async create(brain: any, config?: IntegrationHubConfig): Promise<IntegrationHub> {
const hub = new IntegrationHub(config)
await hub.initialize(brain)
return hub
}
private constructor(config?: IntegrationHubConfig) {
this.config = {
basePath: config?.basePath ?? '',
enable: config?.enable ?? 'all',
config: config?.config ?? {}
}
}
private async initialize(brain: any): Promise<void> {
const toEnable = this.config.enable === 'all'
? (['odata', 'sheets', 'sse', 'webhooks'] as IntegrationType[])
: this.config.enable
// Create context for integration initialization
const context = {
brain,
storage: brain.getStorage?.() || null,
config: {},
log: (message: string, level?: string) => {
// Silent logging - integrations handle their own logging
}
}
for (const type of toEnable) {
const integration = await this.createIntegration(type)
if (integration) {
// Initialize the integration with context (BaseAugmentation pattern)
await integration.initialize(context)
this.integrations.set(type, integration)
this._endpoints[type] = this.getBasePath(type)
}
}
}
private async createIntegration(type: IntegrationType): Promise<IntegrationBase | null> {
const basePath = this.config.basePath
const cfg = this.config.config?.[type]
switch (type) {
case 'odata':
return new ODataIntegration({
...cfg,
basePath: cfg?.basePath ?? `${basePath}/odata`
})
case 'sheets':
return new GoogleSheetsIntegration({
...cfg,
basePath: cfg?.basePath ?? `${basePath}/sheets`
})
case 'sse':
return new SSEIntegration({
...cfg,
basePath: cfg?.basePath ?? `${basePath}/events`
})
case 'webhooks':
return new WebhookIntegration(cfg)
default:
return null
}
}
private getBasePath(type: IntegrationType): string {
const basePath = this.config.basePath
const cfg = this.config.config?.[type] as any
switch (type) {
case 'odata':
return cfg?.basePath ?? `${basePath}/odata`
case 'sheets':
return cfg?.basePath ?? `${basePath}/sheets`
case 'sse':
return cfg?.basePath ?? `${basePath}/events`
case 'webhooks':
return `${basePath}/webhooks`
default:
return ''
}
}
/**
* Get all endpoint paths
*/
get endpoints(): Record<IntegrationType, string> {
return { ...this._endpoints }
}
/**
* Handle an HTTP request and route to the appropriate integration
*
* @example
* ```typescript
* // Express middleware
* app.use('/api', async (req, res) => {
* const response = await hub.handleRequest({
* method: req.method,
* path: req.path,
* query: req.query,
* headers: req.headers,
* body: req.body
* })
*
* if (response.isSSE) {
* // Handle SSE stream
* } else {
* res.status(response.status).set(response.headers).json(response.body)
* }
* })
* ```
*/
async handleRequest(request: IntegrationRequest): Promise<IntegrationResponse> {
const { path } = request
// Route to appropriate integration based on path
for (const [type, basePath] of Object.entries(this._endpoints)) {
if (path.startsWith(basePath) || path === basePath) {
const integration = this.integrations.get(type as IntegrationType)
if (integration && 'handleRequest' in integration) {
const handler = integration as any
const relativePath = path.slice(basePath.length) || '/'
return handler.handleRequest({
...request,
path: relativePath
})
}
}
}
return {
status: 404,
headers: { 'Content-Type': 'application/json' },
body: { error: 'Not found', path }
}
}
/**
* Get a specific integration
*/
get<T extends IntegrationBase>(type: IntegrationType): T | undefined {
return this.integrations.get(type) as T | undefined
}
/**
* Get the OData integration
*/
get odata(): ODataIntegration | undefined {
return this.integrations.get('odata') as ODataIntegration
}
/**
* Get the Google Sheets integration
*/
get sheets(): GoogleSheetsIntegration | undefined {
return this.integrations.get('sheets') as GoogleSheetsIntegration
}
/**
* Get the SSE integration
*/
get sse(): SSEIntegration | undefined {
return this.integrations.get('sse') as SSEIntegration
}
/**
* Get the Webhook integration
*/
get webhooks(): WebhookIntegration | undefined {
return this.integrations.get('webhooks') as WebhookIntegration
}
/**
* Check if an integration is enabled
*/
has(type: IntegrationType): boolean {
return this.integrations.has(type)
}
/**
* Get health status of all integrations
*/
health(): Record<IntegrationType, IntegrationHealthStatus> {
const result: Record<string, IntegrationHealthStatus> = {}
for (const [type, integration] of this.integrations) {
result[type] = integration.health()
}
return result as Record<IntegrationType, IntegrationHealthStatus>
}
/**
* Stop all integrations
*/
async stop(): Promise<void> {
for (const integration of this.integrations.values()) {
await integration.stop()
}
}
/**
* Get connection instructions for each tool
*/
getInstructions(): Record<string, string> {
const base = this.config.basePath || 'http://localhost:3000'
return {
excel: `
Excel Power Query:
1. Data Get Data From Other Sources From OData Feed
2. Enter URL: ${base}/odata
3. Click OK, then Load
`.trim(),
powerbi: `
Power BI:
1. Get Data OData Feed
2. Enter URL: ${base}/odata
3. Click OK, then Load
`.trim(),
googleSheets: `
Google Sheets:
1. Install the Brainy add-on from Google Workspace Marketplace
2. Open sidebar: Extensions Brainy Open
3. Connect to: ${base}/sheets
4. Use custom functions: =BRAINY_QUERY("type:Person", 100)
`.trim(),
realtime: `
Real-time Streaming (SSE):
const source = new EventSource('${base}/events')
source.onmessage = (event) => console.log(JSON.parse(event.data))
`.trim(),
webhooks: `
Webhooks:
POST ${base}/webhooks/register
{
"url": "https://your-server.com/webhook",
"events": { "entityTypes": ["noun"], "operations": ["create", "update"] }
}
`.trim()
}
}
}
/**
* Create an Integration Hub with zero configuration
*
* @example
* ```typescript
* const hub = await createIntegrationHub(brain)
* console.log(hub.endpoints)
* ```
*/
export async function createIntegrationHub(
brain: any,
config?: IntegrationHubConfig
): Promise<IntegrationHub> {
return IntegrationHub.create(brain, config)
}

View file

@ -0,0 +1,279 @@
/**
* Integration Hub - Integration Loader
*
* Lazy-loads integrations with environment detection.
* Zero external dependencies - works everywhere.
*/
import { IntegrationBase } from './IntegrationBase.js'
import { IntegrationConfig } from './types.js'
/**
* Supported integration types
*/
export type IntegrationType =
| 'odata' // Excel Power Query, Power BI, Tableau
| 'sheets' // Google Sheets two-way sync
| 'sse' // Server-Sent Events streaming
| 'webhooks' // Push notifications to external URLs
/**
* Runtime environment
*/
export type RuntimeEnvironment =
| 'node'
| 'browser'
| 'deno'
| 'cloudflare'
| 'bun'
/**
* Integration metadata
*/
export interface IntegrationInfo {
id: IntegrationType
name: string
description: string
environments: RuntimeEnvironment[]
tools: string[] // What tools this works with
}
/**
* Integration catalog - all built-in, zero dependencies
*/
export const INTEGRATION_CATALOG: Record<IntegrationType, IntegrationInfo> = {
odata: {
id: 'odata',
name: 'OData 4.0 API',
description: 'REST API for spreadsheets and BI tools',
environments: ['node', 'browser', 'deno', 'cloudflare', 'bun'],
tools: ['Excel Power Query', 'Power BI', 'Tableau', 'Qlik', 'SAP']
},
sheets: {
id: 'sheets',
name: 'Google Sheets',
description: 'Two-way sync with Google Sheets',
environments: ['node', 'browser', 'deno', 'cloudflare', 'bun'],
tools: ['Google Sheets', 'Apps Script']
},
sse: {
id: 'sse',
name: 'Real-time Streaming',
description: 'Server-Sent Events for live updates',
environments: ['node', 'browser', 'deno', 'cloudflare', 'bun'],
tools: ['Dashboards', 'Live UIs', 'Monitoring']
},
webhooks: {
id: 'webhooks',
name: 'Webhooks',
description: 'Push events to external URLs',
environments: ['node', 'browser', 'deno', 'cloudflare', 'bun'],
tools: ['Zapier', 'IFTTT', 'n8n', 'Custom APIs']
}
}
/**
* Detect current runtime environment
*/
export function detectEnvironment(): RuntimeEnvironment {
// Deno
if (typeof (globalThis as any).Deno !== 'undefined') {
return 'deno'
}
// Bun
if (typeof (globalThis as any).Bun !== 'undefined') {
return 'bun'
}
// Cloudflare Workers
if (
typeof (globalThis as any).caches !== 'undefined' &&
typeof (globalThis as any).HTMLRewriter !== 'undefined'
) {
return 'cloudflare'
}
// Node.js
if (
typeof process !== 'undefined' &&
process.versions &&
process.versions.node
) {
return 'node'
}
// Browser
if (typeof window !== 'undefined') {
return 'browser'
}
return 'node'
}
/**
* Integration loader configuration
*/
export interface IntegrationLoaderConfig {
/** Which integrations to load: array of types, 'all', or 'none' */
integrations?: IntegrationType[] | 'all' | 'none'
/** Override configs per integration */
config?: Partial<Record<IntegrationType, IntegrationConfig>>
/** Custom integrations to add */
custom?: IntegrationBase[]
}
/**
* Lazy-loading integration manager
*
* @example
* ```typescript
* // Load all integrations (recommended)
* const loader = new IntegrationLoader({ integrations: 'all' })
* const integrations = await loader.load()
*
* // Load specific integrations
* const loader = new IntegrationLoader({
* integrations: ['odata', 'sheets']
* })
* ```
*/
export class IntegrationLoader {
private config: IntegrationLoaderConfig
private environment: RuntimeEnvironment
private loaded: Map<IntegrationType, IntegrationBase> = new Map()
constructor(config: IntegrationLoaderConfig = {}) {
this.config = {
integrations: config.integrations ?? 'none',
config: config.config ?? {},
custom: config.custom ?? []
}
this.environment = detectEnvironment()
}
/**
* Get current runtime environment
*/
getEnvironment(): RuntimeEnvironment {
return this.environment
}
/**
* Get all available integrations
*/
getAvailable(): IntegrationInfo[] {
return Object.values(INTEGRATION_CATALOG).filter((info) =>
info.environments.includes(this.environment)
)
}
/**
* Check if an integration is available
*/
isAvailable(type: IntegrationType): boolean {
const info = INTEGRATION_CATALOG[type]
return info?.environments.includes(this.environment) ?? false
}
/**
* Load and instantiate integrations
*/
async load(): Promise<IntegrationBase[]> {
const toLoad = this.resolveIntegrations()
const results: IntegrationBase[] = []
// Load integrations in parallel for speed
const loadPromises = toLoad.map(async (type) => {
try {
const integration = await this.loadOne(type)
if (integration) {
this.loaded.set(type, integration)
return integration
}
} catch (error: any) {
console.warn(`[Brainy] Failed to load ${type}: ${error.message}`)
}
return null
})
const loadedIntegrations = await Promise.all(loadPromises)
results.push(...loadedIntegrations.filter((i): i is IntegrationBase => i !== null))
// Add custom integrations
for (const custom of this.config.custom ?? []) {
results.push(custom)
}
return results
}
/**
* Get a loaded integration by type
*/
get<T extends IntegrationBase = IntegrationBase>(type: IntegrationType): T | undefined {
return this.loaded.get(type) as T | undefined
}
/**
* Check if an integration is loaded
*/
has(type: IntegrationType): boolean {
return this.loaded.has(type)
}
/**
* Get all loaded integrations
*/
all(): IntegrationBase[] {
return Array.from(this.loaded.values())
}
private resolveIntegrations(): IntegrationType[] {
const { integrations } = this.config
if (integrations === 'none') {
return []
}
if (integrations === 'all') {
return this.getAvailable().map((info) => info.id)
}
return (integrations || []).filter((type) => this.isAvailable(type))
}
private async loadOne(type: IntegrationType): Promise<IntegrationBase | null> {
const cfg = this.config.config?.[type]
switch (type) {
case 'odata': {
const { ODataIntegration } = await import('../odata/ODataIntegration.js')
return new ODataIntegration(cfg)
}
case 'sheets': {
const { GoogleSheetsIntegration } = await import('../sheets/GoogleSheetsIntegration.js')
return new GoogleSheetsIntegration(cfg)
}
case 'sse': {
const { SSEIntegration } = await import('../sse/SSEIntegration.js')
return new SSEIntegration(cfg)
}
case 'webhooks': {
const { WebhookIntegration } = await import('../webhooks/WebhookIntegration.js')
return new WebhookIntegration(cfg)
}
default:
return null
}
}
}
/**
* Create an integration loader
*/
export function createIntegrationLoader(config?: IntegrationLoaderConfig): IntegrationLoader {
return new IntegrationLoader(config)
}

View file

@ -0,0 +1,574 @@
/**
* Integration Hub - Tabular Exporter
*
* Converts Brainy entities to tabular formats (rows/columns) for use in
* spreadsheets, SQL databases, and BI tools.
*/
import { Entity, Relation } from '../../types/brainy.types.js'
import {
TabularRow,
RelationTabularRow,
TabularExporterConfig
} from './types.js'
/**
* Converts Brainy entities to tabular formats
*
* Used by SQL, OData, Google Sheets, and CSV integrations to maintain
* consistent entity-to-row mapping across all external tools.
*
* @example
* ```typescript
* const exporter = new TabularExporter({
* flattenMetadata: true,
* includeVectors: false
* })
*
* const rows = exporter.entitiesToRows(entities)
* const csv = exporter.toCSV(entities)
* const odata = exporter.toOData(entities)
* ```
*/
export class TabularExporter {
private config: Required<TabularExporterConfig>
constructor(config: TabularExporterConfig = {}) {
this.config = {
flattenMetadata: config.flattenMetadata ?? true,
metadataPrefix: config.metadataPrefix ?? 'Metadata_',
includeVectors: config.includeVectors ?? false,
dateFormat: config.dateFormat ?? 'ISO8601',
jsonStringify: config.jsonStringify ?? ['data'],
maxFlattenDepth: config.maxFlattenDepth ?? 1, // Flatten one level, stringify deeper
excludeColumns: config.excludeColumns ?? []
}
}
/**
* Convert entities to tabular rows
*/
entitiesToRows(entities: Entity[]): TabularRow[] {
return entities.map((entity) => this.entityToRow(entity))
}
/**
* Convert a single entity to a tabular row
*/
entityToRow(entity: Entity): TabularRow {
const row: TabularRow = {
Id: entity.id,
Type: entity.type,
CreatedAt: this.formatDate(entity.createdAt),
UpdatedAt: entity.updatedAt
? this.formatDate(entity.updatedAt)
: this.formatDate(entity.createdAt),
Confidence: entity.confidence ?? null,
Weight: entity.weight ?? null,
Service: entity.service ?? null,
Data: this.config.jsonStringify.includes('data')
? JSON.stringify(entity.data ?? null)
: entity.data ?? null
}
// Include vector if configured
if (this.config.includeVectors && entity.vector) {
row.Vector = JSON.stringify(Array.from(entity.vector))
}
// Flatten metadata
if (this.config.flattenMetadata && entity.metadata) {
const flatMetadata = this.flattenObject(
entity.metadata,
this.config.metadataPrefix,
this.config.maxFlattenDepth
)
Object.assign(row, flatMetadata)
} else if (entity.metadata) {
row.Metadata = JSON.stringify(entity.metadata)
}
// Remove excluded columns
for (const col of this.config.excludeColumns) {
delete row[col]
}
return row
}
/**
* Convert relations to tabular rows
*/
relationsToRows(relations: Relation[]): RelationTabularRow[] {
return relations.map((rel) => this.relationToRow(rel))
}
/**
* Convert a single relation to a tabular row
*/
relationToRow(relation: Relation): RelationTabularRow {
return {
Id: relation.id,
FromId: relation.from,
ToId: relation.to,
Type: relation.type,
Weight: relation.weight ?? null,
Confidence: relation.confidence ?? null,
CreatedAt: this.formatDate(relation.createdAt),
UpdatedAt: relation.updatedAt
? this.formatDate(relation.updatedAt)
: this.formatDate(relation.createdAt),
Service: relation.service ?? null,
Metadata: relation.metadata ? JSON.stringify(relation.metadata) : null
}
}
/**
* Convert entities to CSV string
*/
toCSV(entities: Entity[], options?: { delimiter?: string }): string {
const delimiter = options?.delimiter ?? ','
const rows = this.entitiesToRows(entities)
if (rows.length === 0) {
return ''
}
// Get all columns from all rows
const columns = this.getAllColumns(rows)
// Header row
const header = columns.map((col) => this.escapeCSV(col)).join(delimiter)
// Data rows
const dataRows = rows.map((row) =>
columns
.map((col) => {
const value = row[col]
return this.escapeCSV(
value === null || value === undefined ? '' : String(value)
)
})
.join(delimiter)
)
return [header, ...dataRows].join('\n')
}
/**
* Convert relations to CSV string
*/
relationsToCSV(
relations: Relation[],
options?: { delimiter?: string }
): string {
const delimiter = options?.delimiter ?? ','
const rows = this.relationsToRows(relations)
if (rows.length === 0) {
return ''
}
const columns: (keyof RelationTabularRow)[] = [
'Id',
'FromId',
'ToId',
'Type',
'Weight',
'Confidence',
'CreatedAt',
'UpdatedAt',
'Service',
'Metadata'
]
const header = columns.join(delimiter)
const dataRows = rows.map((row) =>
columns
.map((col) => {
const value = row[col]
return this.escapeCSV(
value === null || value === undefined ? '' : String(value)
)
})
.join(delimiter)
)
return [header, ...dataRows].join('\n')
}
/**
* Convert entities to OData format (JSON with annotations)
*/
toOData(
entities: Entity[],
options?: {
context?: string
count?: number
nextLink?: string
}
): object {
const rows = this.entitiesToRows(entities)
const result: any = {
'@odata.context': options?.context ?? '$metadata#Entities'
}
if (options?.count !== undefined) {
result['@odata.count'] = options.count
}
result.value = rows.map((row) => this.rowToODataEntity(row))
if (options?.nextLink) {
result['@odata.nextLink'] = options.nextLink
}
return result
}
/**
* Convert relations to OData format
*/
relationsToOData(
relations: Relation[],
options?: {
context?: string
count?: number
nextLink?: string
}
): object {
const rows = this.relationsToRows(relations)
const result: any = {
'@odata.context': options?.context ?? '$metadata#Relationships'
}
if (options?.count !== undefined) {
result['@odata.count'] = options.count
}
result.value = rows
if (options?.nextLink) {
result['@odata.nextLink'] = options.nextLink
}
return result
}
/**
* Parse CSV string to entity-like objects
*/
parseCSV(
csv: string,
options?: { delimiter?: string }
): Partial<Entity>[] {
const delimiter = options?.delimiter ?? ','
const lines = csv.split('\n').filter((line) => line.trim())
if (lines.length < 2) {
return []
}
const headers = this.parseCSVLine(lines[0], delimiter)
const entities: Partial<Entity>[] = []
for (let i = 1; i < lines.length; i++) {
const values = this.parseCSVLine(lines[i], delimiter)
const row: Record<string, any> = {}
for (let j = 0; j < headers.length; j++) {
row[headers[j]] = values[j] ?? ''
}
entities.push(this.rowToEntity(row))
}
return entities
}
/**
* Get schema from entities (column names and types)
*/
getSchema(entities: Entity[]): Array<{
name: string
type: 'string' | 'number' | 'boolean' | 'datetime' | 'json'
nullable: boolean
}> {
const rows = this.entitiesToRows(entities.slice(0, 100)) // Sample first 100
const columns = this.getAllColumns(rows)
const schema: Array<{
name: string
type: 'string' | 'number' | 'boolean' | 'datetime' | 'json'
nullable: boolean
}> = []
for (const col of columns) {
let type: 'string' | 'number' | 'boolean' | 'datetime' | 'json' = 'string'
let nullable = false
for (const row of rows) {
const value = row[col]
if (value === null || value === undefined) {
nullable = true
continue
}
const inferredType = this.inferType(value)
if (type === 'string') {
type = inferredType
} else if (type !== inferredType) {
// Mixed types, fall back to string
type = 'string'
break
}
}
schema.push({ name: col, type, nullable })
}
return schema
}
// Private helper methods
private formatDate(timestamp: number): string {
switch (this.config.dateFormat) {
case 'unix':
return Math.floor(timestamp / 1000).toString()
case 'unix_ms':
return timestamp.toString()
case 'ISO8601':
default:
return new Date(timestamp).toISOString()
}
}
private flattenObject(
obj: Record<string, any>,
prefix: string,
maxDepth: number,
currentDepth = 0
): Record<string, any> {
const result: Record<string, any> = {}
for (const [key, value] of Object.entries(obj)) {
const newKey = `${prefix}${key}`
if (value === null || value === undefined) {
result[newKey] = null
} else if (Array.isArray(value)) {
// Arrays are always JSON stringified
result[newKey] = JSON.stringify(value)
} else if (typeof value === 'object') {
// Objects: flatten if under depth limit, otherwise stringify
if (currentDepth < maxDepth - 1) {
Object.assign(
result,
this.flattenObject(value, `${newKey}_`, maxDepth, currentDepth + 1)
)
} else {
// Max depth reached - stringify the object
result[newKey] = JSON.stringify(value)
}
} else {
// Primitives: use as-is
result[newKey] = value
}
}
return result
}
private getAllColumns(rows: TabularRow[]): string[] {
const columnSet = new Set<string>()
// Standard columns first
const standardColumns = [
'Id',
'Type',
'CreatedAt',
'UpdatedAt',
'Confidence',
'Weight',
'Service',
'Data'
]
for (const col of standardColumns) {
columnSet.add(col)
}
// Add all other columns from rows
for (const row of rows) {
for (const key of Object.keys(row)) {
columnSet.add(key)
}
}
return Array.from(columnSet)
}
private escapeCSV(value: string): string {
// Escape quotes and wrap in quotes if contains special characters
if (
value.includes(',') ||
value.includes('"') ||
value.includes('\n') ||
value.includes('\r')
) {
return `"${value.replace(/"/g, '""')}"`
}
return value
}
private parseCSVLine(line: string, delimiter: string): string[] {
const result: string[] = []
let current = ''
let inQuotes = false
for (let i = 0; i < line.length; i++) {
const char = line[i]
if (inQuotes) {
if (char === '"') {
if (line[i + 1] === '"') {
current += '"'
i++
} else {
inQuotes = false
}
} else {
current += char
}
} else {
if (char === '"') {
inQuotes = true
} else if (char === delimiter) {
result.push(current)
current = ''
} else {
current += char
}
}
}
result.push(current)
return result
}
private rowToEntity(row: Record<string, any>): Partial<Entity> {
const entity: Partial<Entity> = {}
// Map standard columns
if (row.Id) entity.id = row.Id
if (row.Type) entity.type = row.Type as any
if (row.Service) entity.service = row.Service
if (row.Confidence) entity.confidence = parseFloat(row.Confidence)
if (row.Weight) entity.weight = parseFloat(row.Weight)
// Parse timestamps
if (row.CreatedAt) {
entity.createdAt = this.parseDate(row.CreatedAt)
}
if (row.UpdatedAt) {
entity.updatedAt = this.parseDate(row.UpdatedAt)
}
// Parse data
if (row.Data) {
try {
entity.data = JSON.parse(row.Data)
} catch {
entity.data = row.Data
}
}
// Collect metadata from prefixed columns
const metadata: Record<string, any> = {}
for (const [key, value] of Object.entries(row)) {
if (key.startsWith(this.config.metadataPrefix)) {
const metaKey = key.slice(this.config.metadataPrefix.length)
try {
metadata[metaKey] = JSON.parse(value as string)
} catch {
metadata[metaKey] = value
}
}
}
if (Object.keys(metadata).length > 0) {
entity.metadata = metadata
}
return entity
}
private parseDate(value: string): number {
// Try parsing as number (unix timestamp)
const num = Number(value)
if (!isNaN(num)) {
// If it looks like seconds (< year 3000 in seconds)
if (num < 32503680000) {
return num * 1000
}
return num
}
// Try parsing as ISO date
const date = new Date(value)
if (!isNaN(date.getTime())) {
return date.getTime()
}
return Date.now()
}
private rowToODataEntity(row: TabularRow): object {
const result: any = {}
for (const [key, value] of Object.entries(row)) {
if (value === null) {
result[key] = null
} else if (key === 'CreatedAt' || key === 'UpdatedAt') {
// OData datetime format
result[key] = value
} else {
result[key] = value
}
}
return result
}
private inferType(
value: any
): 'string' | 'number' | 'boolean' | 'datetime' | 'json' {
if (typeof value === 'number') {
return 'number'
}
if (typeof value === 'boolean') {
return 'boolean'
}
if (typeof value === 'string') {
// Check if it's a date
if (/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}/.test(value)) {
return 'datetime'
}
// Check if it's JSON
if (
(value.startsWith('{') && value.endsWith('}')) ||
(value.startsWith('[') && value.endsWith(']'))
) {
try {
JSON.parse(value)
return 'json'
} catch {
// Not valid JSON
}
}
}
return 'string'
}
}

View file

@ -0,0 +1,64 @@
/**
* Integration Hub - Core Infrastructure
*
* Shared foundation for all integrations:
* - EventBus: Real-time change notifications
* - TabularExporter: Entity to rows/columns conversion
* - IntegrationBase: Base class for integrations
* - IntegrationHub: Zero-config integration manager
*/
// Event system
export { EventBus } from './EventBus.js'
// Tabular export
export { TabularExporter } from './TabularExporter.js'
// Base class
export {
IntegrationBase,
type HTTPIntegration,
type StreamingIntegration
} from './IntegrationBase.js'
// Integration loader
export {
IntegrationLoader,
createIntegrationLoader,
detectEnvironment,
INTEGRATION_CATALOG,
type IntegrationType,
type RuntimeEnvironment,
type IntegrationInfo,
type IntegrationLoaderConfig
} from './IntegrationLoader.js'
// Zero-config hub
export {
IntegrationHub,
createIntegrationHub,
type IntegrationHubConfig,
type IntegrationRequest,
type IntegrationResponse
} from './IntegrationHub.js'
// Types
export type {
// Events
BrainyEvent,
EventFilter,
EventHandler,
EventSubscription,
// Tabular
TabularRow,
RelationTabularRow,
TabularExporterConfig,
// Config
IntegrationConfig,
IntegrationHealthStatus,
// OData
ODataQueryOptions,
// Webhooks
WebhookRegistration,
WebhookDeliveryResult
} from './types.js'

View file

@ -0,0 +1,261 @@
/**
* Integration Hub - Shared Types
*
* Types for OData, Google Sheets, SSE, and Webhooks integrations.
* Zero external dependencies.
*/
import { Entity, Relation } from '../../types/brainy.types.js'
import { NounType, VerbType } from '../../types/graphTypes.js'
// ============================================================================
// Events - Real-time change notifications
// ============================================================================
/**
* Real-time event emitted when Brainy data changes
*/
export interface BrainyEvent {
/** Unique event identifier */
id: string
/** What changed: noun, verb, or VFS */
entityType: 'noun' | 'verb' | 'vfs'
/** What happened */
operation: 'create' | 'update' | 'delete'
/** The entity ID that was affected */
entityId: string
/** Unix timestamp in milliseconds */
timestamp: number
/** Monotonically increasing sequence ID for ordering/resumption */
sequenceId: bigint
/** NounType if entityType is 'noun' */
nounType?: NounType
/** VerbType if entityType is 'verb' */
verbType?: VerbType
/** Service (multi-tenancy) */
service?: string
/** Full entity data (if includeData is enabled) */
data?: Entity | Relation
}
/**
* Filter for subscribing to events
*/
export interface EventFilter {
/** Filter by entity types */
entityTypes?: ('noun' | 'verb' | 'vfs')[]
/** Filter by operations */
operations?: ('create' | 'update' | 'delete')[]
/** Filter by noun types */
nounTypes?: NounType[]
/** Filter by verb types */
verbTypes?: VerbType[]
/** Filter by service */
service?: string
/** Resume from this sequence ID */
since?: bigint
}
/**
* Event handler function
*/
export type EventHandler = (event: BrainyEvent) => void | Promise<void>
/**
* Event subscription handle
*/
export interface EventSubscription {
id: string
unsubscribe(): void
}
// ============================================================================
// Tabular Export - Entity to rows/columns conversion
// ============================================================================
/**
* Tabular row representation of an entity
*/
export interface TabularRow {
Id: string
Type: string
CreatedAt: string
UpdatedAt: string
Confidence: number | null
Weight: number | null
Service: string | null
Data: string | null
/** Flattened metadata columns (Metadata_*) */
[key: string]: any
}
/**
* Tabular row for relations
*/
export interface RelationTabularRow {
Id: string
FromId: string
ToId: string
Type: string
Weight: number | null
Confidence: number | null
CreatedAt: string
UpdatedAt: string
Service: string | null
Metadata: string | null
}
/**
* Configuration for TabularExporter
*/
export interface TabularExporterConfig {
/** Flatten metadata into separate columns (default: true) */
flattenMetadata?: boolean
/** Prefix for metadata columns (default: 'Metadata_') */
metadataPrefix?: string
/** Include vector embeddings (default: false) */
includeVectors?: boolean
/** Date format (default: 'ISO8601') */
dateFormat?: 'ISO8601' | 'unix' | 'unix_ms'
/** Fields to JSON.stringify (default: ['data']) */
jsonStringify?: string[]
/** Max depth for flattening nested objects (default: 2) */
maxFlattenDepth?: number
/** Columns to exclude */
excludeColumns?: string[]
}
// ============================================================================
// Integration Configuration
// ============================================================================
/**
* Base configuration for all integrations
*/
export interface IntegrationConfig {
/** Enable/disable the integration */
enabled?: boolean
/** Rate limiting */
rateLimit?: {
max: number
windowMs: number
}
/** Authentication */
auth?: {
required: boolean
apiKeys?: string[]
}
/** CORS settings */
cors?: {
origin: string | string[]
methods?: string[]
credentials?: boolean
}
}
/**
* Health status for an integration
*/
export interface IntegrationHealthStatus {
name: string
status: 'healthy' | 'degraded' | 'unhealthy' | 'stopped'
message?: string
uptimeMs?: number
requestCount?: number
errorCount?: number
lastError?: string
checkedAt: number
}
// ============================================================================
// OData - Excel Power Query, Power BI, Tableau
// ============================================================================
/**
* OData query options parsed from URL
*/
export interface ODataQueryOptions {
/** $filter expression */
filter?: string
/** $select columns */
select?: string[]
/** $orderby specification */
orderBy?: Array<{ field: string; direction: 'asc' | 'desc' }>
/** $top (limit) */
top?: number
/** $skip (offset) */
skip?: number
/** $expand relations */
expand?: string[]
/** $count - include total count */
count?: boolean
/** $search - full text search */
search?: string
}
// ============================================================================
// Webhooks - Push notifications
// ============================================================================
/**
* Webhook registration
*/
export interface WebhookRegistration {
id: string
url: string
events: EventFilter
secret?: string
active: boolean
retryPolicy?: {
maxRetries: number
backoffMultiplier: number
initialDelayMs: number
maxDelayMs: number
}
createdAt: number
lastDeliveryAt?: number
failureCount?: number
}
/**
* Webhook delivery result
*/
export interface WebhookDeliveryResult {
webhookId: string
eventId: string
success: boolean
statusCode?: number
error?: string
attempts: number
timestamp: number
}

138
src/integrations/index.ts Normal file
View file

@ -0,0 +1,138 @@
/**
* Brainy Integration Hub
*
* Connect Brainy to external tools with zero configuration:
* - Google Sheets (two-way sync)
* - Excel / Power BI / Tableau (OData 4.0)
* - Real-time dashboards (SSE streaming)
* - External webhooks (push notifications)
*
* @example Enable integrations (v7.4.0 - recommended)
* ```typescript
* import { Brainy } from '@soulcraft/brainy'
*
* const brain = new Brainy({ integrations: true })
* await brain.init()
*
* // Access the hub
* console.log(brain.hub.endpoints)
* // { odata: '/odata', sheets: '/sheets', sse: '/events', webhooks: '/webhooks' }
*
* // Handle requests (Express, Hono, etc.)
* app.all('/odata/*', async (req, res) => {
* const response = await brain.hub.handleRequest(req)
* res.status(response.status).json(response.body)
* })
* ```
*/
// ============================================================================
// Integration Hub (used internally by brain.hub)
// ============================================================================
export {
IntegrationHub,
createIntegrationHub,
type IntegrationHubConfig,
type IntegrationRequest,
type IntegrationResponse
} from './core/IntegrationHub.js'
// ============================================================================
// Core Infrastructure
// ============================================================================
export {
// Event system for real-time updates
EventBus,
// Entity → rows/columns conversion
TabularExporter,
// Base class for custom integrations
IntegrationBase,
// Lazy-loading manager
IntegrationLoader,
createIntegrationLoader,
// Environment detection
detectEnvironment,
// Integration catalog
INTEGRATION_CATALOG
} from './core/index.js'
// Types
export type {
HTTPIntegration,
StreamingIntegration,
IntegrationType,
RuntimeEnvironment,
IntegrationInfo,
IntegrationLoaderConfig
} from './core/index.js'
export type {
BrainyEvent,
EventFilter,
EventHandler,
EventSubscription,
TabularRow,
RelationTabularRow,
TabularExporterConfig,
IntegrationConfig,
IntegrationHealthStatus,
ODataQueryOptions,
WebhookRegistration,
WebhookDeliveryResult
} from './core/index.js'
// ============================================================================
// Individual Integrations
// ============================================================================
/**
* OData 4.0 Integration
*
* Connect Excel Power Query, Power BI, Tableau, and any OData client.
*/
export { ODataIntegration, type ODataConfig } from './odata/index.js'
/**
* Google Sheets Integration
*
* Two-way sync between Brainy and Google Sheets via Apps Script.
*/
export { GoogleSheetsIntegration, type GoogleSheetsConfig } from './sheets/index.js'
/**
* SSE Streaming Integration
*
* Real-time event streaming via Server-Sent Events.
*/
export { SSEIntegration, type SSEConfig } from './sse/index.js'
/**
* Webhook Integration
*
* Push events to external URLs with retry and signing.
*/
export { WebhookIntegration, type WebhookConfig } from './webhooks/index.js'
// ============================================================================
// OData Utilities (for advanced use)
// ============================================================================
export {
parseODataQuery,
parseFilter,
parseOrderBy,
parseSelect,
odataToFindParams,
applyFilter,
applySelect,
applyOrderBy,
applyPagination
} from './odata/ODataQueryParser.js'
export {
generateEdmx,
generateMetadataJson,
generateServiceDocument
} from './odata/EdmxGenerator.js'

View file

@ -0,0 +1,238 @@
/**
* EDMX Metadata Generator
*
* Generates OData $metadata XML document describing the Brainy data model.
* This allows tools like Excel Power Query and Power BI to discover the schema.
*/
import { NounType, VerbType } from '../../types/graphTypes.js'
/**
* OData property type mapping
*/
interface PropertyDef {
name: string
type: string
nullable: boolean
}
/**
* Generate EDMX metadata XML for Brainy schema
*
* @param options Configuration options
* @returns XML string
*/
export function generateEdmx(options?: {
namespace?: string
containerName?: string
includeRelationships?: boolean
}): string {
const namespace = options?.namespace ?? 'Brainy'
const containerName = options?.containerName ?? 'BrainyContainer'
const includeRelationships = options?.includeRelationships ?? true
const entityProperties = getEntityProperties()
const relationshipProperties = getRelationshipProperties()
let xml = `<?xml version="1.0" encoding="utf-8"?>
<edmx:Edmx Version="4.0" xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx">
<edmx:DataServices>
<Schema Namespace="${namespace}" xmlns="http://docs.oasis-open.org/odata/ns/edm">
<!-- Entity Type -->
<EntityType Name="Entity">
<Key>
<PropertyRef Name="Id"/>
</Key>
${entityProperties.map((p) => ` <Property Name="${p.name}" Type="${p.type}" Nullable="${p.nullable}"/>`).join('\n')}
</EntityType>
${includeRelationships ? generateRelationshipType(relationshipProperties) : ''}
<!-- Enum: NounType -->
<EnumType Name="NounType">
${Object.values(NounType)
.map((v, i) => ` <Member Name="${v}" Value="${i}"/>`)
.join('\n')}
</EnumType>
${includeRelationships ? generateVerbTypeEnum() : ''}
<!-- Entity Container -->
<EntityContainer Name="${containerName}">
<EntitySet Name="Entities" EntityType="${namespace}.Entity"/>
${includeRelationships ? ` <EntitySet Name="Relationships" EntityType="${namespace}.Relationship"/>` : ''}
</EntityContainer>
</Schema>
</edmx:DataServices>
</edmx:Edmx>`
return xml
}
/**
* Generate JSON-based OData metadata (for modern clients)
*/
export function generateMetadataJson(options?: {
namespace?: string
includeRelationships?: boolean
}): object {
const namespace = options?.namespace ?? 'Brainy'
const includeRelationships = options?.includeRelationships ?? true
const entityProps = getEntityProperties()
const relProps = getRelationshipProperties()
const schema: any = {
$Version: '4.0',
[`${namespace}`]: {
Entity: {
$Kind: 'EntityType',
$Key: ['Id'],
...Object.fromEntries(
entityProps.map((p) => [
p.name,
{
$Type: p.type.replace('Edm.', ''),
$Nullable: p.nullable
}
])
)
},
NounType: {
$Kind: 'EnumType',
...Object.fromEntries(
Object.values(NounType).map((v, i) => [v, i])
)
}
},
[`${namespace}.Container`]: {
$Kind: 'EntityContainer',
Entities: {
$Collection: true,
$Type: `${namespace}.Entity`
}
}
}
if (includeRelationships) {
schema[namespace].Relationship = {
$Kind: 'EntityType',
$Key: ['Id'],
...Object.fromEntries(
relProps.map((p) => [
p.name,
{
$Type: p.type.replace('Edm.', ''),
$Nullable: p.nullable
}
])
)
}
schema[namespace].VerbType = {
$Kind: 'EnumType',
...Object.fromEntries(
Object.values(VerbType).map((v, i) => [v, i])
)
}
schema[`${namespace}.Container`].Relationships = {
$Collection: true,
$Type: `${namespace}.Relationship`
}
}
return schema
}
/**
* Get service document (root OData response)
*/
export function generateServiceDocument(
baseUrl: string,
options?: { includeRelationships?: boolean }
): object {
const includeRelationships = options?.includeRelationships ?? true
const collections = [
{
name: 'Entities',
kind: 'EntitySet',
url: 'Entities'
}
]
if (includeRelationships) {
collections.push({
name: 'Relationships',
kind: 'EntitySet',
url: 'Relationships'
})
}
return {
'@odata.context': `${baseUrl}/$metadata`,
value: collections
}
}
// Private helpers
function getEntityProperties(): PropertyDef[] {
return [
{ name: 'Id', type: 'Edm.String', nullable: false },
{ name: 'Type', type: 'Edm.String', nullable: false },
{ name: 'CreatedAt', type: 'Edm.DateTimeOffset', nullable: false },
{ name: 'UpdatedAt', type: 'Edm.DateTimeOffset', nullable: true },
{ name: 'Confidence', type: 'Edm.Double', nullable: true },
{ name: 'Weight', type: 'Edm.Double', nullable: true },
{ name: 'Service', type: 'Edm.String', nullable: true },
{ name: 'Data', type: 'Edm.String', nullable: true },
// Common metadata fields (flattened)
{ name: 'Metadata_name', type: 'Edm.String', nullable: true },
{ name: 'Metadata_title', type: 'Edm.String', nullable: true },
{ name: 'Metadata_description', type: 'Edm.String', nullable: true },
{ name: 'Metadata_email', type: 'Edm.String', nullable: true },
{ name: 'Metadata_url', type: 'Edm.String', nullable: true },
{ name: 'Metadata_tags', type: 'Edm.String', nullable: true },
{ name: 'Metadata_category', type: 'Edm.String', nullable: true },
{ name: 'Metadata_status', type: 'Edm.String', nullable: true },
{ name: 'Metadata_priority', type: 'Edm.Int32', nullable: true }
]
}
function getRelationshipProperties(): PropertyDef[] {
return [
{ name: 'Id', type: 'Edm.String', nullable: false },
{ name: 'FromId', type: 'Edm.String', nullable: false },
{ name: 'ToId', type: 'Edm.String', nullable: false },
{ name: 'Type', type: 'Edm.String', nullable: false },
{ name: 'Weight', type: 'Edm.Double', nullable: true },
{ name: 'Confidence', type: 'Edm.Double', nullable: true },
{ name: 'CreatedAt', type: 'Edm.DateTimeOffset', nullable: false },
{ name: 'UpdatedAt', type: 'Edm.DateTimeOffset', nullable: true },
{ name: 'Service', type: 'Edm.String', nullable: true },
{ name: 'Metadata', type: 'Edm.String', nullable: true }
]
}
function generateRelationshipType(properties: PropertyDef[]): string {
return ` <!-- Relationship Type -->
<EntityType Name="Relationship">
<Key>
<PropertyRef Name="Id"/>
</Key>
${properties.map((p) => ` <Property Name="${p.name}" Type="${p.type}" Nullable="${p.nullable}"/>`).join('\n')}
</EntityType>`
}
function generateVerbTypeEnum(): string {
return ` <!-- Enum: VerbType -->
<EnumType Name="VerbType">
${Object.values(VerbType)
.map((v, i) => ` <Member Name="${v}" Value="${i}"/>`)
.join('\n')}
</EnumType>`
}

View file

@ -0,0 +1,710 @@
/**
* OData Integration
*
* Exposes Brainy data via OData 4.0 REST API for:
* - Excel Power Query
* - Power BI
* - Tableau
* - Any OData-compatible BI tool
*
* Zero external dependencies - works in all environments.
*/
import {
IntegrationBase,
HTTPIntegration
} from '../core/IntegrationBase.js'
import { IntegrationConfig, ODataQueryOptions } from '../core/types.js'
import { AugmentationManifest } from '../../augmentations/manifest.js'
import { Entity, Relation, FindParams } from '../../types/brainy.types.js'
import { NounType } from '../../types/graphTypes.js'
import {
parseODataQuery,
applyFilter,
applySelect,
applyOrderBy,
applyPagination
} from './ODataQueryParser.js'
import {
generateEdmx,
generateMetadataJson,
generateServiceDocument
} from './EdmxGenerator.js'
/**
* OData integration configuration
*/
export interface ODataConfig extends IntegrationConfig {
/** Base path for OData routes (default: '/odata') */
basePath?: string
/** Port to listen on (only used when running standalone server) */
port?: number
/** Include relationships endpoint (default: true) */
includeRelationships?: boolean
/** Maximum page size (default: 1000) */
maxPageSize?: number
/** Default page size (default: 100) */
defaultPageSize?: number
/** Namespace for metadata (default: 'Brainy') */
namespace?: string
}
/**
* OData request context
*/
interface ODataRequest {
method: string
path: string
query: string
body?: any
headers: Record<string, string>
}
/**
* OData response
*/
interface ODataResponse {
status: number
headers: Record<string, string>
body: any
}
/**
* OData Integration
*
* Provides OData 4.0 REST API for Brainy data access from spreadsheets and BI tools.
*
* Routes:
* - GET /odata - Service document
* - GET /odata/$metadata - EDMX metadata
* - GET /odata/Entities - List entities with OData queries
* - GET /odata/Entities('id') - Get single entity
* - GET /odata/Relationships - List relationships
* - GET /odata/Relationships('id') - Get single relationship
* - POST /odata/Entities - Create entity
* - PATCH /odata/Entities('id') - Update entity
* - DELETE /odata/Entities('id') - Delete entity
*
* Supports: $filter, $select, $orderby, $top, $skip, $count, $search
*
* @example
* ```typescript
* // Register with Brainy
* brain.augmentations.register(new ODataIntegration({
* basePath: '/odata',
* maxPageSize: 1000
* }))
*
* // Connect from Excel Power Query:
* // Data → Get Data → From OData Feed → http://localhost:3000/odata
* ```
*/
export class ODataIntegration extends IntegrationBase implements HTTPIntegration {
readonly name = 'odata'
// HTTPIntegration
port: number
basePath: string
private odataConfig: ODataConfig & {
enabled: boolean
basePath: string
port: number
includeRelationships: boolean
maxPageSize: number
defaultPageSize: number
namespace: string
}
constructor(config?: ODataConfig) {
super(config)
this.odataConfig = {
enabled: config?.enabled ?? true,
basePath: config?.basePath ?? '/odata',
port: config?.port ?? 0,
includeRelationships: config?.includeRelationships ?? true,
maxPageSize: config?.maxPageSize ?? 1000,
defaultPageSize: config?.defaultPageSize ?? 100,
namespace: config?.namespace ?? 'Brainy',
rateLimit: config?.rateLimit,
auth: config?.auth,
cors: config?.cors
}
this.port = this.odataConfig.port
this.basePath = this.odataConfig.basePath
}
/**
* Start the integration (registers routes with API server)
*/
protected async onStart(): Promise<void> {
this.log('OData integration started')
}
/**
* Stop the integration
*/
protected async onStop(): Promise<void> {
this.log('OData integration stopped')
}
/**
* Handle an OData request
*
* This is the main entry point for processing OData requests.
* Can be called directly or integrated with an HTTP server.
*
* @param request OData request
* @returns OData response
*/
async handleRequest(request: ODataRequest): Promise<ODataResponse> {
this.recordRequest()
try {
const { method, path } = request
const relativePath = path.startsWith(this.basePath)
? path.slice(this.basePath.length)
: path
// Route the request
if (method === 'GET') {
if (relativePath === '' || relativePath === '/') {
return this.getServiceDocument(request)
}
if (relativePath === '/$metadata') {
return this.getMetadata(request)
}
if (relativePath.startsWith('/Entities')) {
return this.handleEntities(request, relativePath)
}
if (relativePath.startsWith('/Relationships')) {
return this.handleRelationships(request, relativePath)
}
}
if (method === 'POST') {
if (relativePath === '/Entities') {
return this.createEntity(request)
}
if (relativePath === '/Relationships') {
return this.createRelationship(request)
}
}
if (method === 'PATCH') {
if (relativePath.startsWith('/Entities(')) {
return this.updateEntity(request, relativePath)
}
}
if (method === 'DELETE') {
if (relativePath.startsWith('/Entities(')) {
return this.deleteEntity(request, relativePath)
}
if (relativePath.startsWith('/Relationships(')) {
return this.deleteRelationship(request, relativePath)
}
}
return this.errorResponse(404, 'Not Found')
} catch (error: any) {
this.recordError(error)
return this.errorResponse(500, error.message)
}
}
/**
* Get registered routes
*/
getRoutes(): Array<{ method: string; path: string; description: string }> {
const routes = [
{ method: 'GET', path: `${this.basePath}`, description: 'Service document' },
{ method: 'GET', path: `${this.basePath}/$metadata`, description: 'EDMX metadata' },
{ method: 'GET', path: `${this.basePath}/Entities`, description: 'List entities' },
{ method: 'GET', path: `${this.basePath}/Entities('id')`, description: 'Get entity' },
{ method: 'POST', path: `${this.basePath}/Entities`, description: 'Create entity' },
{ method: 'PATCH', path: `${this.basePath}/Entities('id')`, description: 'Update entity' },
{ method: 'DELETE', path: `${this.basePath}/Entities('id')`, description: 'Delete entity' }
]
if (this.odataConfig.includeRelationships) {
routes.push(
{ method: 'GET', path: `${this.basePath}/Relationships`, description: 'List relationships' },
{ method: 'GET', path: `${this.basePath}/Relationships('id')`, description: 'Get relationship' },
{ method: 'POST', path: `${this.basePath}/Relationships`, description: 'Create relationship' },
{ method: 'DELETE', path: `${this.basePath}/Relationships('id')`, description: 'Delete relationship' }
)
}
return routes
}
/**
* Get augmentation manifest
*/
getManifest(): AugmentationManifest {
return {
id: 'odata',
name: 'OData Integration',
version: '1.0.0',
description: 'OData 4.0 API for Excel, Power BI, and BI tools',
longDescription:
'Exposes Brainy data via OData 4.0 REST API, enabling direct connections from Excel Power Query, Power BI, Tableau, and any OData-compatible tool. Supports $filter, $select, $orderby, $top, $skip, $count queries.',
category: 'integration',
status: 'stable',
configSchema: {
type: 'object',
properties: {
enabled: { type: 'boolean', default: true },
basePath: { type: 'string', default: '/odata' },
maxPageSize: { type: 'number', default: 1000 },
defaultPageSize: { type: 'number', default: 100 },
includeRelationships: { type: 'boolean', default: true },
namespace: { type: 'string', default: 'Brainy' }
}
},
configDefaults: {
enabled: true,
basePath: '/odata',
maxPageSize: 1000,
defaultPageSize: 100,
includeRelationships: true,
namespace: 'Brainy'
},
features: [
'Excel Power Query support',
'Power BI direct connect',
'OData $filter queries',
'OData $select, $orderby',
'Pagination with $top/$skip',
'$count support',
'$search semantic search'
],
keywords: ['odata', 'excel', 'power-bi', 'bi', 'rest', 'api']
}
}
// Route handlers
private getServiceDocument(_request: ODataRequest): ODataResponse {
const baseUrl = `${this.basePath}`
return this.jsonResponse(
generateServiceDocument(baseUrl, {
includeRelationships: this.odataConfig.includeRelationships
})
)
}
private getMetadata(request: ODataRequest): ODataResponse {
// Check Accept header for JSON vs XML
const accept = request.headers['accept'] || ''
if (accept.includes('application/json')) {
return this.jsonResponse(
generateMetadataJson({
namespace: this.odataConfig.namespace,
includeRelationships: this.odataConfig.includeRelationships
})
)
}
// Default to XML EDMX
return {
status: 200,
headers: {
'Content-Type': 'application/xml',
'OData-Version': '4.0'
},
body: generateEdmx({
namespace: this.odataConfig.namespace,
includeRelationships: this.odataConfig.includeRelationships
})
}
}
private async handleEntities(
request: ODataRequest,
path: string
): Promise<ODataResponse> {
// Single entity: /Entities('id')
const idMatch = path.match(/^\/Entities\('([^']+)'\)/)
if (idMatch) {
return this.handleGetEntity(idMatch[1])
}
// Collection: /Entities?$filter=...
return this.listEntities(request)
}
private async listEntities(request: ODataRequest): Promise<ODataResponse> {
const options = parseODataQuery(request.query)
// Apply pagination limits
const pageSize = Math.min(
options.top ?? this.odataConfig.defaultPageSize,
this.odataConfig.maxPageSize
)
options.top = pageSize
// Query from Brainy
const findParams: FindParams = {
limit: pageSize + 1, // +1 to detect if there are more
offset: options.skip
}
// Apply orderby
if (options.orderBy && options.orderBy.length > 0) {
findParams.orderBy = options.orderBy[0].field
findParams.order = options.orderBy[0].direction
}
// Apply $search as semantic query
if (options.search) {
findParams.query = options.search
}
// Get entities
let entities = await this.queryEntities(findParams)
// Apply $filter (in-memory for complex filters)
if (options.filter) {
const rows = this.exporter.entitiesToRows(entities)
const filteredRows = applyFilter(rows, options.filter)
// Map back to entities (simple approach)
const filteredIds = new Set(filteredRows.map((r) => r.Id))
entities = entities.filter((e) => filteredIds.has(e.id))
}
// Check for more results
const hasMore = entities.length > pageSize
if (hasMore) {
entities = entities.slice(0, pageSize)
}
// Convert to tabular format
let rows = this.exporter.entitiesToRows(entities)
// Apply $select
if (options.select && options.select.length > 0) {
rows = applySelect(rows, options.select) as any
}
// Apply $orderby (if not handled by storage)
if (options.orderBy && options.orderBy.length > 0) {
rows = applyOrderBy(rows, options.orderBy)
}
// Build response
const result: any = {
'@odata.context': `${this.basePath}/$metadata#Entities`
}
// $count
if (options.count) {
// For accurate count, we'd need to query without limit
// For now, use the page count
result['@odata.count'] = entities.length
}
result.value = rows
// Next link for pagination
if (hasMore) {
const nextSkip = (options.skip ?? 0) + pageSize
result['@odata.nextLink'] = `${this.basePath}/Entities?$top=${pageSize}&$skip=${nextSkip}`
}
return this.jsonResponse(result)
}
private async handleGetEntity(id: string): Promise<ODataResponse> {
const entity = await super.getEntity(id)
if (!entity) {
return this.errorResponse(404, `Entity '${id}' not found`)
}
const row = this.exporter.entityToRow(entity)
return this.jsonResponse({
'@odata.context': `${this.basePath}/$metadata#Entities/$entity`,
...row
})
}
private async createEntity(request: ODataRequest): Promise<ODataResponse> {
if (!this.context) {
return this.errorResponse(500, 'Integration not initialized')
}
const body = request.body
if (!body || !body.Type) {
return this.errorResponse(400, 'Missing required field: Type')
}
const entity = await this.context.brain.add({
type: body.Type as NounType,
data: body.Data ? JSON.parse(body.Data) : undefined,
metadata: this.extractMetadata(body),
confidence: body.Confidence,
weight: body.Weight,
service: body.Service
})
const row = this.exporter.entityToRow(entity)
return {
status: 201,
headers: {
'Content-Type': 'application/json',
'OData-Version': '4.0',
'Location': `${this.basePath}/Entities('${entity.id}')`
},
body: JSON.stringify({
'@odata.context': `${this.basePath}/$metadata#Entities/$entity`,
...row
})
}
}
private async updateEntity(
request: ODataRequest,
path: string
): Promise<ODataResponse> {
if (!this.context) {
return this.errorResponse(500, 'Integration not initialized')
}
const idMatch = path.match(/^\/Entities\('([^']+)'\)/)
if (!idMatch) {
return this.errorResponse(400, 'Invalid entity ID')
}
const id = idMatch[1]
const body = request.body
const updates: any = { id }
if (body.Type) updates.type = body.Type as NounType
if (body.Data) updates.data = JSON.parse(body.Data)
if (body.Confidence !== undefined) updates.confidence = body.Confidence
if (body.Weight !== undefined) updates.weight = body.Weight
const metadata = this.extractMetadata(body)
if (Object.keys(metadata).length > 0) {
updates.metadata = metadata
updates.merge = true
}
await this.context.brain.update(updates)
return {
status: 204,
headers: { 'OData-Version': '4.0' },
body: null
}
}
private async deleteEntity(
_request: ODataRequest,
path: string
): Promise<ODataResponse> {
if (!this.context) {
return this.errorResponse(500, 'Integration not initialized')
}
const idMatch = path.match(/^\/Entities\('([^']+)'\)/)
if (!idMatch) {
return this.errorResponse(400, 'Invalid entity ID')
}
await this.context.brain.delete(idMatch[1])
return {
status: 204,
headers: { 'OData-Version': '4.0' },
body: null
}
}
private async handleRelationships(
request: ODataRequest,
path: string
): Promise<ODataResponse> {
// Single relationship: /Relationships('id')
const idMatch = path.match(/^\/Relationships\('([^']+)'\)/)
if (idMatch) {
return this.getRelationship(idMatch[1])
}
// Collection
return this.listRelationships(request)
}
private async listRelationships(
request: ODataRequest
): Promise<ODataResponse> {
const options = parseODataQuery(request.query)
const pageSize = Math.min(
options.top ?? this.odataConfig.defaultPageSize,
this.odataConfig.maxPageSize
)
const relations = await this.queryRelations({
limit: pageSize,
offset: options.skip
})
let rows = this.exporter.relationsToRows(relations)
// Apply $filter
if (options.filter) {
rows = applyFilter(rows, options.filter) as any
}
// Apply $select
if (options.select && options.select.length > 0) {
rows = applySelect(rows, options.select) as any
}
// Apply $orderby
if (options.orderBy && options.orderBy.length > 0) {
rows = applyOrderBy(rows, options.orderBy)
}
return this.jsonResponse({
'@odata.context': `${this.basePath}/$metadata#Relationships`,
value: rows
})
}
private async getRelationship(id: string): Promise<ODataResponse> {
const relations = await this.queryRelations({ limit: 1000 })
const relation = relations.find((r) => r.id === id)
if (!relation) {
return this.errorResponse(404, `Relationship '${id}' not found`)
}
const row = this.exporter.relationToRow(relation)
return this.jsonResponse({
'@odata.context': `${this.basePath}/$metadata#Relationships/$entity`,
...row
})
}
private async createRelationship(
request: ODataRequest
): Promise<ODataResponse> {
if (!this.context) {
return this.errorResponse(500, 'Integration not initialized')
}
const body = request.body
if (!body || !body.FromId || !body.ToId || !body.Type) {
return this.errorResponse(
400,
'Missing required fields: FromId, ToId, Type'
)
}
const relation = await this.context.brain.relate({
from: body.FromId,
to: body.ToId,
type: body.Type,
weight: body.Weight,
confidence: body.Confidence,
metadata: body.Metadata ? JSON.parse(body.Metadata) : undefined,
service: body.Service
})
const row = this.exporter.relationToRow(relation)
return {
status: 201,
headers: {
'Content-Type': 'application/json',
'OData-Version': '4.0',
'Location': `${this.basePath}/Relationships('${relation.id}')`
},
body: JSON.stringify({
'@odata.context': `${this.basePath}/$metadata#Relationships/$entity`,
...row
})
}
}
private async deleteRelationship(
_request: ODataRequest,
path: string
): Promise<ODataResponse> {
if (!this.context) {
return this.errorResponse(500, 'Integration not initialized')
}
const idMatch = path.match(/^\/Relationships\('([^']+)'\)/)
if (!idMatch) {
return this.errorResponse(400, 'Invalid relationship ID')
}
await this.context.brain.unrelate(idMatch[1])
return {
status: 204,
headers: { 'OData-Version': '4.0' },
body: null
}
}
// Helpers
private jsonResponse(data: any): ODataResponse {
return {
status: 200,
headers: {
'Content-Type': 'application/json',
'OData-Version': '4.0'
},
body: JSON.stringify(data)
}
}
private errorResponse(status: number, message: string): ODataResponse {
return {
status,
headers: {
'Content-Type': 'application/json',
'OData-Version': '4.0'
},
body: JSON.stringify({
error: {
code: String(status),
message
}
})
}
}
private extractMetadata(body: Record<string, any>): Record<string, any> {
const metadata: Record<string, any> = {}
const prefix = 'Metadata_'
for (const [key, value] of Object.entries(body)) {
if (key.startsWith(prefix)) {
const metaKey = key.slice(prefix.length)
metadata[metaKey] = value
}
}
return metadata
}
}

View file

@ -0,0 +1,650 @@
/**
* OData Query Parser
*
* Lightweight parser for OData query parameters without heavy dependencies.
* Supports $filter, $select, $orderby, $top, $skip, $count, $search, $expand.
*/
import { ODataQueryOptions } from '../core/types.js'
/**
* Parse OData filter expression to a structured filter object
*
* Supports:
* - eq, ne, gt, ge, lt, le comparisons
* - and, or logical operators
* - contains(), startswith(), endswith() string functions
* - Parentheses for grouping
*
* @param filter OData $filter string
* @returns Parsed filter object for query execution
*/
export function parseFilter(filter: string): any {
if (!filter || filter.trim() === '') {
return null
}
// Tokenize
const tokens = tokenize(filter)
// Parse expression
return parseExpression(tokens, 0).result
}
/**
* Parse OData $orderby specification
*/
export function parseOrderBy(
orderby: string
): Array<{ field: string; direction: 'asc' | 'desc' }> {
if (!orderby || orderby.trim() === '') {
return []
}
return orderby.split(',').map((part) => {
const trimmed = part.trim()
const parts = trimmed.split(/\s+/)
return {
field: parts[0],
direction: (parts[1]?.toLowerCase() as 'asc' | 'desc') || 'asc'
}
})
}
/**
* Parse OData $select specification
*/
export function parseSelect(select: string): string[] {
if (!select || select.trim() === '') {
return []
}
return select.split(',').map((s) => s.trim())
}
/**
* Parse OData $expand specification
*/
export function parseExpand(expand: string): string[] {
if (!expand || expand.trim() === '') {
return []
}
return expand.split(',').map((s) => s.trim())
}
/**
* Parse full OData query string to options object
*/
export function parseODataQuery(queryString: string): ODataQueryOptions {
const params = new URLSearchParams(queryString)
const options: ODataQueryOptions = {}
// $filter
const filter = params.get('$filter')
if (filter) {
options.filter = filter
}
// $select
const select = params.get('$select')
if (select) {
options.select = parseSelect(select)
}
// $orderby
const orderby = params.get('$orderby')
if (orderby) {
options.orderBy = parseOrderBy(orderby)
}
// $top
const top = params.get('$top')
if (top) {
options.top = parseInt(top, 10)
}
// $skip
const skip = params.get('$skip')
if (skip) {
options.skip = parseInt(skip, 10)
}
// $expand
const expand = params.get('$expand')
if (expand) {
options.expand = parseExpand(expand)
}
// $count
const count = params.get('$count')
if (count === 'true') {
options.count = true
}
// $search
const search = params.get('$search')
if (search) {
options.search = search
}
return options
}
/**
* Convert OData options to Brainy FindParams
*/
export function odataToFindParams(options: ODataQueryOptions): any {
const findParams: any = {}
// Top/Skip -> Limit/Offset
if (options.top !== undefined) {
findParams.limit = options.top
}
if (options.skip !== undefined) {
findParams.offset = options.skip
}
// OrderBy
if (options.orderBy && options.orderBy.length > 0) {
findParams.orderBy = options.orderBy[0].field
findParams.order = options.orderBy[0].direction
}
// Search -> Query
if (options.search) {
findParams.query = options.search
}
// Filter -> Where (simplified)
if (options.filter) {
const parsed = parseFilter(options.filter)
if (parsed) {
findParams.where = filterToWhere(parsed)
}
}
return findParams
}
/**
* Apply OData filter to an array of entities
*/
export function applyFilter<T extends Record<string, any>>(
entities: T[],
filter: string
): T[] {
if (!filter) return entities
const parsed = parseFilter(filter)
if (!parsed) return entities
return entities.filter((entity) => evaluateFilter(entity, parsed))
}
/**
* Apply OData select to transform entities
*/
export function applySelect<T extends Record<string, any>>(
entities: T[],
select: string[]
): Partial<T>[] {
if (!select || select.length === 0) return entities
return entities.map((entity) => {
const result: Partial<T> = {}
for (const field of select) {
if (field in entity) {
result[field as keyof T] = entity[field]
}
}
return result
})
}
/**
* Apply OData orderby to sort entities
*/
export function applyOrderBy<T extends Record<string, any>>(
entities: T[],
orderBy: Array<{ field: string; direction: 'asc' | 'desc' }>
): T[] {
if (!orderBy || orderBy.length === 0) return entities
return [...entities].sort((a, b) => {
for (const { field, direction } of orderBy) {
const aVal = getNestedValue(a, field)
const bVal = getNestedValue(b, field)
let cmp = 0
if (aVal === bVal) {
cmp = 0
} else if (aVal === null || aVal === undefined) {
cmp = 1
} else if (bVal === null || bVal === undefined) {
cmp = -1
} else if (typeof aVal === 'string' && typeof bVal === 'string') {
cmp = aVal.localeCompare(bVal)
} else {
cmp = aVal < bVal ? -1 : 1
}
if (cmp !== 0) {
return direction === 'desc' ? -cmp : cmp
}
}
return 0
})
}
/**
* Apply top/skip pagination
*/
export function applyPagination<T>(
entities: T[],
top?: number,
skip?: number
): T[] {
let result = entities
if (skip !== undefined && skip > 0) {
result = result.slice(skip)
}
if (top !== undefined && top > 0) {
result = result.slice(0, top)
}
return result
}
// Private helpers
type Token = {
type:
| 'identifier'
| 'string'
| 'number'
| 'boolean'
| 'null'
| 'operator'
| 'function'
| 'lparen'
| 'rparen'
| 'comma'
value: string
}
function tokenize(input: string): Token[] {
const tokens: Token[] = []
let i = 0
while (i < input.length) {
// Skip whitespace
if (/\s/.test(input[i])) {
i++
continue
}
// String literal
if (input[i] === "'") {
let str = ''
i++
while (i < input.length && input[i] !== "'") {
if (input[i] === "'" && input[i + 1] === "'") {
str += "'"
i += 2
} else {
str += input[i]
i++
}
}
i++ // Skip closing quote
tokens.push({ type: 'string', value: str })
continue
}
// Number
if (/\d/.test(input[i]) || (input[i] === '-' && /\d/.test(input[i + 1]))) {
let num = ''
while (i < input.length && /[\d.\-]/.test(input[i])) {
num += input[i]
i++
}
tokens.push({ type: 'number', value: num })
continue
}
// Parentheses
if (input[i] === '(') {
tokens.push({ type: 'lparen', value: '(' })
i++
continue
}
if (input[i] === ')') {
tokens.push({ type: 'rparen', value: ')' })
i++
continue
}
// Comma
if (input[i] === ',') {
tokens.push({ type: 'comma', value: ',' })
i++
continue
}
// Identifier or keyword
if (/[a-zA-Z_]/.test(input[i])) {
let ident = ''
while (i < input.length && /[a-zA-Z0-9_./]/.test(input[i])) {
ident += input[i]
i++
}
const lower = ident.toLowerCase()
// Operators
if (['eq', 'ne', 'gt', 'ge', 'lt', 'le', 'and', 'or', 'not'].includes(lower)) {
tokens.push({ type: 'operator', value: lower })
}
// Boolean
else if (lower === 'true' || lower === 'false') {
tokens.push({ type: 'boolean', value: lower })
}
// Null
else if (lower === 'null') {
tokens.push({ type: 'null', value: 'null' })
}
// Functions
else if (
['contains', 'startswith', 'endswith', 'tolower', 'toupper', 'length', 'trim', 'substring'].includes(
lower
)
) {
tokens.push({ type: 'function', value: lower })
}
// Identifier (field name)
else {
tokens.push({ type: 'identifier', value: ident })
}
continue
}
// Unknown character, skip
i++
}
return tokens
}
function parseExpression(
tokens: Token[],
pos: number
): { result: any; pos: number } {
return parseOr(tokens, pos)
}
function parseOr(
tokens: Token[],
pos: number
): { result: any; pos: number } {
let { result: left, pos: nextPos } = parseAnd(tokens, pos)
while (nextPos < tokens.length && tokens[nextPos]?.value === 'or') {
nextPos++ // Skip 'or'
const { result: right, pos: newPos } = parseAnd(tokens, nextPos)
left = { or: [left, right] }
nextPos = newPos
}
return { result: left, pos: nextPos }
}
function parseAnd(
tokens: Token[],
pos: number
): { result: any; pos: number } {
let { result: left, pos: nextPos } = parsePrimary(tokens, pos)
while (nextPos < tokens.length && tokens[nextPos]?.value === 'and') {
nextPos++ // Skip 'and'
const { result: right, pos: newPos } = parsePrimary(tokens, nextPos)
left = { and: [left, right] }
nextPos = newPos
}
return { result: left, pos: nextPos }
}
function parsePrimary(
tokens: Token[],
pos: number
): { result: any; pos: number } {
if (pos >= tokens.length) {
return { result: null, pos }
}
const token = tokens[pos]
// Parenthesized expression
if (token.type === 'lparen') {
const { result, pos: endPos } = parseExpression(tokens, pos + 1)
// Skip rparen
return { result, pos: endPos + 1 }
}
// Function call
if (token.type === 'function') {
return parseFunction(tokens, pos)
}
// Not operator
if (token.value === 'not') {
const { result, pos: endPos } = parsePrimary(tokens, pos + 1)
return { result: { not: result }, pos: endPos }
}
// Comparison: identifier operator value
if (token.type === 'identifier') {
const field = token.value
pos++
if (pos >= tokens.length || tokens[pos].type !== 'operator') {
return { result: { field, exists: true }, pos }
}
const op = tokens[pos].value
pos++
if (pos >= tokens.length) {
return { result: { field, op, value: null }, pos }
}
const valueToken = tokens[pos]
let value: any
switch (valueToken.type) {
case 'string':
value = valueToken.value
break
case 'number':
value = parseFloat(valueToken.value)
break
case 'boolean':
value = valueToken.value === 'true'
break
case 'null':
value = null
break
default:
value = valueToken.value
}
return { result: { field, op, value }, pos: pos + 1 }
}
return { result: null, pos: pos + 1 }
}
function parseFunction(
tokens: Token[],
pos: number
): { result: any; pos: number } {
const funcName = tokens[pos].value
pos++ // Skip function name
if (tokens[pos]?.type !== 'lparen') {
return { result: null, pos }
}
pos++ // Skip '('
const args: any[] = []
while (pos < tokens.length && tokens[pos]?.type !== 'rparen') {
if (tokens[pos].type === 'comma') {
pos++
continue
}
if (tokens[pos].type === 'identifier') {
args.push({ field: tokens[pos].value })
pos++
} else if (tokens[pos].type === 'string') {
args.push({ value: tokens[pos].value })
pos++
} else if (tokens[pos].type === 'number') {
args.push({ value: parseFloat(tokens[pos].value) })
pos++
} else {
pos++
}
}
pos++ // Skip ')'
return { result: { func: funcName, args }, pos }
}
function evaluateFilter(entity: Record<string, any>, filter: any): boolean {
if (!filter) return true
// Logical operators
if (filter.and) {
return filter.and.every((f: any) => evaluateFilter(entity, f))
}
if (filter.or) {
return filter.or.some((f: any) => evaluateFilter(entity, f))
}
if (filter.not) {
return !evaluateFilter(entity, filter.not)
}
// Function
if (filter.func) {
return evaluateFunction(entity, filter.func, filter.args)
}
// Comparison
if (filter.field && filter.op) {
const fieldValue = getNestedValue(entity, filter.field)
return compareValues(fieldValue, filter.op, filter.value)
}
return true
}
function evaluateFunction(
entity: Record<string, any>,
func: string,
args: any[]
): boolean {
if (args.length < 2) return false
const fieldValue = args[0].field
? String(getNestedValue(entity, args[0].field) ?? '')
: ''
const searchValue = args[1].value ?? ''
switch (func) {
case 'contains':
return fieldValue.toLowerCase().includes(searchValue.toLowerCase())
case 'startswith':
return fieldValue.toLowerCase().startsWith(searchValue.toLowerCase())
case 'endswith':
return fieldValue.toLowerCase().endsWith(searchValue.toLowerCase())
default:
return false
}
}
function compareValues(fieldValue: any, op: string, filterValue: any): boolean {
// Handle null comparisons
if (filterValue === null) {
switch (op) {
case 'eq':
return fieldValue === null || fieldValue === undefined
case 'ne':
return fieldValue !== null && fieldValue !== undefined
default:
return false
}
}
// Null field value
if (fieldValue === null || fieldValue === undefined) {
return op === 'ne'
}
// Type coercion for comparisons
let fv = fieldValue
let cv = filterValue
if (typeof filterValue === 'number' && typeof fieldValue === 'string') {
fv = parseFloat(fieldValue)
}
if (typeof filterValue === 'string' && typeof fieldValue === 'number') {
cv = parseFloat(filterValue)
}
switch (op) {
case 'eq':
return fv === cv || (typeof fv === 'string' && fv.toLowerCase() === String(cv).toLowerCase())
case 'ne':
return fv !== cv
case 'gt':
return fv > cv
case 'ge':
return fv >= cv
case 'lt':
return fv < cv
case 'le':
return fv <= cv
default:
return false
}
}
function getNestedValue(obj: Record<string, any>, path: string): any {
const parts = path.split(/[./]/)
let value: any = obj
for (const part of parts) {
if (value === null || value === undefined) return undefined
value = value[part]
}
return value
}
function filterToWhere(filter: any): any {
if (!filter) return {}
// Simple comparison
if (filter.field && filter.op && filter.op === 'eq') {
return { [filter.field]: filter.value }
}
// For more complex filters, return as-is for storage adapter to handle
return filter
}

View file

@ -0,0 +1,24 @@
/**
* OData Integration Module
*
* Provides OData 4.0 REST API for Excel, Power BI, and BI tools.
*/
export { ODataIntegration, type ODataConfig } from './ODataIntegration.js'
export {
parseODataQuery,
parseFilter,
parseOrderBy,
parseSelect,
parseExpand,
odataToFindParams,
applyFilter,
applySelect,
applyOrderBy,
applyPagination
} from './ODataQueryParser.js'
export {
generateEdmx,
generateMetadataJson,
generateServiceDocument
} from './EdmxGenerator.js'

View file

@ -0,0 +1,771 @@
/**
* Google Sheets Integration
*
* Provides REST API endpoints optimized for Google Apps Script to enable
* two-way sync between Brainy and Google Sheets.
*
* Features:
* - Custom function support (=BRAINY_QUERY(), =BRAINY_ADD())
* - Real-time updates via SSE subscription
* - Batch operations for performance
* - Simple authentication (API key or Bearer token)
*
* Zero external dependencies - works in all environments.
*/
import { IntegrationBase, HTTPIntegration } from '../core/IntegrationBase.js'
import { IntegrationConfig } from '../core/types.js'
import { AugmentationManifest } from '../../augmentations/manifest.js'
import { Entity, FindParams } from '../../types/brainy.types.js'
import { NounType, VerbType } from '../../types/graphTypes.js'
/**
* Google Sheets integration configuration
*/
export interface GoogleSheetsConfig extends IntegrationConfig {
/** Base path for API routes (default: '/sheets') */
basePath?: string
/** Port to listen on (only for standalone) */
port?: number
/** Maximum results per query (default: 1000) */
maxResults?: number
/** Default query limit (default: 100) */
defaultLimit?: number
/** Allow write operations (default: true) */
allowWrite?: boolean
/** CORS origins to allow (default: Google Sheets) */
allowedOrigins?: string[]
}
/**
* Sheets API request
*/
interface SheetsRequest {
method: string
path: string
query: Record<string, string>
body?: any
headers: Record<string, string>
}
/**
* Sheets API response
*/
interface SheetsResponse {
status: number
headers: Record<string, string>
body: any
}
/**
* Google Sheets Integration
*
* Provides a simple REST API optimized for Google Apps Script custom functions.
*
* Endpoints:
* - GET /sheets/query - Query entities (for =BRAINY_QUERY())
* - GET /sheets/entity/:id - Get single entity (for =BRAINY_GET())
* - GET /sheets/similar - Semantic search (for =BRAINY_SIMILAR())
* - GET /sheets/relations - Get relationships (for =BRAINY_RELATIONS())
* - POST /sheets/add - Add entity (for sidebar)
* - POST /sheets/batch - Batch operations (for range sync)
* - GET /sheets/schema - Get type schema (for sidebar dropdown)
* - GET /sheets/stream - SSE for real-time updates
*
* Response Format (optimized for Sheets):
* ```json
* {
* "headers": ["Id", "Type", "Name", "Email"],
* "rows": [
* ["uuid-1", "person", "John Doe", "john@example.com"],
* ["uuid-2", "person", "Jane Doe", "jane@example.com"]
* ],
* "count": 2,
* "hasMore": false
* }
* ```
*
* @example
* ```typescript
* brain.augmentations.register(new GoogleSheetsIntegration({
* basePath: '/sheets',
* maxResults: 1000
* }))
* ```
*/
export class GoogleSheetsIntegration
extends IntegrationBase
implements HTTPIntegration
{
readonly name = 'sheets'
port: number
basePath: string
private sheetsConfig: GoogleSheetsConfig & {
enabled: boolean
basePath: string
port: number
maxResults: number
defaultLimit: number
allowWrite: boolean
allowedOrigins: string[]
}
private sseClients: Map<string, (event: string, data: any) => void> =
new Map()
constructor(config?: GoogleSheetsConfig) {
super(config)
this.sheetsConfig = {
enabled: config?.enabled ?? true,
basePath: config?.basePath ?? '/sheets',
port: config?.port ?? 0,
maxResults: config?.maxResults ?? 1000,
defaultLimit: config?.defaultLimit ?? 100,
allowWrite: config?.allowWrite ?? true,
allowedOrigins: config?.allowedOrigins ?? [
'https://docs.google.com',
'https://script.google.com'
],
rateLimit: config?.rateLimit,
auth: config?.auth,
cors: config?.cors ?? {
origin: [
'https://docs.google.com',
'https://script.google.com',
'*'
],
methods: ['GET', 'POST', 'OPTIONS'],
credentials: true
}
}
this.port = this.sheetsConfig.port
this.basePath = this.sheetsConfig.basePath
}
protected async onStart(): Promise<void> {
// Subscribe to changes for real-time sync
this.subscribeToChanges(
{ entityTypes: ['noun', 'verb'] },
(event) => {
// Broadcast to all SSE clients
this.broadcastSSE('change', {
type: event.entityType,
operation: event.operation,
entityId: event.entityId,
timestamp: event.timestamp
})
}
)
this.log('Google Sheets integration started')
}
protected async onStop(): Promise<void> {
// Close all SSE connections
this.sseClients.clear()
this.log('Google Sheets integration stopped')
}
/**
* Handle a Sheets API request
*/
async handleRequest(request: SheetsRequest): Promise<SheetsResponse> {
this.recordRequest()
try {
const { method, path } = request
const relativePath = path.startsWith(this.basePath)
? path.slice(this.basePath.length)
: path
// CORS preflight
if (method === 'OPTIONS') {
return this.corsResponse()
}
// Route the request
if (method === 'GET') {
if (relativePath === '/query' || relativePath === '') {
return this.handleQuery(request)
}
if (relativePath.startsWith('/entity/')) {
const id = relativePath.slice('/entity/'.length)
return this.handleGetEntity(id)
}
if (relativePath === '/similar') {
return this.handleSimilar(request)
}
if (relativePath === '/relations') {
return this.handleRelations(request)
}
if (relativePath === '/schema') {
return this.handleSchema()
}
if (relativePath === '/stream') {
return this.handleStream(request)
}
if (relativePath === '/health') {
return this.handleHealth()
}
}
if (method === 'POST' && this.sheetsConfig.allowWrite) {
if (relativePath === '/add') {
return this.handleAdd(request)
}
if (relativePath === '/update') {
return this.handleUpdate(request)
}
if (relativePath === '/delete') {
return this.handleDelete(request)
}
if (relativePath === '/batch') {
return this.handleBatch(request)
}
if (relativePath === '/relate') {
return this.handleRelate(request)
}
}
return this.errorResponse(404, 'Not Found')
} catch (error: any) {
this.recordError(error)
return this.errorResponse(500, error.message)
}
}
/**
* Get registered routes
*/
getRoutes(): Array<{ method: string; path: string; description: string }> {
const routes = [
{ method: 'GET', path: `${this.basePath}/query`, description: 'Query entities' },
{ method: 'GET', path: `${this.basePath}/entity/:id`, description: 'Get entity by ID' },
{ method: 'GET', path: `${this.basePath}/similar`, description: 'Semantic search' },
{ method: 'GET', path: `${this.basePath}/relations`, description: 'Get relationships' },
{ method: 'GET', path: `${this.basePath}/schema`, description: 'Get schema' },
{ method: 'GET', path: `${this.basePath}/stream`, description: 'SSE stream' },
{ method: 'GET', path: `${this.basePath}/health`, description: 'Health check' }
]
if (this.sheetsConfig.allowWrite) {
routes.push(
{ method: 'POST', path: `${this.basePath}/add`, description: 'Add entity' },
{ method: 'POST', path: `${this.basePath}/update`, description: 'Update entity' },
{ method: 'POST', path: `${this.basePath}/delete`, description: 'Delete entity' },
{ method: 'POST', path: `${this.basePath}/batch`, description: 'Batch operations' },
{ method: 'POST', path: `${this.basePath}/relate`, description: 'Create relationship' }
)
}
return routes
}
/**
* Register an SSE client
*/
registerSSEClient(
clientId: string,
sendFn: (event: string, data: any) => void
): () => void {
this.sseClients.set(clientId, sendFn)
return () => this.sseClients.delete(clientId)
}
/**
* Get manifest
*/
getManifest(): AugmentationManifest {
return {
id: 'sheets',
name: 'Google Sheets Integration',
version: '1.0.0',
description: 'Two-way sync between Brainy and Google Sheets',
longDescription:
'Enables real-time bidirectional synchronization with Google Sheets. Use custom functions like =BRAINY_QUERY() directly in cells, or the sidebar for browsing and editing.',
category: 'integration',
status: 'stable',
configSchema: {
type: 'object',
properties: {
enabled: { type: 'boolean', default: true },
basePath: { type: 'string', default: '/sheets' },
maxResults: { type: 'number', default: 1000 },
defaultLimit: { type: 'number', default: 100 },
allowWrite: { type: 'boolean', default: true }
}
},
configDefaults: {
enabled: true,
basePath: '/sheets',
maxResults: 1000,
defaultLimit: 100,
allowWrite: true
},
features: [
'Custom functions (=BRAINY_QUERY)',
'Real-time sync via SSE',
'Batch operations',
'Semantic search support',
'Type schema discovery'
],
keywords: ['google-sheets', 'spreadsheet', 'sync', 'real-time']
}
}
// Route handlers
private async handleQuery(request: SheetsRequest): Promise<SheetsResponse> {
const { query } = request
// Build find params
const findParams: FindParams = {
limit: Math.min(
parseInt(query.limit) || this.sheetsConfig.defaultLimit,
this.sheetsConfig.maxResults
)
}
// Query string (semantic search)
if (query.q) {
findParams.query = query.q
}
// Type filter
if (query.type) {
const types = query.type.split(',') as NounType[]
findParams.type = types.length === 1 ? types[0] : types
}
// Offset pagination
if (query.offset) {
findParams.offset = parseInt(query.offset)
}
// Sort
if (query.orderBy) {
findParams.orderBy = query.orderBy
findParams.order = (query.order as 'asc' | 'desc') || 'desc'
}
// Execute query
const entities = await this.queryEntities(findParams)
// Convert to sheets format
return this.entitiesToSheetsResponse(entities)
}
private async handleGetEntity(id: string): Promise<SheetsResponse> {
const entity = await this.getEntity(id)
if (!entity) {
return this.errorResponse(404, 'Entity not found')
}
return this.entitiesToSheetsResponse([entity])
}
private async handleSimilar(request: SheetsRequest): Promise<SheetsResponse> {
if (!this.context) {
return this.errorResponse(500, 'Not initialized')
}
const { query } = request
if (!query.q && !query.to) {
return this.errorResponse(400, 'Missing q or to parameter')
}
const limit = Math.min(
parseInt(query.limit) || 10,
this.sheetsConfig.maxResults
)
// Semantic similarity search
let results: any[]
if (query.to) {
// Similar to entity
results = await this.context.brain.similar({
to: query.to,
limit,
threshold: query.threshold ? parseFloat(query.threshold) : undefined
})
} else {
// Similar to query text
results = await this.context.brain.find({
query: query.q,
limit
})
}
const entities = results.map((r: any) => r.entity || r)
return this.entitiesToSheetsResponse(entities)
}
private async handleRelations(
request: SheetsRequest
): Promise<SheetsResponse> {
const { query } = request
const params: any = {
limit: Math.min(
parseInt(query.limit) || 100,
this.sheetsConfig.maxResults
)
}
if (query.from) params.from = query.from
if (query.to) params.to = query.to
if (query.type) params.type = query.type as VerbType
const relations = await this.queryRelations(params)
// Convert to sheets format
const headers = [
'Id',
'FromId',
'ToId',
'Type',
'Weight',
'Confidence',
'CreatedAt'
]
const rows = relations.map((r) => [
r.id,
r.from,
r.to,
r.type,
r.weight ?? 1,
r.confidence ?? 1,
new Date(r.createdAt).toISOString()
])
return this.jsonResponse({
headers,
rows,
count: rows.length,
hasMore: false
})
}
private handleSchema(): SheetsResponse {
return this.jsonResponse({
nounTypes: Object.values(NounType),
verbTypes: Object.values(VerbType),
entityFields: ['id', 'type', 'data', 'metadata', 'confidence', 'weight', 'service', 'createdAt', 'updatedAt'],
commonMetadataFields: ['name', 'title', 'description', 'email', 'url', 'tags', 'category', 'status', 'priority']
})
}
private handleStream(_request: SheetsRequest): SheetsResponse {
// This returns headers for SSE - actual streaming is handled by the HTTP server
return {
status: 200,
headers: {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
Connection: 'keep-alive',
'Access-Control-Allow-Origin': '*'
},
body: 'sse' // Signal to HTTP server to handle as SSE
}
}
private handleHealth(): SheetsResponse {
return this.jsonResponse({
status: 'ok',
integration: 'sheets',
uptime: this.startedAt ? Date.now() - this.startedAt : 0,
requests: this.requestCount
})
}
private async handleAdd(request: SheetsRequest): Promise<SheetsResponse> {
if (!this.context) {
return this.errorResponse(500, 'Not initialized')
}
const { body } = request
if (!body || !body.type) {
return this.errorResponse(400, 'Missing required field: type')
}
const entity = await this.context.brain.add({
type: body.type as NounType,
data: body.data,
metadata: body.metadata,
confidence: body.confidence,
weight: body.weight,
service: body.service
})
return this.entitiesToSheetsResponse([entity])
}
private async handleUpdate(request: SheetsRequest): Promise<SheetsResponse> {
if (!this.context) {
return this.errorResponse(500, 'Not initialized')
}
const { body } = request
if (!body || !body.id) {
return this.errorResponse(400, 'Missing required field: id')
}
await this.context.brain.update({
id: body.id,
data: body.data,
metadata: body.metadata,
type: body.type as NounType,
confidence: body.confidence,
weight: body.weight,
merge: body.merge ?? true
})
const updated = await this.getEntity(body.id)
return this.entitiesToSheetsResponse(updated ? [updated] : [])
}
private async handleDelete(request: SheetsRequest): Promise<SheetsResponse> {
if (!this.context) {
return this.errorResponse(500, 'Not initialized')
}
const { body } = request
if (!body || !body.id) {
return this.errorResponse(400, 'Missing required field: id')
}
await this.context.brain.delete(body.id)
return this.jsonResponse({ success: true, deleted: body.id })
}
private async handleBatch(request: SheetsRequest): Promise<SheetsResponse> {
if (!this.context) {
return this.errorResponse(500, 'Not initialized')
}
const { body } = request
if (!body || !Array.isArray(body.operations)) {
return this.errorResponse(400, 'Missing operations array')
}
const results: any[] = []
for (const op of body.operations) {
try {
switch (op.action) {
case 'add':
const added = await this.context.brain.add({
type: op.type as NounType,
data: op.data,
metadata: op.metadata
})
results.push({ success: true, id: added.id, action: 'add' })
break
case 'update':
await this.context.brain.update({
id: op.id,
data: op.data,
metadata: op.metadata,
merge: true
})
results.push({ success: true, id: op.id, action: 'update' })
break
case 'delete':
await this.context.brain.delete(op.id)
results.push({ success: true, id: op.id, action: 'delete' })
break
default:
results.push({
success: false,
error: `Unknown action: ${op.action}`
})
}
} catch (error: any) {
results.push({
success: false,
id: op.id,
action: op.action,
error: error.message
})
}
}
return this.jsonResponse({
results,
total: body.operations.length,
successful: results.filter((r) => r.success).length,
failed: results.filter((r) => !r.success).length
})
}
private async handleRelate(request: SheetsRequest): Promise<SheetsResponse> {
if (!this.context) {
return this.errorResponse(500, 'Not initialized')
}
const { body } = request
if (!body || !body.from || !body.to || !body.type) {
return this.errorResponse(400, 'Missing required fields: from, to, type')
}
const relation = await this.context.brain.relate({
from: body.from,
to: body.to,
type: body.type as VerbType,
weight: body.weight,
metadata: body.metadata
})
return this.jsonResponse({
success: true,
relation: {
id: relation.id,
from: relation.from,
to: relation.to,
type: relation.type
}
})
}
// Helpers
private entitiesToSheetsResponse(entities: Entity[]): SheetsResponse {
if (entities.length === 0) {
return this.jsonResponse({
headers: ['Id', 'Type', 'CreatedAt'],
rows: [],
count: 0,
hasMore: false
})
}
// Collect all unique metadata keys
const metadataKeys = new Set<string>()
for (const entity of entities) {
if (entity.metadata) {
for (const key of Object.keys(entity.metadata)) {
metadataKeys.add(key)
}
}
}
// Build headers
const baseHeaders = ['Id', 'Type', 'Confidence', 'Weight', 'CreatedAt']
const metaHeaders = Array.from(metadataKeys).sort()
const headers = [...baseHeaders, ...metaHeaders, 'Data']
// Build rows
const rows = entities.map((entity) => {
const baseValues = [
entity.id,
entity.type,
entity.confidence ?? '',
entity.weight ?? '',
new Date(entity.createdAt).toISOString()
]
const metaValues = metaHeaders.map((key) => {
const val = entity.metadata?.[key]
if (val === undefined || val === null) return ''
if (typeof val === 'object') return JSON.stringify(val)
return val
})
const dataValue = entity.data ? JSON.stringify(entity.data) : ''
return [...baseValues, ...metaValues, dataValue]
})
return this.jsonResponse({
headers,
rows,
count: rows.length,
hasMore: false
})
}
private broadcastSSE(event: string, data: any): void {
for (const [_, sendFn] of this.sseClients) {
try {
sendFn(event, data)
} catch {
// Client disconnected
}
}
}
private jsonResponse(data: any): SheetsResponse {
return {
status: 200,
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Methods': 'GET, POST, OPTIONS',
'Access-Control-Allow-Headers': 'Content-Type, Authorization'
},
body: JSON.stringify(data)
}
}
private errorResponse(status: number, message: string): SheetsResponse {
return {
status,
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*'
},
body: JSON.stringify({ error: message })
}
}
private corsResponse(): SheetsResponse {
return {
status: 204,
headers: {
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Methods': 'GET, POST, OPTIONS',
'Access-Control-Allow-Headers': 'Content-Type, Authorization',
'Access-Control-Max-Age': '86400'
},
body: null
}
}
}
/**
* Package export for @soulcraft/brainy-sheets
*/
export const integration = {
name: 'sheets',
version: '1.0.0',
description: 'Google Sheets two-way sync with real-time updates',
environments: ['node', 'browser', 'deno', 'cloudflare', 'bun'],
create: (brain: any, config?: GoogleSheetsConfig) =>
new GoogleSheetsIntegration(config),
defaultConfig: {
basePath: '/sheets',
maxResults: 1000,
defaultLimit: 100,
allowWrite: true
}
}

View file

@ -0,0 +1,12 @@
/**
* Google Sheets Integration Module
*
* Provides two-way sync between Brainy and Google Sheets.
* Works in all environments (Node.js, browser, Deno, Cloudflare, Bun).
*/
export {
GoogleSheetsIntegration,
integration,
type GoogleSheetsConfig
} from './GoogleSheetsIntegration.js'

View file

@ -0,0 +1,449 @@
/**
* SSE (Server-Sent Events) Integration
*
* Provides real-time streaming of Brainy events to clients.
* Universal - works in all environments.
*/
import {
IntegrationBase,
HTTPIntegration,
StreamingIntegration
} from '../core/IntegrationBase.js'
import {
IntegrationConfig,
EventFilter,
BrainyEvent
} from '../core/types.js'
import { AugmentationManifest } from '../../augmentations/manifest.js'
/**
* SSE integration configuration
*/
export interface SSEConfig extends IntegrationConfig {
/** Base path for SSE endpoint (default: '/events') */
basePath?: string
/** Heartbeat interval in ms (default: 30000) */
heartbeatInterval?: number
/** Maximum clients (default: 1000) */
maxClients?: number
/** Include full entity data in events (default: false) */
includeData?: boolean
}
/**
* SSE client connection
*/
interface SSEClient {
id: string
filter: EventFilter
send: (event: string, data: any, id?: string) => void
close: () => void
lastEventId?: string
connectedAt: number
}
/**
* SSE Integration
*
* Enables real-time streaming of Brainy changes via Server-Sent Events.
* Clients can subscribe to specific event types and receive updates instantly.
*
* Endpoint:
* - GET /events - SSE stream
*
* Query Parameters:
* - types: Entity types to subscribe to (noun,verb,vfs)
* - operations: Operations to subscribe to (create,update,delete)
* - nounTypes: Specific noun types (person,document,...)
* - since: Resume from sequence ID (Last-Event-ID)
*
* @example
* ```typescript
* brain.augmentations.register(new SSEIntegration({
* basePath: '/events',
* heartbeatInterval: 30000
* }))
*
* // Client-side:
* const source = new EventSource('/events?types=noun&operations=create,update')
* source.onmessage = (event) => {
* const data = JSON.parse(event.data)
* console.log('Change:', data)
* }
* ```
*/
export class SSEIntegration
extends IntegrationBase
implements HTTPIntegration, StreamingIntegration
{
readonly name = 'sse'
port: number
basePath: string
private sseConfig: SSEConfig & {
enabled: boolean
basePath: string
heartbeatInterval: number
maxClients: number
includeData: boolean
}
private clients: Map<string, SSEClient> = new Map()
private heartbeatTimer?: ReturnType<typeof setInterval>
private clientIdCounter = 0
constructor(config?: SSEConfig) {
super(config)
this.sseConfig = {
enabled: config?.enabled ?? true,
basePath: config?.basePath ?? '/events',
heartbeatInterval: config?.heartbeatInterval ?? 30000,
maxClients: config?.maxClients ?? 1000,
includeData: config?.includeData ?? false,
rateLimit: config?.rateLimit,
auth: config?.auth,
cors: config?.cors
}
this.port = 0
this.basePath = this.sseConfig.basePath
}
protected async onStart(): Promise<void> {
// Subscribe to all Brainy events
this.subscribeToChanges({}, (event) => {
this.broadcastEvent(event)
})
// Start heartbeat
this.heartbeatTimer = setInterval(() => {
this.sendHeartbeat()
}, this.sseConfig.heartbeatInterval)
this.log('SSE integration started')
}
protected async onStop(): Promise<void> {
// Clear heartbeat
if (this.heartbeatTimer) {
clearInterval(this.heartbeatTimer)
}
// Close all clients
for (const [_, client] of this.clients) {
client.close()
}
this.clients.clear()
this.log('SSE integration stopped')
}
/**
* Handle incoming HTTP request for SSE
*/
handleRequest(request: {
method: string
path: string
query: Record<string, string>
headers: Record<string, string>
}): {
status: number
headers: Record<string, string>
body: string | null
isSSE: boolean
} {
const { method, path, query, headers } = request
if (method !== 'GET') {
return {
status: 405,
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ error: 'Method not allowed' }),
isSSE: false
}
}
const relativePath = path.startsWith(this.basePath)
? path.slice(this.basePath.length)
: path
if (relativePath !== '' && relativePath !== '/') {
return {
status: 404,
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ error: 'Not found' }),
isSSE: false
}
}
// Parse filter from query params
const filter = this.parseFilter(query, headers)
// Return SSE headers - actual stream handling is done by the server
return {
status: 200,
headers: {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
Connection: 'keep-alive',
'Access-Control-Allow-Origin': '*',
'X-Accel-Buffering': 'no'
},
body: null,
isSSE: true
}
}
/**
* Register an SSE client (called by HTTP server)
*/
registerClient(
sendFn: (event: string, data: any, id?: string) => void,
closeFn: () => void,
query: Record<string, string>,
headers: Record<string, string>
): string {
if (this.clients.size >= this.sseConfig.maxClients) {
throw new Error('Maximum clients reached')
}
const clientId = `sse-${++this.clientIdCounter}-${Date.now()}`
const filter = this.parseFilter(query, headers)
const client: SSEClient = {
id: clientId,
filter,
send: sendFn,
close: closeFn,
lastEventId: headers['last-event-id'],
connectedAt: Date.now()
}
this.clients.set(clientId, client)
this.recordRequest()
// Send initial connection event
sendFn('connected', { clientId, filter }, clientId)
// Replay missed events if resuming
if (filter.since !== undefined) {
const missed = this.eventBus.getEventsSince(filter.since)
for (const event of missed) {
if (this.matchesFilter(event, filter)) {
sendFn('change', this.formatEvent(event), String(event.sequenceId))
}
}
}
this.log(`SSE client connected: ${clientId}`)
return clientId
}
/**
* Unregister an SSE client
*/
unregisterClient(clientId: string): void {
this.clients.delete(clientId)
this.log(`SSE client disconnected: ${clientId}`)
}
/**
* Get connected client count
*/
getClientCount(): number {
return this.clients.size
}
/**
* Stream implementation for StreamingIntegration interface
*/
stream(
filter: EventFilter,
callback: (event: any) => void
): { close: () => void } {
const subscription = this.eventBus.subscribe(filter, (event) => {
callback(this.formatEvent(event))
})
return {
close: () => subscription.unsubscribe()
}
}
/**
* Get routes
*/
getRoutes(): Array<{ method: string; path: string; description: string }> {
return [
{
method: 'GET',
path: this.basePath,
description: 'Server-Sent Events stream'
}
]
}
/**
* Get manifest
*/
getManifest(): AugmentationManifest {
return {
id: 'sse',
name: 'SSE Streaming',
version: '1.0.0',
description: 'Real-time event streaming via Server-Sent Events',
longDescription:
'Enables real-time streaming of Brainy changes to web clients, Google Sheets, and dashboards via standard Server-Sent Events.',
category: 'integration',
status: 'stable',
configSchema: {
type: 'object',
properties: {
enabled: { type: 'boolean', default: true },
basePath: { type: 'string', default: '/events' },
heartbeatInterval: { type: 'number', default: 30000 },
maxClients: { type: 'number', default: 1000 },
includeData: { type: 'boolean', default: false }
}
},
configDefaults: {
enabled: true,
basePath: '/events',
heartbeatInterval: 30000,
maxClients: 1000,
includeData: false
},
features: [
'Real-time streaming',
'Event filtering',
'Automatic reconnection support',
'Event replay on reconnect',
'Heartbeat keep-alive'
],
keywords: ['sse', 'streaming', 'real-time', 'events']
}
}
// Private methods
private parseFilter(
query: Record<string, string>,
headers: Record<string, string>
): EventFilter {
const filter: EventFilter = {}
if (query.types) {
filter.entityTypes = query.types.split(',') as any[]
}
if (query.operations) {
filter.operations = query.operations.split(',') as any[]
}
if (query.nounTypes) {
filter.nounTypes = query.nounTypes.split(',') as any[]
}
if (query.verbTypes) {
filter.verbTypes = query.verbTypes.split(',') as any[]
}
if (query.service) {
filter.service = query.service
}
// Resume from Last-Event-ID header or query param
const since = headers['last-event-id'] || query.since
if (since) {
filter.since = BigInt(since)
}
return filter
}
private matchesFilter(event: BrainyEvent, filter: EventFilter): boolean {
if (filter.entityTypes?.length && !filter.entityTypes.includes(event.entityType)) {
return false
}
if (filter.operations?.length && !filter.operations.includes(event.operation)) {
return false
}
if (filter.nounTypes?.length && event.nounType && !filter.nounTypes.includes(event.nounType)) {
return false
}
if (filter.verbTypes?.length && event.verbType && !filter.verbTypes.includes(event.verbType)) {
return false
}
if (filter.service && event.service !== filter.service) {
return false
}
return true
}
private formatEvent(event: BrainyEvent): any {
const formatted: any = {
id: event.id,
type: event.entityType,
operation: event.operation,
entityId: event.entityId,
timestamp: event.timestamp
}
if (event.nounType) formatted.nounType = event.nounType
if (event.verbType) formatted.verbType = event.verbType
if (event.service) formatted.service = event.service
if (this.sseConfig.includeData && event.data) {
formatted.data = event.data
}
return formatted
}
private broadcastEvent(event: BrainyEvent): void {
const eventId = String(event.sequenceId)
for (const [_, client] of this.clients) {
if (this.matchesFilter(event, client.filter)) {
try {
client.send('change', this.formatEvent(event), eventId)
} catch {
// Client disconnected, will be cleaned up
}
}
}
}
private sendHeartbeat(): void {
const timestamp = Date.now()
for (const [_, client] of this.clients) {
try {
client.send('heartbeat', { timestamp })
} catch {
// Client disconnected
}
}
}
}
/**
* Package export for @soulcraft/brainy-sse
*/
export const integration = {
name: 'sse',
version: '1.0.0',
description: 'Real-time event streaming via Server-Sent Events',
environments: ['node', 'browser', 'deno', 'cloudflare', 'bun'],
create: (brain: any, config?: SSEConfig) => new SSEIntegration(config),
defaultConfig: {
basePath: '/events',
heartbeatInterval: 30000,
maxClients: 1000
}
}

View file

@ -0,0 +1,12 @@
/**
* SSE (Server-Sent Events) Integration Module
*
* Provides real-time streaming of Brainy events.
* Works in all environments.
*/
export {
SSEIntegration,
integration,
type SSEConfig
} from './SSEIntegration.js'

View file

@ -0,0 +1,548 @@
/**
* Webhook Integration
*
* Delivers Brainy events to external URLs via HTTP POST.
* Supports HMAC signing, retry policies, and batching.
*
* Zero external dependencies - works in all environments.
*/
import { IntegrationBase } from '../core/IntegrationBase.js'
import {
IntegrationConfig,
EventFilter,
WebhookRegistration,
WebhookDeliveryResult,
BrainyEvent
} from '../core/types.js'
import { AugmentationManifest } from '../../augmentations/manifest.js'
/**
* Webhook integration configuration
*/
export interface WebhookConfig extends IntegrationConfig {
/** Default retry policy */
defaultRetryPolicy?: {
maxRetries: number
backoffMultiplier: number
initialDelayMs: number
maxDelayMs: number
}
/** Batch delivery settings */
batch?: {
enabled: boolean
maxSize: number
maxDelayMs: number
}
/** Request timeout in ms (default: 30000) */
timeout?: number
/** Include full entity data (default: false) */
includeData?: boolean
}
/**
* Pending delivery
*/
interface PendingDelivery {
webhook: WebhookRegistration
events: BrainyEvent[]
attempts: number
nextAttempt: number
}
/**
* Webhook Integration
*
* Enables push notifications to external systems when Brainy data changes.
* Supports HMAC-SHA256 signing for security, automatic retry with exponential
* backoff, and optional event batching.
*
* Methods:
* - register(webhook) - Register a new webhook
* - unregister(id) - Remove a webhook
* - list() - List all webhooks
* - getDeliveryHistory(webhookId) - Get delivery history
*
* @example
* ```typescript
* const webhooks = new WebhookIntegration()
* brain.augmentations.register(webhooks)
*
* // Register a webhook
* await webhooks.register({
* url: 'https://example.com/webhook',
* events: { entityTypes: ['noun'], operations: ['create', 'update'] },
* secret: 'my-signing-secret'
* })
* ```
*/
export class WebhookIntegration extends IntegrationBase {
readonly name = 'webhooks'
private webhookConfig: WebhookConfig & {
enabled: boolean
defaultRetryPolicy: {
maxRetries: number
backoffMultiplier: number
initialDelayMs: number
maxDelayMs: number
}
batch: {
enabled: boolean
maxSize: number
maxDelayMs: number
}
timeout: number
includeData: boolean
}
private webhooks: Map<string, WebhookRegistration> = new Map()
private pendingDeliveries: PendingDelivery[] = []
private deliveryHistory: WebhookDeliveryResult[] = []
private deliveryTimer?: ReturnType<typeof setInterval>
private batchBuffer: Map<string, BrainyEvent[]> = new Map()
private batchTimer?: ReturnType<typeof setTimeout>
private webhookIdCounter = 0
constructor(config?: WebhookConfig) {
super(config)
this.webhookConfig = {
enabled: config?.enabled ?? true,
defaultRetryPolicy: config?.defaultRetryPolicy ?? {
maxRetries: 5,
backoffMultiplier: 2,
initialDelayMs: 1000,
maxDelayMs: 60000
},
batch: config?.batch ?? {
enabled: false,
maxSize: 100,
maxDelayMs: 5000
},
timeout: config?.timeout ?? 30000,
includeData: config?.includeData ?? false,
rateLimit: config?.rateLimit,
auth: config?.auth,
cors: config?.cors
}
}
protected async onStart(): Promise<void> {
// Subscribe to all events
this.subscribeToChanges({}, (event) => {
this.handleEvent(event)
})
// Start delivery processor
this.deliveryTimer = setInterval(() => {
this.processDeliveries()
}, 1000)
this.log('Webhook integration started')
}
protected async onStop(): Promise<void> {
if (this.deliveryTimer) {
clearInterval(this.deliveryTimer)
}
if (this.batchTimer) {
clearTimeout(this.batchTimer)
}
// Attempt to deliver remaining events
await this.flushBatches()
await this.processDeliveries()
this.log('Webhook integration stopped')
}
/**
* Register a new webhook
*/
async register(
webhook: Omit<WebhookRegistration, 'id' | 'active' | 'createdAt'>
): Promise<WebhookRegistration> {
const id = `wh-${++this.webhookIdCounter}-${Date.now()}`
const registration: WebhookRegistration = {
id,
url: webhook.url,
events: webhook.events,
secret: webhook.secret,
active: true,
retryPolicy: webhook.retryPolicy ?? this.webhookConfig.defaultRetryPolicy,
createdAt: Date.now()
}
this.webhooks.set(id, registration)
this.log(`Registered webhook: ${id} -> ${webhook.url}`)
return registration
}
/**
* Unregister a webhook
*/
async unregister(id: string): Promise<boolean> {
const existed = this.webhooks.delete(id)
if (existed) {
this.log(`Unregistered webhook: ${id}`)
}
return existed
}
/**
* List all webhooks
*/
list(): WebhookRegistration[] {
return Array.from(this.webhooks.values())
}
/**
* Get a specific webhook
*/
get(id: string): WebhookRegistration | undefined {
return this.webhooks.get(id)
}
/**
* Update a webhook
*/
async update(
id: string,
updates: Partial<Omit<WebhookRegistration, 'id' | 'createdAt'>>
): Promise<WebhookRegistration | null> {
const webhook = this.webhooks.get(id)
if (!webhook) return null
const updated = { ...webhook, ...updates }
this.webhooks.set(id, updated)
return updated
}
/**
* Get delivery history for a webhook
*/
getDeliveryHistory(
webhookId?: string,
limit = 100
): WebhookDeliveryResult[] {
let history = this.deliveryHistory
if (webhookId) {
history = history.filter((d) => d.webhookId === webhookId)
}
return history.slice(-limit)
}
/**
* Manually trigger a test delivery
*/
async testDelivery(webhookId: string): Promise<WebhookDeliveryResult> {
const webhook = this.webhooks.get(webhookId)
if (!webhook) {
throw new Error(`Webhook not found: ${webhookId}`)
}
const testEvent: BrainyEvent = {
id: 'test-event',
entityType: 'noun',
operation: 'create',
entityId: 'test-entity',
timestamp: Date.now(),
sequenceId: 0n
}
return this.deliverEvent(webhook, [testEvent])
}
/**
* Get manifest
*/
getManifest(): AugmentationManifest {
return {
id: 'webhooks',
name: 'Webhooks',
version: '1.0.0',
description: 'Push events to external URLs',
longDescription:
'Delivers Brainy events to external webhooks via HTTP POST. Supports HMAC-SHA256 signing, exponential backoff retry, and event batching.',
category: 'integration',
status: 'stable',
configSchema: {
type: 'object',
properties: {
enabled: { type: 'boolean', default: true },
timeout: { type: 'number', default: 30000 },
includeData: { type: 'boolean', default: false }
}
},
configDefaults: {
enabled: true,
timeout: 30000,
includeData: false
},
features: [
'HMAC-SHA256 signing',
'Exponential backoff retry',
'Event batching',
'Delivery history tracking'
],
keywords: ['webhooks', 'push', 'notifications', 'events']
}
}
// Private methods
private handleEvent(event: BrainyEvent): void {
for (const [_, webhook] of this.webhooks) {
if (!webhook.active) continue
if (!this.matchesFilter(event, webhook.events)) continue
if (this.webhookConfig.batch.enabled) {
this.addToBatch(webhook.id, event)
} else {
this.queueDelivery(webhook, [event])
}
}
}
private matchesFilter(event: BrainyEvent, filter: EventFilter): boolean {
if (
filter.entityTypes?.length &&
!filter.entityTypes.includes(event.entityType)
) {
return false
}
if (
filter.operations?.length &&
!filter.operations.includes(event.operation)
) {
return false
}
if (
filter.nounTypes?.length &&
event.nounType &&
!filter.nounTypes.includes(event.nounType)
) {
return false
}
return true
}
private addToBatch(webhookId: string, event: BrainyEvent): void {
let batch = this.batchBuffer.get(webhookId)
if (!batch) {
batch = []
this.batchBuffer.set(webhookId, batch)
}
batch.push(event)
// Flush if batch is full
if (batch.length >= this.webhookConfig.batch.maxSize) {
this.flushBatch(webhookId)
}
// Set timer for delayed flush
if (!this.batchTimer) {
this.batchTimer = setTimeout(() => {
this.flushBatches()
this.batchTimer = undefined
}, this.webhookConfig.batch.maxDelayMs)
}
}
private flushBatch(webhookId: string): void {
const batch = this.batchBuffer.get(webhookId)
if (!batch || batch.length === 0) return
const webhook = this.webhooks.get(webhookId)
if (webhook) {
this.queueDelivery(webhook, [...batch])
}
this.batchBuffer.delete(webhookId)
}
private async flushBatches(): Promise<void> {
for (const webhookId of this.batchBuffer.keys()) {
this.flushBatch(webhookId)
}
}
private queueDelivery(
webhook: WebhookRegistration,
events: BrainyEvent[]
): void {
this.pendingDeliveries.push({
webhook,
events,
attempts: 0,
nextAttempt: Date.now()
})
}
private async processDeliveries(): Promise<void> {
const now = Date.now()
const ready = this.pendingDeliveries.filter((d) => d.nextAttempt <= now)
for (const delivery of ready) {
const result = await this.deliverEvent(delivery.webhook, delivery.events)
// Remove from pending
const index = this.pendingDeliveries.indexOf(delivery)
if (index >= 0) {
this.pendingDeliveries.splice(index, 1)
}
// Track history
this.deliveryHistory.push(result)
if (this.deliveryHistory.length > 1000) {
this.deliveryHistory.shift()
}
// Retry if failed
if (!result.success && delivery.attempts < (delivery.webhook.retryPolicy?.maxRetries ?? 5)) {
const policy = delivery.webhook.retryPolicy ?? this.webhookConfig.defaultRetryPolicy
const delay = Math.min(
policy.initialDelayMs * Math.pow(policy.backoffMultiplier, delivery.attempts),
policy.maxDelayMs
)
this.pendingDeliveries.push({
...delivery,
attempts: delivery.attempts + 1,
nextAttempt: now + delay
})
} else if (!result.success) {
// Max retries reached - update webhook failure count
const webhook = this.webhooks.get(delivery.webhook.id)
if (webhook) {
webhook.failureCount = (webhook.failureCount ?? 0) + 1
}
}
}
}
private async deliverEvent(
webhook: WebhookRegistration,
events: BrainyEvent[]
): Promise<WebhookDeliveryResult> {
const payload = {
events: events.map((e) => ({
id: e.id,
type: e.entityType,
operation: e.operation,
entityId: e.entityId,
timestamp: e.timestamp,
nounType: e.nounType,
verbType: e.verbType,
service: e.service,
data: this.webhookConfig.includeData ? e.data : undefined
})),
deliveredAt: Date.now()
}
const body = JSON.stringify(payload)
const headers: Record<string, string> = {
'Content-Type': 'application/json',
'User-Agent': 'Brainy-Webhook/1.0',
'X-Brainy-Delivery': events[0]?.id ?? 'batch'
}
// Sign if secret is configured
if (webhook.secret) {
const signature = await this.sign(body, webhook.secret)
headers['X-Brainy-Signature'] = `sha256=${signature}`
}
try {
const controller = new AbortController()
const timeoutId = setTimeout(
() => controller.abort(),
this.webhookConfig.timeout
)
const response = await fetch(webhook.url, {
method: 'POST',
headers,
body,
signal: controller.signal
})
clearTimeout(timeoutId)
const success = response.ok
if (success) {
// Update last delivery time
const wh = this.webhooks.get(webhook.id)
if (wh) {
wh.lastDeliveryAt = Date.now()
wh.failureCount = 0
}
}
return {
webhookId: webhook.id,
eventId: events[0]?.id ?? 'batch',
success,
statusCode: response.status,
attempts: 1,
timestamp: Date.now()
}
} catch (error: any) {
return {
webhookId: webhook.id,
eventId: events[0]?.id ?? 'batch',
success: false,
error: error.message,
attempts: 1,
timestamp: Date.now()
}
}
}
private async sign(payload: string, secret: string): Promise<string> {
// Use Web Crypto API (works in all environments)
const encoder = new TextEncoder()
const keyData = encoder.encode(secret)
const messageData = encoder.encode(payload)
const key = await crypto.subtle.importKey(
'raw',
keyData,
{ name: 'HMAC', hash: 'SHA-256' },
false,
['sign']
)
const signature = await crypto.subtle.sign('HMAC', key, messageData)
const signatureArray = new Uint8Array(signature)
return Array.from(signatureArray)
.map((b) => b.toString(16).padStart(2, '0'))
.join('')
}
}
/**
* Package export for @soulcraft/brainy-webhooks
*/
export const integration = {
name: 'webhooks',
version: '1.0.0',
description: 'Push events to external URLs via webhooks',
environments: ['node', 'browser', 'deno', 'cloudflare', 'bun'],
create: (brain: any, config?: WebhookConfig) =>
new WebhookIntegration(config),
defaultConfig: {
timeout: 30000,
includeData: false
}
}

View file

@ -0,0 +1,12 @@
/**
* Webhook Integration Module
*
* Push events to external URLs via HTTP POST.
* Works in all environments.
*/
export {
WebhookIntegration,
integration,
type WebhookConfig
} from './WebhookIntegration.js'

View file

@ -622,6 +622,42 @@ export type AggregateMetric =
// ============= Configuration =============
/**
* Integration Hub configuration (v7.4.0)
*
* Enables external tool integrations: Excel, Power BI, Google Sheets, etc.
* Works in all environments with zero external dependencies.
*
* @example
* ```typescript
* // Enable all integrations with defaults
* new Brainy({ integrations: true })
*
* // Custom configuration
* new Brainy({
* integrations: {
* basePath: '/api/v1',
* enable: ['odata', 'sheets']
* }
* })
* ```
*/
export interface IntegrationsConfig {
/** Base path for all integration endpoints (default: '') */
basePath?: string
/** Which integrations to enable (default: all) */
enable?: ('odata' | 'sheets' | 'sse' | 'webhooks')[] | 'all'
/** Per-integration config overrides */
config?: {
odata?: { basePath?: string }
sheets?: { basePath?: string }
sse?: { basePath?: string; heartbeatInterval?: number }
webhooks?: { maxRetries?: number }
}
}
/**
* Brainy configuration
*/
@ -696,6 +732,13 @@ export interface BrainyConfig {
// Logging configuration
verbose?: boolean // Enable verbose logging
silent?: boolean // Suppress all logging output
// Integration Hub (v7.4.0)
// Enable external tool integrations: Excel, Power BI, Google Sheets, etc.
// - true: Enable all integrations with default paths
// - false/undefined: Disable integrations (default)
// - IntegrationsConfig: Custom configuration
integrations?: boolean | IntegrationsConfig
}
// ============= Neural API Types =============

View file

@ -0,0 +1,427 @@
/**
* Integration Hub - Core Tests
*
* Tests for EventBus, TabularExporter, and core functionality.
*/
import { describe, it, expect, beforeEach, vi } from 'vitest'
import { EventBus } from '../../src/integrations/core/EventBus.js'
import { TabularExporter } from '../../src/integrations/core/TabularExporter.js'
import { NounType, VerbType } from '../../src/types/graphTypes.js'
import type { Entity, Relation } from '../../src/types/brainy.types.js'
describe('EventBus', () => {
let eventBus: EventBus
beforeEach(() => {
eventBus = new EventBus()
})
describe('subscribe and emit', () => {
it('should emit events to subscribers', async () => {
const handler = vi.fn()
eventBus.subscribe({}, handler)
eventBus.emit({
entityType: 'noun',
operation: 'create',
entityId: 'test-123'
})
expect(handler).toHaveBeenCalledTimes(1)
expect(handler).toHaveBeenCalledWith(
expect.objectContaining({
entityType: 'noun',
operation: 'create',
entityId: 'test-123'
})
)
})
it('should filter by entity type', () => {
const handler = vi.fn()
eventBus.subscribe({ entityTypes: ['verb'] }, handler)
eventBus.emit({
entityType: 'noun',
operation: 'create',
entityId: 'test-123'
})
expect(handler).not.toHaveBeenCalled()
eventBus.emit({
entityType: 'verb',
operation: 'create',
entityId: 'test-456'
})
expect(handler).toHaveBeenCalledTimes(1)
})
it('should filter by operation', () => {
const handler = vi.fn()
eventBus.subscribe({ operations: ['update', 'delete'] }, handler)
eventBus.emit({
entityType: 'noun',
operation: 'create',
entityId: 'test-123'
})
expect(handler).not.toHaveBeenCalled()
eventBus.emit({
entityType: 'noun',
operation: 'update',
entityId: 'test-456'
})
expect(handler).toHaveBeenCalledTimes(1)
})
it('should filter by noun type', () => {
const handler = vi.fn()
eventBus.subscribe({ nounTypes: [NounType.Person] }, handler)
eventBus.emit({
entityType: 'noun',
operation: 'create',
entityId: 'test-123',
nounType: NounType.Document
})
expect(handler).not.toHaveBeenCalled()
eventBus.emit({
entityType: 'noun',
operation: 'create',
entityId: 'test-456',
nounType: NounType.Person
})
expect(handler).toHaveBeenCalledTimes(1)
})
})
describe('unsubscribe', () => {
it('should stop receiving events after unsubscribe', () => {
const handler = vi.fn()
const subscription = eventBus.subscribe({}, handler)
eventBus.emit({
entityType: 'noun',
operation: 'create',
entityId: 'test-123'
})
expect(handler).toHaveBeenCalledTimes(1)
subscription.unsubscribe()
eventBus.emit({
entityType: 'noun',
operation: 'create',
entityId: 'test-456'
})
expect(handler).toHaveBeenCalledTimes(1) // Still 1
})
})
describe('sequence IDs', () => {
it('should increment sequence IDs', () => {
const events: any[] = []
eventBus.subscribe({}, (e) => events.push(e))
eventBus.emit({ entityType: 'noun', operation: 'create', entityId: '1' })
eventBus.emit({ entityType: 'noun', operation: 'create', entityId: '2' })
eventBus.emit({ entityType: 'noun', operation: 'create', entityId: '3' })
expect(events[0].sequenceId).toBe(1n)
expect(events[1].sequenceId).toBe(2n)
expect(events[2].sequenceId).toBe(3n)
})
it('should return current sequence ID', () => {
expect(eventBus.getCurrentSequenceId()).toBe(0n)
eventBus.emit({ entityType: 'noun', operation: 'create', entityId: '1' })
expect(eventBus.getCurrentSequenceId()).toBe(1n)
eventBus.emit({ entityType: 'noun', operation: 'create', entityId: '2' })
expect(eventBus.getCurrentSequenceId()).toBe(2n)
})
})
describe('event buffer', () => {
it('should buffer events for replay', () => {
eventBus.emit({ entityType: 'noun', operation: 'create', entityId: '1' })
eventBus.emit({ entityType: 'noun', operation: 'create', entityId: '2' })
eventBus.emit({ entityType: 'noun', operation: 'create', entityId: '3' })
const events = eventBus.getEventsSince(1n)
expect(events).toHaveLength(2)
expect(events[0].entityId).toBe('2')
expect(events[1].entityId).toBe('3')
})
})
describe('helper methods', () => {
it('should emit noun events', () => {
const handler = vi.fn()
eventBus.subscribe({}, handler)
eventBus.emitNoun('create', 'entity-1', NounType.Person)
expect(handler).toHaveBeenCalledWith(
expect.objectContaining({
entityType: 'noun',
operation: 'create',
entityId: 'entity-1',
nounType: NounType.Person
})
)
})
it('should emit verb events', () => {
const handler = vi.fn()
eventBus.subscribe({}, handler)
eventBus.emitVerb('create', 'rel-1', VerbType.RelatedTo)
expect(handler).toHaveBeenCalledWith(
expect.objectContaining({
entityType: 'verb',
operation: 'create',
entityId: 'rel-1',
verbType: VerbType.RelatedTo
})
)
})
it('should emit VFS events', () => {
const handler = vi.fn()
eventBus.subscribe({}, handler)
eventBus.emitVFS('create', 'file-1')
expect(handler).toHaveBeenCalledWith(
expect.objectContaining({
entityType: 'vfs',
operation: 'create',
entityId: 'file-1'
})
)
})
})
})
describe('TabularExporter', () => {
let exporter: TabularExporter
const mockEntity: Entity = {
id: 'entity-123',
type: NounType.Person,
createdAt: 1704067200000, // 2024-01-01
updatedAt: 1704153600000, // 2024-01-02
vector: new Float32Array([0.1, 0.2, 0.3]),
confidence: 0.95,
weight: 0.8,
service: 'test-service',
data: { content: 'Hello world' },
metadata: {
name: 'John Doe',
email: 'john@example.com',
nested: { key: 'value' }
}
}
const mockRelation: Relation = {
id: 'rel-123',
from: 'entity-1',
to: 'entity-2',
type: VerbType.RelatedTo,
weight: 0.9,
confidence: 0.85,
createdAt: 1704067200000,
service: 'test-service',
metadata: { reason: 'test' }
}
beforeEach(() => {
exporter = new TabularExporter()
})
describe('entitiesToRows', () => {
it('should convert entity to row', () => {
const rows = exporter.entitiesToRows([mockEntity])
expect(rows).toHaveLength(1)
expect(rows[0].Id).toBe('entity-123')
expect(rows[0].Type).toBe('person')
expect(rows[0].Confidence).toBe(0.95)
expect(rows[0].Weight).toBe(0.8)
expect(rows[0].Service).toBe('test-service')
})
it('should flatten metadata by default', () => {
const rows = exporter.entitiesToRows([mockEntity])
expect(rows[0].Metadata_name).toBe('John Doe')
expect(rows[0].Metadata_email).toBe('john@example.com')
})
it('should JSON stringify nested metadata', () => {
const rows = exporter.entitiesToRows([mockEntity])
expect(rows[0].Metadata_nested).toBe('{"key":"value"}')
})
it('should JSON stringify data field', () => {
const rows = exporter.entitiesToRows([mockEntity])
expect(rows[0].Data).toBe('{"content":"Hello world"}')
})
it('should format dates as ISO 8601', () => {
const rows = exporter.entitiesToRows([mockEntity])
expect(rows[0].CreatedAt).toMatch(/^\d{4}-\d{2}-\d{2}T/)
})
})
describe('configuration', () => {
it('should not flatten metadata when disabled', () => {
exporter = new TabularExporter({ flattenMetadata: false })
const rows = exporter.entitiesToRows([mockEntity])
expect(rows[0].Metadata_name).toBeUndefined()
expect(rows[0].Metadata).toBeDefined()
})
it('should include vectors when enabled', () => {
exporter = new TabularExporter({ includeVectors: true })
const rows = exporter.entitiesToRows([mockEntity])
expect(rows[0].Vector).toBeDefined()
})
it('should use custom metadata prefix', () => {
exporter = new TabularExporter({ metadataPrefix: 'M_' })
const rows = exporter.entitiesToRows([mockEntity])
expect(rows[0].M_name).toBe('John Doe')
expect(rows[0].Metadata_name).toBeUndefined()
})
it('should format dates as unix timestamps', () => {
exporter = new TabularExporter({ dateFormat: 'unix' })
const rows = exporter.entitiesToRows([mockEntity])
expect(rows[0].CreatedAt).toBe('1704067200')
})
})
describe('relationsToRows', () => {
it('should convert relation to row', () => {
const rows = exporter.relationsToRows([mockRelation])
expect(rows).toHaveLength(1)
expect(rows[0].Id).toBe('rel-123')
expect(rows[0].FromId).toBe('entity-1')
expect(rows[0].ToId).toBe('entity-2')
expect(rows[0].Type).toBe('relatedTo')
expect(rows[0].Weight).toBe(0.9)
expect(rows[0].Confidence).toBe(0.85)
})
})
describe('toCSV', () => {
it('should generate valid CSV', () => {
const csv = exporter.toCSV([mockEntity])
const lines = csv.split('\n')
expect(lines.length).toBeGreaterThan(1)
expect(lines[0]).toContain('Id')
expect(lines[0]).toContain('Type')
expect(lines[1]).toContain('entity-123')
})
it('should escape special characters', () => {
const entityWithComma: Entity = {
...mockEntity,
metadata: { name: 'Doe, John' }
}
const csv = exporter.toCSV([entityWithComma])
expect(csv).toContain('"Doe, John"')
})
it('should escape quotes', () => {
const entityWithQuotes: Entity = {
...mockEntity,
metadata: { name: 'John "Jack" Doe' }
}
const csv = exporter.toCSV([entityWithQuotes])
expect(csv).toContain('""Jack""')
})
})
describe('parseCSV', () => {
it('should parse CSV back to entities', () => {
const csv = exporter.toCSV([mockEntity])
const entities = exporter.parseCSV(csv)
expect(entities).toHaveLength(1)
expect(entities[0].id).toBe('entity-123')
expect(entities[0].type).toBe('person')
})
})
describe('toOData', () => {
it('should generate OData response', () => {
const odata = exporter.toOData([mockEntity])
expect(odata).toHaveProperty('@odata.context')
expect(odata).toHaveProperty('value')
expect((odata as any).value).toHaveLength(1)
})
it('should include count when provided', () => {
const odata = exporter.toOData([mockEntity], { count: 100 })
expect((odata as any)['@odata.count']).toBe(100)
})
it('should include nextLink when provided', () => {
const odata = exporter.toOData([mockEntity], {
nextLink: '/odata/Entities?$skip=100'
})
expect((odata as any)['@odata.nextLink']).toBe('/odata/Entities?$skip=100')
})
})
describe('getSchema', () => {
it('should infer schema from entities', () => {
const schema = exporter.getSchema([mockEntity])
expect(schema).toContainEqual(
expect.objectContaining({ name: 'Id', type: 'string' })
)
expect(schema).toContainEqual(
expect.objectContaining({ name: 'Confidence', type: 'number' })
)
})
})
})

View file

@ -0,0 +1,343 @@
/**
* OData Integration Tests
*
* Tests for OData query parsing, EDMX generation, and filtering.
*/
import { describe, it, expect } from 'vitest'
import {
parseODataQuery,
parseFilter,
parseOrderBy,
parseSelect,
applyFilter,
applySelect,
applyOrderBy,
applyPagination
} from '../../src/integrations/odata/ODataQueryParser.js'
import {
generateEdmx,
generateMetadataJson,
generateServiceDocument
} from '../../src/integrations/odata/EdmxGenerator.js'
describe('OData Query Parser', () => {
describe('parseODataQuery', () => {
it('should parse $top', () => {
const options = parseODataQuery('$top=10')
expect(options.top).toBe(10)
})
it('should parse $skip', () => {
const options = parseODataQuery('$skip=20')
expect(options.skip).toBe(20)
})
it('should parse $select', () => {
const options = parseODataQuery('$select=Id,Type,Name')
expect(options.select).toEqual(['Id', 'Type', 'Name'])
})
it('should parse $orderby', () => {
const options = parseODataQuery('$orderby=CreatedAt desc')
expect(options.orderBy).toEqual([
{ field: 'CreatedAt', direction: 'desc' }
])
})
it('should parse $count', () => {
const options = parseODataQuery('$count=true')
expect(options.count).toBe(true)
})
it('should parse $filter', () => {
const options = parseODataQuery("$filter=Type eq 'person'")
expect(options.filter).toBe("Type eq 'person'")
})
it('should parse $search', () => {
const options = parseODataQuery('$search=machine learning')
expect(options.search).toBe('machine learning')
})
it('should parse multiple options', () => {
const options = parseODataQuery(
'$top=10&$skip=20&$orderby=Name asc&$count=true'
)
expect(options.top).toBe(10)
expect(options.skip).toBe(20)
expect(options.orderBy).toEqual([{ field: 'Name', direction: 'asc' }])
expect(options.count).toBe(true)
})
})
describe('parseFilter', () => {
it('should parse simple eq comparison', () => {
const filter = parseFilter("Type eq 'person'")
expect(filter).toEqual({
field: 'Type',
op: 'eq',
value: 'person'
})
})
it('should parse ne comparison', () => {
const filter = parseFilter("Status ne 'deleted'")
expect(filter).toEqual({
field: 'Status',
op: 'ne',
value: 'deleted'
})
})
it('should parse numeric comparisons', () => {
const gtFilter = parseFilter('Weight gt 0.5')
expect(gtFilter.op).toBe('gt')
expect(gtFilter.value).toBe(0.5)
const ltFilter = parseFilter('Count lt 100')
expect(ltFilter.op).toBe('lt')
expect(ltFilter.value).toBe(100)
})
it('should parse boolean values', () => {
const filter = parseFilter('Active eq true')
expect(filter.value).toBe(true)
})
it('should parse null values', () => {
const filter = parseFilter('DeletedAt eq null')
expect(filter.value).toBe(null)
})
it('should parse and expressions', () => {
const filter = parseFilter("Type eq 'person' and Status eq 'active'")
expect(filter).toEqual({
and: [
{ field: 'Type', op: 'eq', value: 'person' },
{ field: 'Status', op: 'eq', value: 'active' }
]
})
})
it('should parse or expressions', () => {
const filter = parseFilter("Type eq 'person' or Type eq 'organization'")
expect(filter).toEqual({
or: [
{ field: 'Type', op: 'eq', value: 'person' },
{ field: 'Type', op: 'eq', value: 'organization' }
]
})
})
it('should parse contains function', () => {
const filter = parseFilter("contains(Name, 'John')")
expect(filter).toEqual({
func: 'contains',
args: [{ field: 'Name' }, { value: 'John' }]
})
})
it('should parse startswith function', () => {
const filter = parseFilter("startswith(Email, 'admin')")
expect(filter).toEqual({
func: 'startswith',
args: [{ field: 'Email' }, { value: 'admin' }]
})
})
})
describe('parseOrderBy', () => {
it('should parse single field', () => {
const orderBy = parseOrderBy('Name')
expect(orderBy).toEqual([{ field: 'Name', direction: 'asc' }])
})
it('should parse with direction', () => {
const orderBy = parseOrderBy('CreatedAt desc')
expect(orderBy).toEqual([{ field: 'CreatedAt', direction: 'desc' }])
})
it('should parse multiple fields', () => {
const orderBy = parseOrderBy('Type asc, CreatedAt desc')
expect(orderBy).toEqual([
{ field: 'Type', direction: 'asc' },
{ field: 'CreatedAt', direction: 'desc' }
])
})
})
describe('parseSelect', () => {
it('should parse comma-separated fields', () => {
const select = parseSelect('Id, Type, Name')
expect(select).toEqual(['Id', 'Type', 'Name'])
})
})
describe('applyFilter', () => {
const testData = [
{ Id: '1', Type: 'person', Name: 'John Doe', Weight: 0.8 },
{ Id: '2', Type: 'person', Name: 'Jane Smith', Weight: 0.6 },
{ Id: '3', Type: 'organization', Name: 'Acme Corp', Weight: 0.9 },
{ Id: '4', Type: 'document', Name: 'Report', Weight: 0.5 }
]
it('should filter by eq', () => {
const result = applyFilter(testData, "Type eq 'person'")
expect(result).toHaveLength(2)
expect(result.every((r) => r.Type === 'person')).toBe(true)
})
it('should filter by gt', () => {
const result = applyFilter(testData, 'Weight gt 0.7')
expect(result).toHaveLength(2)
expect(result.every((r) => r.Weight > 0.7)).toBe(true)
})
it('should filter with contains', () => {
const result = applyFilter(testData, "contains(Name, 'Doe')")
expect(result).toHaveLength(1)
expect(result[0].Name).toBe('John Doe')
})
it('should filter with and', () => {
const result = applyFilter(
testData,
"Type eq 'person' and Weight gt 0.7"
)
expect(result).toHaveLength(1)
expect(result[0].Name).toBe('John Doe')
})
})
describe('applySelect', () => {
const testData = [
{ Id: '1', Type: 'person', Name: 'John', Extra: 'data' }
]
it('should select specified fields only', () => {
const result = applySelect(testData, ['Id', 'Name'])
expect(result[0]).toHaveProperty('Id')
expect(result[0]).toHaveProperty('Name')
expect(result[0]).not.toHaveProperty('Type')
expect(result[0]).not.toHaveProperty('Extra')
})
it('should return all fields if select is empty', () => {
const result = applySelect(testData, [])
expect(result[0]).toHaveProperty('Extra')
})
})
describe('applyOrderBy', () => {
const testData = [
{ Name: 'Charlie', Weight: 0.5 },
{ Name: 'Alice', Weight: 0.9 },
{ Name: 'Bob', Weight: 0.7 }
]
it('should sort ascending by default', () => {
const result = applyOrderBy(testData, [
{ field: 'Name', direction: 'asc' }
])
expect(result[0].Name).toBe('Alice')
expect(result[1].Name).toBe('Bob')
expect(result[2].Name).toBe('Charlie')
})
it('should sort descending', () => {
const result = applyOrderBy(testData, [
{ field: 'Weight', direction: 'desc' }
])
expect(result[0].Weight).toBe(0.9)
expect(result[2].Weight).toBe(0.5)
})
})
describe('applyPagination', () => {
const testData = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
it('should apply top (limit)', () => {
const result = applyPagination(testData, 3)
expect(result).toEqual([1, 2, 3])
})
it('should apply skip (offset)', () => {
const result = applyPagination(testData, undefined, 5)
expect(result).toEqual([6, 7, 8, 9, 10])
})
it('should apply both top and skip', () => {
const result = applyPagination(testData, 3, 2)
expect(result).toEqual([3, 4, 5])
})
})
})
describe('EDMX Generator', () => {
describe('generateEdmx', () => {
it('should generate valid XML', () => {
const xml = generateEdmx()
expect(xml).toContain('<?xml version="1.0"')
expect(xml).toContain('<edmx:Edmx')
expect(xml).toContain('EntityType Name="Entity"')
})
it('should include entity properties', () => {
const xml = generateEdmx()
expect(xml).toContain('Property Name="Id"')
expect(xml).toContain('Property Name="Type"')
expect(xml).toContain('Property Name="CreatedAt"')
})
it('should include NounType enum', () => {
const xml = generateEdmx()
expect(xml).toContain('EnumType Name="NounType"')
expect(xml).toContain('Member Name="person"')
})
it('should include relationships when enabled', () => {
const xml = generateEdmx({ includeRelationships: true })
expect(xml).toContain('EntityType Name="Relationship"')
expect(xml).toContain('EnumType Name="VerbType"')
})
it('should exclude relationships when disabled', () => {
const xml = generateEdmx({ includeRelationships: false })
expect(xml).not.toContain('EntityType Name="Relationship"')
})
it('should use custom namespace', () => {
const xml = generateEdmx({ namespace: 'MyApp' })
expect(xml).toContain('Namespace="MyApp"')
})
})
describe('generateMetadataJson', () => {
it('should generate JSON metadata', () => {
const metadata = generateMetadataJson()
expect(metadata).toHaveProperty('$Version', '4.0')
expect(metadata).toHaveProperty('Brainy')
})
})
describe('generateServiceDocument', () => {
it('should generate service document', () => {
const doc = generateServiceDocument('http://localhost/odata')
expect(doc).toHaveProperty('@odata.context')
expect(doc).toHaveProperty('value')
expect((doc as any).value).toContainEqual(
expect.objectContaining({ name: 'Entities' })
)
})
it('should include relationships when enabled', () => {
const doc = generateServiceDocument('http://localhost/odata', {
includeRelationships: true
})
expect((doc as any).value).toContainEqual(
expect.objectContaining({ name: 'Relationships' })
)
})
})
})