feat: add universal adapters for browser compatibility
- Create universal adapters for cross-platform support (browser/Node/serverless) - Replace Node.js-specific imports with universal implementations - Add OPFS support for browser persistent storage - Maintain same BrainyData interface across all environments - Enable real Brainy usage in browser console UI - Keep package size optimized (no bloat) Universal adapters in /src/universal/: - uuid.ts: Cross-platform UUID generation - crypto.ts: Browser/Node crypto operations - fs.ts: OPFS/FileSystem/Memory storage adapter - path.ts: Universal path operations - events.ts: EventEmitter compatibility layer This enables 'write once, run anywhere' for Brainy while maintaining the exact same API. No breaking changes to existing code.
This commit is contained in:
parent
6e8869ecb7
commit
93804be354
31 changed files with 2120 additions and 2028 deletions
|
|
@ -1,263 +0,0 @@
|
|||
# 🔧 Notion Connector - Quantum Vault Implementation
|
||||
|
||||
**⚠️ This is a preview of what exists in `brainy-quantum-vault` (private repository)**
|
||||
|
||||
*Full implementation available to premium license holders only*
|
||||
|
||||
## 🧠 **Implementation Overview**
|
||||
|
||||
The Notion connector in the Quantum Vault provides seamless sync between Notion workspaces and your Brainy vector + graph database.
|
||||
|
||||
### **File Structure (in brainy-quantum-vault):**
|
||||
```
|
||||
brainy-quantum-vault/src/connectors/notion/
|
||||
├── index.ts # Main NotionConnector class
|
||||
├── auth/
|
||||
│ ├── oauth.ts # OAuth 2.0 flow implementation
|
||||
│ └── tokens.ts # Token management and refresh
|
||||
├── sync/
|
||||
│ ├── pages.ts # Page content extraction
|
||||
│ ├── databases.ts # Database schema and records
|
||||
│ └── blocks.ts # Block-level content parsing
|
||||
├── mapping/
|
||||
│ ├── schema.ts # Notion → Brainy schema mapping
|
||||
│ ├── entities.ts # Entity extraction (people, dates, etc.)
|
||||
│ └── relationships.ts # Relationship detection
|
||||
├── utils/
|
||||
│ ├── rate-limiter.ts # Notion API rate limiting
|
||||
│ ├── retry.ts # Exponential backoff retry logic
|
||||
│ └── validation.ts # Data validation and sanitization
|
||||
├── types/
|
||||
│ ├── notion.ts # Notion API type definitions
|
||||
│ └── brainy.ts # Brainy-specific types
|
||||
└── tests/
|
||||
├── integration.test.ts
|
||||
└── unit.test.ts
|
||||
```
|
||||
|
||||
## 🚀 **Key Features**
|
||||
|
||||
### **🔄 Intelligent Sync**
|
||||
```typescript
|
||||
// Real implementation (Quantum Vault only)
|
||||
export class NotionConnector implements IConnector {
|
||||
readonly id = 'notion'
|
||||
readonly name = 'Notion Workspace Sync'
|
||||
readonly version = '1.2.3'
|
||||
readonly supportedTypes = ['pages', 'databases', 'blocks', 'users']
|
||||
|
||||
private client: Client
|
||||
private brainy: BrainyData
|
||||
private rateLimiter: RateLimiter
|
||||
private licenseValidator: LicenseValidator
|
||||
|
||||
async initialize(config: ConnectorConfig): Promise<void> {
|
||||
// 1. Validate premium license with quantum vault servers
|
||||
await this.licenseValidator.validate(config.licenseKey)
|
||||
|
||||
// 2. Initialize Notion API client with credentials
|
||||
this.client = new Client({
|
||||
auth: config.credentials.accessToken,
|
||||
// Custom retry logic for production reliability
|
||||
retry: this.createRetryConfig()
|
||||
})
|
||||
|
||||
// 3. Set up intelligent rate limiting (3 requests/second)
|
||||
this.rateLimiter = new RateLimiter({
|
||||
requestsPerSecond: 3,
|
||||
burstAllowance: 10
|
||||
})
|
||||
|
||||
// 4. Test connection and validate permissions
|
||||
await this.testConnection()
|
||||
}
|
||||
|
||||
async startSync(): Promise<SyncResult> {
|
||||
const startTime = Date.now()
|
||||
let synced = 0, failed = 0, skipped = 0
|
||||
const errors: any[] = []
|
||||
|
||||
try {
|
||||
// Phase 1: Sync workspace users and permissions
|
||||
const users = await this.syncUsers()
|
||||
synced += users.synced
|
||||
failed += users.failed
|
||||
|
||||
// Phase 2: Sync database schemas
|
||||
const databases = await this.syncDatabases()
|
||||
synced += databases.synced
|
||||
failed += databases.failed
|
||||
|
||||
// Phase 3: Sync pages with intelligent chunking
|
||||
const pages = await this.syncPages()
|
||||
synced += pages.synced
|
||||
failed += pages.failed
|
||||
|
||||
// Phase 4: Extract relationships using AI
|
||||
await this.extractRelationships()
|
||||
|
||||
return {
|
||||
synced,
|
||||
failed,
|
||||
skipped,
|
||||
duration: Date.now() - startTime,
|
||||
timestamp: new Date().toISOString(),
|
||||
errors,
|
||||
metadata: {
|
||||
lastSyncId: this.generateSyncId(),
|
||||
hasMore: false
|
||||
}
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
// Advanced error handling and retry logic
|
||||
throw new ConnectorError('Notion sync failed', error)
|
||||
}
|
||||
}
|
||||
|
||||
private async syncPages(): Promise<SyncResult> {
|
||||
const pages = await this.client.search({
|
||||
filter: { object: 'page' },
|
||||
sort: { timestamp: 'last_edited_time', direction: 'descending' }
|
||||
})
|
||||
|
||||
let synced = 0
|
||||
for (const page of pages.results) {
|
||||
await this.rateLimiter.wait() // Respect rate limits
|
||||
|
||||
try {
|
||||
// Extract page content with block-level parsing
|
||||
const content = await this.extractPageContent(page)
|
||||
|
||||
// AI-powered entity extraction
|
||||
const entities = await this.extractEntities(content)
|
||||
|
||||
// Store in Brainy with rich metadata
|
||||
const brainyId = await this.brainy.add(content.text, {
|
||||
source: 'notion',
|
||||
type: 'page',
|
||||
notionId: page.id,
|
||||
title: content.title,
|
||||
url: content.url,
|
||||
lastModified: page.last_edited_time,
|
||||
entities,
|
||||
// Rich metadata for filtering
|
||||
workspace: content.workspace,
|
||||
database: content.parent_database,
|
||||
tags: content.tags
|
||||
})
|
||||
|
||||
// Create relationships
|
||||
await this.createRelationships(brainyId, entities, content)
|
||||
|
||||
synced++
|
||||
} catch (error) {
|
||||
// Log error but continue processing
|
||||
console.error(`Failed to sync page ${page.id}:`, error)
|
||||
// Would implement sophisticated error tracking
|
||||
}
|
||||
}
|
||||
|
||||
return { synced, failed: 0, skipped: 0, duration: 0, timestamp: '' }
|
||||
}
|
||||
|
||||
private async extractEntities(content: any): Promise<any[]> {
|
||||
// AI-powered entity extraction using Brainy's neural capabilities
|
||||
// This would use the Neural Import system we built!
|
||||
|
||||
// Extract @mentions as person entities
|
||||
const mentions = content.text.match(/@([^\\s]+)/g) || []
|
||||
const personEntities = mentions.map(mention => ({
|
||||
type: 'person',
|
||||
value: mention.substring(1),
|
||||
source: 'mention'
|
||||
}))
|
||||
|
||||
// Extract dates, URLs, etc.
|
||||
const dateMatches = content.text.match(/\\d{4}-\\d{2}-\\d{2}/g) || []
|
||||
const dateEntities = dateMatches.map(date => ({
|
||||
type: 'date',
|
||||
value: date,
|
||||
source: 'text_extraction'
|
||||
}))
|
||||
|
||||
return [...personEntities, ...dateEntities]
|
||||
}
|
||||
|
||||
private async createRelationships(brainyId: string, entities: any[], content: any): Promise<void> {
|
||||
// Create "author" relationships
|
||||
if (content.created_by) {
|
||||
const authorId = await this.findOrCreateUser(content.created_by)
|
||||
await this.brainy.relate(authorId, brainyId, 'created')
|
||||
}
|
||||
|
||||
// Create "mentions" relationships
|
||||
for (const entity of entities) {
|
||||
if (entity.type === 'person') {
|
||||
const personId = await this.findOrCreatePerson(entity.value)
|
||||
await this.brainy.relate(brainyId, personId, 'mentions')
|
||||
}
|
||||
}
|
||||
|
||||
// Database relationships
|
||||
if (content.parent_database) {
|
||||
const dbId = await this.findOrCreateDatabase(content.parent_database)
|
||||
await this.brainy.relate(dbId, brainyId, 'contains')
|
||||
}
|
||||
}
|
||||
|
||||
// ... many more sophisticated methods for handling:
|
||||
// - OAuth token refresh
|
||||
// - Incremental sync with change detection
|
||||
// - Error recovery and retry logic
|
||||
// - Database schema mapping
|
||||
// - Block-level content extraction
|
||||
// - Webhook integration for real-time updates
|
||||
// - Enterprise permission handling
|
||||
}
|
||||
```
|
||||
|
||||
## 🔒 **Premium Features**
|
||||
|
||||
### **🧠 AI-Powered Intelligence**
|
||||
- **Entity Recognition**: Automatically detects people, companies, dates, locations
|
||||
- **Relationship Mapping**: Understands mentions, references, hierarchies
|
||||
- **Content Understanding**: Semantic analysis of page content
|
||||
|
||||
### **⚡ Production Reliability**
|
||||
- **Rate Limit Management**: Intelligent request throttling
|
||||
- **Error Recovery**: Exponential backoff with retry logic
|
||||
- **Incremental Sync**: Only sync changed content
|
||||
- **Webhook Integration**: Real-time updates from Notion
|
||||
|
||||
### **🔐 Enterprise Security**
|
||||
- **OAuth 2.0 Flow**: Secure authentication
|
||||
- **Token Management**: Automatic refresh handling
|
||||
- **Permission Mapping**: Respects Notion workspace permissions
|
||||
- **Audit Logging**: Complete operation tracking
|
||||
|
||||
## 📊 **Usage Statistics**
|
||||
|
||||
Premium license holders report:
|
||||
- **⚡ 10x faster** than building custom integrations
|
||||
- **🎯 95% sync accuracy** with AI-powered entity detection
|
||||
- **🔄 Real-time updates** with webhook integration
|
||||
- **📈 Enterprise scale** handling 100K+ pages
|
||||
|
||||
## 🎯 **Get Quantum Vault Access**
|
||||
|
||||
Ready to unlock the full Notion connector?
|
||||
|
||||
```bash
|
||||
# Start your free trial
|
||||
cortex license trial notion-connector
|
||||
|
||||
# After activation, install from private registry
|
||||
npm install @soulcraft/brainy-quantum-vault
|
||||
```
|
||||
|
||||
**[Start Free Trial →](https://soulcraft-research.com/brainy/trial)**
|
||||
|
||||
---
|
||||
|
||||
*The complete implementation awaits in the Quantum Vault...* 🔒⚛️✨
|
||||
Loading…
Add table
Add a link
Reference in a new issue