335 releases 523 tags

RSS feed

• v4.9.0 db23836b32

  Compare

  v4.9.0 - Universal Relationship Extraction Stable

  dpsifr released this 2025-10-29 00:28:18 +01:00 | 663 commits to main since this release

  🎯 Summary

  This release implements Universal Relationship Extraction with full provenance tracking, transforming Brainy imports from simple entity extractors into true knowledge graph builders.

  Key Problem Solved

  Before v4.9.0:

  • Imports created entities but minimal semantic relationships
  • No document entity representing the source file
  • No way to trace entities back to their source
  • Generic "related_to" relationships with no semantic diversity
  • VFS structural relationships mixed with knowledge graph relationships

  After v4.9.0:

  • ✅ Document entity created for every import
  • ✅ Full provenance tracking (document → entity relationships)
  • ✅ Enhanced column detection (7 relationship types vs 1)
  • ✅ Type-based relationship inference (smarter relationship types)
  • ✅ Clear separation of VFS/semantic/provenance relationships

  ---

  📋 Changes Implemented

  1. Document Entity Creation (HIGH IMPACT)

  Creates a `NounType.Document` entity for every imported file with metadata tracking total entities, breakdown by type, and import timestamp.

  2. Provenance Relationships (HIGH IMPACT)

  Creates `document → entity` relationship for EVERY imported entity, enabling:

  • "Show source" functionality
  • Full data lineage tracking
  • Bidirectional navigation (entity ↔ document)

  3. Relationship Type Metadata (CRITICAL)

  All relationships now marked with `relationshipType`:

  • `'provenance'` - Document → entity relationships
  • `'semantic'` - Knowledge graph relationships
  • `'vfs'` - Virtual file system structure

  4. Enhanced Column Detection (MEDIUM IMPACT)

  Expanded from 1 to 7 relationship column patterns:

  • Location, Home, Lives In → `located_at`
  • Owner, Owned By, Wielder → `part_of`
  • Created By, Made By, Author → `created_by`
  • Uses, Utilizes, Weapon → `uses`
  • Member Of, Part Of, Group → `part_of`
  • Knows, Friend, Ally → `friend_of`
  • Connection, Link, See Also → `related_to`

  5. Type-Based Inference (MEDIUM IMPACT)

  Automatically infers better relationship types based on:

  • Context analysis (keywords in relationship descriptions)
  • Entity type patterns (Person + Location → `located_at`, etc.)
  • 10+ type-based inference rules

  ---

  📊 Expected Impact

  For a typical 1,149-row import:

  Before v4.9.0:

  • Entities: 1,149
  • Relationships: 581 (ALL "contains" - VFS only)
  • Document Entity: ❌ None
  • Provenance: ❌ None

  After v4.9.0:

  • Entities: 1,150 (1,149 + 1 document)
  • Relationships: ~3,900 total
    • 1,149 provenance (document → entity)
    • ~1,500 semantic (entity ↔ entity, diverse types)
    • 581 VFS (directory structure, marked separately)

  ---

  🚀 Upgrade Instructions

  1. Upgrade Brainy:
     ```bash
     npm install @soulcraft/brainy@4.9.0
     ```

  2. Re-import your files:
     ```bash
     brainy import "your-file.xlsx"
     ```

  3. Query your knowledge graph:
     ```javascript
     // Get all semantic relationships (filter out VFS)
     const semanticRels = await brain.getRelations({
     metadata: { relationshipType: 'semantic' }
     })

     // Get provenance (source documents)
     const documents = await brain.find({ type: 'document' })
     ```

  ---

  🔧 Technical Details

  Files Modified:

  • src/import/ImportCoordinator.ts (+175 lines)
  • src/importers/SmartExcelImporter.ts (+65 lines)
  • src/vfs/VirtualFileSystem.ts (+2 lines)

  Backward Compatibility:

  ✅ 100% backward compatible - all new features work automatically

  Universal Support:

  ✅ Works for ALL 7 import formats: Excel, PDF, CSV, JSON, Markdown, YAML, DOCX

  ---

  Release Date: 2025-10-28
  Type: Minor Version (New Features)
  Breaking Changes: No
  Migration Required: No

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    1 download