soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

335 releases 524 tags

RSS feed
  • v4.2.0 28642f6f9c

    v4.2.0 Stable

    dpsifr released this 2025-10-23 02:38:43 +02:00 | 738 commits to main since this release

    Full Changelog: https://github.com/soulcraftlabs/brainy/compare/v4.1.4...v4.2.0

    Downloads
  • v4.1.4 cf35ce5044

    v4.1.4 Stable

    dpsifr released this 2025-10-22 00:30:45 +02:00 | 740 commits to main since this release

    Full Changelog: https://github.com/soulcraftlabs/brainy/compare/v4.1.3...v4.1.4

    Downloads
  • v4.1.3 1001af9a34

    v4.1.3 Stable

    dpsifr released this 2025-10-21 22:40:10 +02:00 | 742 commits to main since this release

    Full Changelog: https://github.com/soulcraftlabs/brainy/compare/v4.1.2...v4.1.3

    Downloads
  • v4.1.2 0a9d7ffa65

    v4.1.2 Stable

    dpsifr released this 2025-10-21 20:31:03 +02:00 | 745 commits to main since this release

    🐛 Bug Fixes

    Critical: Count Synchronization Race Condition

    Fixed critical bug affecting all storage adapters where entity and relationship counts were not tracked correctly during add(), relate(), and import() operations.

    Root Cause: Race condition where count increment code tried to read metadata before it was saved to storage.

    Impact: This bug affected all storage backends (FileSystem, GCS, R2, Azure, Memory, OPFS, S3, TypeAware) in brainy v4.0.0+.

    What Was Fixed

    • ✅ Centralized count tracking in baseStorage.ts (fixes ALL adapters automatically)
    • ✅ Added verb type to VerbMetadata for proper count tracking
    • ✅ Refactored verb count methods to prevent mutex deadlocks
    • ✅ Removed broken count increment code from 6 storage adapters
    • ✅ Added rebuildCounts() utility to repair corrupted counts from actual storage data
    • ✅ Added comprehensive integration tests (11 tests covering all operations)

    Files Modified

    Core Fixes:

    • src/storage/baseStorage.ts - Centralized noun/verb count tracking
    • src/storage/adapters/baseStorageAdapter.ts - Fixed verb count deadlocks
    • src/brainy.ts - Added verb type to metadata

    Storage Adapter Cleanup:

    • src/storage/adapters/fileSystemStorage.ts
    • src/storage/adapters/gcsStorage.ts
    • src/storage/adapters/r2Storage.ts
    • src/storage/adapters/azureBlobStorage.ts
    • src/storage/adapters/memoryStorage.ts

    New Utilities:

    • src/utils/rebuildCounts.ts - Production utility to rebuild counts.json
    • tests/integration/count-synchronization.test.ts - Integration tests

    Usage

    For existing databases with corrupted counts, use the rebuild utility:

    import { rebuildCounts } from '@soulcraft/brainy/utils/rebuildCounts'

const brain = new Brainy({ 
  storage: { type: 'filesystem', path: './brainy-data' } 
})
await brain.init()

const result = await rebuildCounts(brain.storage)
console.log(`Rebuilt: ${result.nounCount} entities, ${result.verbCount} relationships`)

    Full Changelog: https://github.com/soulcraftlabs/brainy/compare/v4.1.1...v4.1.2

    Downloads
  • v4.1.1 e5c56ed285

    v4.1.1 Stable

    dpsifr released this 2025-10-20 20:43:38 +02:00 | 747 commits to main since this release

    Full Changelog: https://github.com/soulcraftlabs/brainy/compare/v4.1.0...v4.1.1

    Downloads
  • v4.1.0 fcf710c398

    v4.1.0 Stable

    dpsifr released this 2025-10-20 20:19:13 +02:00 | 749 commits to main since this release

    Full Changelog: https://github.com/soulcraftlabs/brainy/compare/v4.0.2...v4.1.0

    Downloads
  • v4.0.1 69dbd5973c

    v4.0.1 - Documentation Fixes Stable

    dpsifr released this 2025-10-18 00:10:55 +02:00 | 753 commits to main since this release

    Full Changelog: https://github.com/soulcraftlabs/brainy/compare/v4.0.0...v4.0.1

    Downloads
  • v4.0.0 dbce385683

    v4.0.0 - Enterprise-Scale Cost Optimization & Performance Stable

    dpsifr released this 2025-10-17 23:49:41 +02:00 | 755 commits to main since this release

    🎉 Major Release - Cost Optimization & Enterprise Features

    v4.0.0 focuses on production cost optimization and enterprise-scale features

    ✨ Highlights

    💰 Up to 96% Storage Cost Savings

    • Cloud storage lifecycle management (GCS Autoclass, AWS Intelligent-Tiering, Azure)
    • Automatic tier transitions and archival
    • Cost Impact: $138,000/year → $5,940/year @ 500TB scale

    ⚡ Performance at Billion-Scale

    • 1000x faster batch deletions (533 entities/sec vs 0.5/sec)
    • 60-80% FileSystem compression with gzip
    • OPFS quota monitoring for browser storage

    🎨 Enhanced CLI System (47 Commands)

    • 9 storage management commands
    • 2 enhanced import commands (files, directories, URLs)
    • Interactive modes for all operations
    • Beautiful terminal UI

    🛡️ Zero Breaking Changes

    100% backward compatible. No migration required. All features are opt-in.

    📖 Documentation

    See CHANGELOG.md for complete details.

    Downloads
  • v3.50.2 0f57ee6bb4

    v3.50.2 - Emergency Hotfix: Complete Metadata Explosion Fix Stable

    dpsifr released this 2025-10-17 01:31:10 +02:00 | 759 commits to main since this release

    🚨 Emergency Hotfix for v3.50.1

    Critical fix for incomplete v3.50.1 release - numeric field names still being indexed

    Problem

    v3.50.1 prevented vector fields by name ('vector', 'embedding') but missed vectors stored as objects with numeric keys.

    Studio Team Diagnostics Showed:

    • 212,531 chunk files with NUMERIC field names still being created
    • Files like: __chunk__54716_0.json with "field": "54716"
    • 424,837 total files (expected ~1,200)
    • Root cause: Vectors stored as objects {0: 0.1, 1: 0.2, ...} bypassed v3.50.1's field name check

    Solution

    Added regex check in extractIndexableFields():

    if (/^\d+$/.test(key)) continue  // Skip purely numeric field names

    This catches:

    • Array indices as object keys: "0", "1", "2", "100"
    • High-dimensional indices: "54716", "100000", "100001"
    • Works with v3.50.1's semantic checks: 'vector', 'embedding', 'embeddings'

    Impact

    • ✅ File reduction: 424,837 → ~1,200 files (354x reduction)
    • ✅ Prevents 212K+ chunk files from being created
    • ✅ Fixes server hangs during initialization
    • ✅ Completes the fix started in v3.50.1

    Test Results

    • ✅ Added new test: "should NOT index objects with numeric keys (v3.50.2 fix)"
    • ✅ 8/8 integration tests passing
    • ✅ Verifies NO chunk files have numeric field names

    Migration for Studio Team

    After upgrading to v3.50.2:

    1. Delete corrupted chunk files:

      cd brainy-data
rm -rf _system/

    2. Restart server - metadata index will rebuild correctly

    3. Verify file count:

      find brainy-data -type f | wc -l
# Should be ~1,200 (was 424,837)

    4. Verify no numeric field names:

      find _system -name '__chunk__*' -exec grep -l '"field":"[0-9]\+' {} \; | wc -l
# Should be 0 (was 212,531)

    Files Modified

    • src/utils/metadataIndex.ts (line 1106) - Added numeric field name exclusion
    • tests/integration/metadata-vector-exclusion.test.ts - Added v3.50.2 test case

    Install

    npm install @soulcraft/brainy@3.50.2

    Thank you to the Soulcraft Studio team for the detailed diagnostics that made this fix possible!

    Downloads
  • v3.50.1 52d1602907

    v3.50.1 - Critical Metadata Explosion Fix Stable

    dpsifr released this 2025-10-17 01:13:00 +02:00 | 762 commits to main since this release

    Critical Bug Fix: Metadata Explosion

    Fixes the metadata explosion bug reported by Soulcraft Studio team where 69,429 chunk files were created for only 1,143 entities.

    Impact

    • ✅ File reduction: 69,429 → ~1,200 files (58x reduction)
    • ✅ Storage reduction: 3.3GB → ~10MB metadata (330x reduction)
    • ✅ Fixes server initialization hangs
    • ✅ Fixes VFS getDescendants() hanging
    • ✅ Fixes Graph View UI not loading

    Root Cause

    Vector embeddings (384-dimensional arrays) were being indexed in metadata, creating a separate chunk file for each dimension.

    Solution

    • Added NEVER_INDEX Set excluding vector fields
    • Added safety check for large arrays (>10 elements)
    • Preserves small array indexing (tags, categories)

    Test Results

    • 7/7 integration tests passing
    • Verified: 6 chunk files for 10 entities (was 7,210 before fix)
    • 611/622 unit tests passing

    For Soulcraft Studio Team

    To apply the fix to your existing data:

    1. Upgrade to v3.50.1: npm install @soulcraft/brainy@3.50.1
    2. Delete corrupted metadata: rm -rf brainy-data/_system/
    3. Restart your app - metadata will rebuild automatically

    See full CHANGELOG for details.

    Full Changelog: https://github.com/soulcraftlabs/brainy/compare/v3.50.0...v3.50.1

    Downloads