brainy/CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file. See standard-version for commit guidelines.

[1.0.0-rc.1] - 2025-01-14

🎉 MAJOR RELEASE CANDIDATE - THE GREAT CLEANUP

This release represents a complete architectural consolidation and API unification. We've eliminated duplicate code, standardized all operations, and enhanced security while maintaining all existing functionality and performance optimizations.

✨ NEW UNIFIED API - ONE Way to Do Everything

Added

• 🧠 7 Core Methods - Consolidated from 40+ scattered methods to 7 unified operations:

  1. add() - Smart data addition (auto/guided/explicit/literal modes)
  2. search() - Triple-power search (vector + graph + facets)
  3. import() - Neural import with semantic type detection
  4. addNoun() - Explicit noun creation with strongly-typed NounType
  5. addVerb() - Relationship creation between nouns
  6. update() - Smart updates with automatic index synchronization
  7. delete() - Smart delete with soft delete default

• 🔐 Universal Encryption System

  • encryptData() / decryptData() - Universal crypto utilities
  • setConfig(key, value, { encrypt: true }) - Encrypted configuration storage
  • brainy add --encrypt - Per-item encryption via CLI
  • brainy init --encryption - Master encryption setup
  • Works in all environments: Browser, Node.js, Serverless

• 🐳 Container-Ready Deployment

  • BrainyData.preloadModel() - Download models during container build
  • BrainyData.warmup() - Production-optimized initialization
  • Local-files-only mode for offline containers
  • GPU acceleration (WebGPU/CUDA) with CPU fallback

🔄 CLI TRANSFORMATION

Changed

• Simplified from 40+ commands to 9 clean commands:
  • brainy init - Initialize with encryption/storage options
  • brainy add - Smart data addition (replaces addSmart, literal, neural modes)
  • brainy search - Unified search with filters
  • brainy update - Update existing data/metadata
  • brainy delete - Smart delete (soft delete by default)
  • brainy chat - AI conversations with your data
  • brainy import - Bulk data import
  • brainy status - Comprehensive system status
  • brainy config - Configuration management

🏗️ ARCHITECTURE CONSOLIDATION

Removed

• ❌ Duplicate pipeline implementations - 3 different Cortex classes → 1 unified
• ❌ addSmart() method - Functionality merged into add() with smart defaults
• ❌ Legacy CLI commands - 40+ commands → 9 focused commands
• ❌ cortex-legacy.ts - Replaced with clean unified implementation

Enhanced

• 📦 Package Size Reduced 16% - From 2.52MB to 2.1MB despite major feature additions
• 🔄 Soft Delete by Default - Preserves indexes, no reindexing needed
• ⚡ All Scaling Optimizations Preserved - 3-tier LRU cache, realtime streaming, memory modes

⚠️ BREAKING CHANGES

• CLI Commands - Most existing CLI commands renamed/consolidated
• addSmart() method - Removed, use add() with smart defaults
• Pipeline Classes - Multiple pipeline types consolidated into one

🚀 MIGRATION FROM 0.x

// OLD (0.x)
await brainy.addSmart(data, metadata)
await brainy.searchSimilar(query, 10)

// NEW (1.0)  
await brainy.add(data, metadata) // Smart by default!
await brainy.search(query, 10)   // Same power, cleaner API

# OLD (0.x)
brainy add-smart "data"
brainy search-similar "query"

# NEW (1.0)
brainy add "data"      # Smart by default!
brainy search "query"  # Same power, simpler

💬 FEEDBACK & SUPPORT

We'd love your feedback on this release candidate! Please:

• 🐛 Report bugs via GitHub Issues
• 💬 Share feedback via GitHub Discussions

Target Timeline: 2-4 weeks of testing, then 1.0.0 final release.

0.57.0 (2025-08-08)

⚠ BREAKING CHANGES

• CLI command renamed from cortex to brainy
• Neural Import renamed to Cortex augmentation

Changed

• CLI: Renamed from cortex to brainy for better package alignment

  • Now use brainy chat instead of cortex chat
  • npx @soulcraft/brainy now works automatically
  • Better alignment with package name

• Cortex Augmentation: Renamed from Neural Import

  • Better conceptual clarity: Cortex = AI intelligence layer
  • Class renamed: CortexSenseAugmentation (was NeuralImportSenseAugmentation)
  • Augmentation name: cortex-sense (was neural-import-sense)
  • The cortex is where thinking happens - perfect metaphor for AI processing

Migration Guide

CLI Commands

# Old
cortex chat "What's in my data?"
cortex neural import data.csv

# New  
brainy chat "What's in my data?"
brainy import data.csv --cortex

Code Changes

// Old
import { NeuralImportSenseAugmentation } from '@soulcraft/brainy'

// New
import { CortexSenseAugmentation } from '@soulcraft/brainy'

Why These Changes?

1. CLI Alignment: brainy command matches the package name @soulcraft/brainy
2. Better Metaphor: Cortex (brain's processing layer) better represents AI intelligence than generic "neural"
3. Clearer Architecture: CLI = brainy, AI = Cortex, Database = BrainyData

0.56.0 (2025-08-08)

Added

• Cortex CLI: Complete command center for Brainy database management

  • Interactive configuration wizard with atomic age aesthetics
  • Import/export system supporting CSV, JSON, and YAML formats
  • Backup and restore with compression (tar.gz)
  • Neural Import augmentation for AI-powered data understanding
  • Performance monitoring and health dashboard
  • Cloudflare R2 storage configuration support
  • Premium feature integration hooks for Quantum Vault
  • Chat functionality with OpenAI, Anthropic, and Ollama support

• BrainyChat: Real-time AI-powered conversations with vector + graph context

  • Natural language queries against your database
  • Multiple LLM provider support (OpenAI, Anthropic, Ollama)
  • Context-aware responses using vector similarity search
  • Chat history and session management

• Neural Import: Default SENSE augmentation for intelligent data processing

  • AI-powered entity extraction and relationship mapping
  • Automatic data structuring and categorization
  • Confidence scoring and insight generation
  • Batch processing with intelligent caching

• Quantum Vault: Premium closed-source repository (private)

  • Enterprise-grade connectors (Notion, Salesforce, Slack, Asana)
  • Advanced licensing system with trial support
  • Neural enhancement packs for specialized domains
  • Revenue projections: $127.5M over 3 years

Fixed

• Fixed TypeScript compilation errors in Cortex CLI
• Fixed color and emoji property issues in terminal output
• Resolved augmentation system type definitions
• Fixed error handling for unknown error types
• Corrected CortexConfig interface definition

Documentation

• Added comprehensive Brainy Chat implementation guide
• Created performance impact documentation proving zero overhead
• Added launch checklist for Quantum Vault
• Created aggressive revenue projections ($127.5M target)
• Reorganized README for better flow and clarity

Security

• Removed sensitive pitch deck from Git history completely
• Added .gitignore rules to prevent future sensitive file commits
• Implemented proper error handling for secure operations

0.55.0 (2025-08-08)

Added

• Cortex CLI: Initial implementation of command center
  • Basic structure and configuration management
  • Atomic age inspired UI with retro terminal aesthetics

0.52.0 (2025-08-07)

Fixed

• restore Brainy logo in README header (59d6344)

Changed

• bump version to 0.50.0 (1964607)
• bump version to 0.51.0 (cef71a9)
• bump version to 0.51.1 (e0cbe2e)
• bump version to 0.51.2 (82f20b0)
• remove unused rollup dependencies (e197c1e)

Added

• add automatic high-volume handling for S3 storage adapter (680ee10)
• add intelligent verb scoring system for automatic relationship weighting (bf89b01)
• add zero-configuration Brainy service template with augmentation-first architecture (b605d8a)
• establish Brainy as world's only true Vector + Graph database (c5e2f2c)
• restore comprehensive feature sections to README (5541c0a)
• restore developer-friendly sections and personality (0481b76)
• revolutionize README with problem-focused approach (afbeccd)

0.48.0 (2025-08-06)

Added

• add GPU acceleration for embeddings with smart device auto-detection (56c5641)

0.47.0 (2025-08-06)

⚠ BREAKING CHANGES

• Complete migration from TensorFlow.js to Transformers.js for embedding generation

This is a major architectural change that replaces TensorFlow.js (USE model) with Transformers.js (all-MiniLM-L6-v2) for significantly improved performance and reduced complexity.

Key Changes:

• Replace TensorFlow.js Universal Sentence Encoder with Transformers.js all-MiniLM-L6-v2
• Reduce model size from 525MB to 87MB (83% reduction)
• Reduce embedding dimensions from 512 to 384 (faster distance calculations)
• Remove TensorFlow.js Float32Array patching (caused ONNX conflicts)
• Implement smart bundled model detection for offline operation
• Add explicit model download script for Docker deployments
• Remove complex environment variables in favor of simple configuration
• Update all distance functions to use optimized pure JavaScript
• Remove TensorFlow-specific utilities and type definitions

Performance Improvements:

• Model loading: 5x faster (87MB vs 525MB)
• Memory usage: 75% reduction (~200-400MB vs ~1.5GB)
• Distance calculations: Faster pure JS vs GPU overhead for small vectors
• Cold start performance: Significantly improved

Files Changed:

• Updated package.json: New dependencies, simplified scripts
• Rewrote src/utils/embedding.ts: Complete Transformers.js implementation
• Updated src/utils/distance.ts: Optimized JavaScript distance functions
• Simplified src/setup.ts: Removed TensorFlow-specific patching
• Simplified src/utils/textEncoding.ts: Only Node.js TextEncoder/Decoder patches
• Deleted src/utils/robustModelLoader.ts: TensorFlow-specific loader
• Deleted src/types/tensorflowTypes.ts: TensorFlow type definitions
• Added scripts/download-models.cjs: Docker-compatible model downloader
• Added comprehensive documentation: README.md, OFFLINE_MODELS.md, analysis docs

Testing:

• All 19 tests passing
• Removed test mocking in favor of real implementation testing
• Updated test environment for Transformers.js compatibility
• Performance tests validate improved efficiency

This migration resolves production issues with Docker egress limitations and provides a more robust, performant foundation for vector operations.

• feat!: migrate from TensorFlow.js to Transformers.js with ONNX Runtime (79b1d5e)

Fixed

• resolve test failures and browser environment issues (c368864)

0.46.0 (2025-08-06)

Fixed

• improve local model loading with USE-lite tokenizer support (5eaf306)

Added

• add brainy-models-package v0.8.0 with USE-lite model (334b469)

Changed

• add models-download to gitignore (6e31a3c)

0.45.0 (2025-08-06)

Fixed

• improve model loading reliability with better error handling and updated fallback URLs (933f305)

0.44.0 (2025-08-05)

Fixed

• include all JavaScript modules in npm package (b37debb)

0.43.0 (2025-08-05)

⚠ BREAKING CHANGES

• Models are no longer bundled with the package. They are now loaded dynamically from CDN or custom paths.

Added

• package: add initial package.json for test consumer setup (f50e220)
• reliability: implement automatic offline model detection for production (042a545)
• test: add test script for BrainyData functionality (b7859e4)

Documentation

• readme: add security and enterprise benefits for offline models (5b022cf)
• readme: improve user engagement flow and add advanced features (2acd2e0)

Changed

• clean up deprecated functions and unused code (0214168)
• clean up project for release (d429a62)
• gitignore: add node_modules directory for test consumer to .gitignore (45eef34)
• gitignore: ignore npm package artifacts (6ed3ac5)
• release: 1.0.0 (53edf16)
• simplify build system and improve model loading flexibility (3d4c759)

0.41.0 (2025-08-05)

Added

• tools: update feature description for clarity (5e15dab)

Fixed

• security: resolve critical vulnerability in form-data dependency (8450af5)
• storage: resolve pagination warnings and improve S3 adapter performance (d7a1c1b)

0.40.0 (2025-08-05)

Fixed

• core: resolve TypeScript compilation errors and test failures (67db734)

0.39.0 (2025-08-04)

Added

• pagination: implement cursor-based pagination and enhance search caching (0f538f3)

Documentation

• add comprehensive performance docs and rebrand to Zero-to-Smart™ (80ca8e3)

0.38.0 (2025-08-04)

Documentation

• add distributed deployment architecture and enhancement proposals (4bb7a9f)
• add revised distributed implementation plan with practical phases (e3978e5)
• streamline README for better readability and user engagement (2492fe4)

Added

• distributed: add distributed mode with multi-instance coordination (8e4b0ef)
• docs: add S3 migration guide for optimized data transfer strategies (7b4c779)
• safety: enhance claude-commit with mandatory review and safety features (c20cc39)
• tools: add claude-commit AI-powered git commit tool (d05e320)
• tools: propagate safety features to all projects (8854b37)

0.37.0 (2025-08-04)

Fixed

• build: resolve TypeScript compilation errors in optimization modules (0e2bce1)
• types: add explicit ArrayBuffer type assertions for compression (7196fe2)
• types: resolve remaining ArrayBuffer type issues in compression methods (eb8c95e)

Added

• auto-configuration: implement automatic configuration system for optimal settings (aa64f49)
• docs: add comprehensive user guides and installation instructions for Brainy (d4dafbf)
• docs: update README and add large-scale optimizations guide for v0.36.0 (ae01bea)
• hnsw: implement comprehensive large-scale search optimizations (c39eee6)
• partitioning: simplify partition strategies and enable auto-tuning of semantic clusters (1015c33)

0.36.0 (2025-08-03)

Changed

• add CLAUDE.md to .gitignore (4c0b920)

Documentation

• add guidelines for Conventional Commit format and structured commit messages (f9a8595)

Added

• add verb and noun metadata handling in storage adapters (9778f1b)
• refactor verb storage to use HNSWVerb for improved performance (75ccf0f)

0.35.0 (2025-08-02)

0.34.0 (2025-08-02)

Changed

• release: 0.2.0 [skip ci] (c9ca141)
• release: 0.3.0 [skip ci] (437360c)
• release: 0.4.0 [skip ci] (be3a108)
• release: 0.5.0 (a05ebb5)
• release: 0.6.0 (26cb41a)
• release: 0.7.0 (153abe8)

0.33.0 (2025-08-01)

0.32.0 (2025-08-01)

0.31.0 (2025-07-31)

0.30.0 (2025-07-31)

0.29.0 (2025-07-31)

0.28.0 (2025-07-31)

0.27.1 (2025-07-31)

Changed

• changelog: remove manual changelog update script (72a649e)
• versioning: switch to standard-version for automated changelog generation (1f6a70d)

[0.26.0] - 2025-07-30

Added

• Organized documentation structure with docs/ directory
• Proper CHANGELOG.md for release management
• Statistics optimizations implemented across all storage adapters
• In-memory caching of statistics data
• Batched updates with adaptive flush timing
• Time-based partitioning for statistics files
• Error handling and retry mechanisms for statistics operations

Changed

• Moved technical documentation to docs/technical/
• Moved development documentation to docs/development/
• Moved guides to docs/guides/
• Archived temporary documentation files
• Refactored BaseStorageAdapter to include shared optimizations
• Updated FileSystemStorage, MemoryStorage, and OPFSStorage with new statistics handling
• Improved performance through reduced storage operations
• Enhanced scalability with time-based partitioning

Fixed

• Fixed FileSystemStorage constructor path operations issue where path module was used before being fully loaded
• Deferred path operations to init() method when path module is guaranteed to be available
• Resolved "Cannot read properties of undefined (reading 'join')" error

Technical Details

• Added scheduleBatchUpdate() and flushStatistics() methods to BaseStorageAdapter
• Updated core statistics methods: saveStatistics(), getStatistics(), incrementStatistic(), decrementStatistic(), and updateHnswIndexSize()
• Maintained backward compatibility with legacy statistics files
• Added fallback mechanisms for multiple storage locations

[Previous Versions]

For detailed implementation notes and technical summaries of previous versions, see:

• docs/technical/ - Technical documentation and analysis

---

How to Update This Changelog

This project now uses standard-version to automatically generate the changelog from commit messages.

Commit Message Format

Follow the Conventional Commits specification for your commit messages:

<type>(<scope>): <description>

[optional body]

[optional footer(s)]

Where <type> is one of:

• feat: A new feature (maps to Added section)
• fix: A bug fix (maps to Fixed section)
• chore: Regular maintenance tasks (maps to Changed section)
• docs: Documentation changes (maps to Documentation section)
• refactor: Code changes that neither fix bugs nor add features (maps to Changed section)
• perf: Performance improvements (maps to Changed section)

Examples:

feat(storage): add new file system adapter
fix(hnsw): resolve index corruption on large datasets
docs(readme): update installation instructions
refactor(core): simplify graph traversal algorithm

Releasing a New Version

To release a new version:

1. Ensure all changes are committed
2. Run one of:
   • npm run release (for patch version)
   • npm run release:patch (same as above)
   • npm run release:minor (for minor version)
   • npm run release:major (for major version)
3. Push changes with tags: git push --follow-tags origin main

The changelog will be automatically updated based on your commit messages.