soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions 2
brainy/CHANGES.md
dpsifr 86fb1220b6 **feat(core, migration, docs): introduce dimension mismatch resolution tools and migration guide** 
- **Core**:
  - Added `check-database.js` to verify database status and validate search functionality.
  - Created `fix-dimension-mismatch.js` to handle re-embedding of existing data to resolve dimension mismatch from 3 to 512.
  - Improved test cases by updating vector operations to support 512 dimensions, replacing previously hardcoded dimensions.

- **Migration**:
  - Developed `DIMENSION_MISMATCH_SUMMARY.md`, detailing the root cause, solution, and preventive strategies for dimension mismatch issues.
  - Added `production-migration-guide.md` for structured production migration with detailed steps on re-embedding strategies, batching, and error handling.

- **Tests**:
  - Enhanced test coverage with 512-dimensional vector validation.
  - Introduced helper functions for consistent vector testing behavior and streamlined search test cases.

- **Documentation**:
  - Updated project documentation to highlight the resolution process for dimension mismatches, emphasizing preventive mechanisms such as auto-migration and version tracking.

**Purpose**: Address critical dimension mismatch issues caused by embedding changes, restore functionality, and provide a roadmap for robust prevention strategies and migration processes.
2025-07-25 13:38:56 -07:00

1.2 KiB
Raw Blame History

Brainy Changes Log

2025-07-25

Changed

  • BREAKING CHANGE: Standardized vector dimensions to 512 throughout the codebase. The dimensions configuration option has been removed from BrainyDataConfig. All vectors are now required to have exactly 512 dimensions, matching the Universal Sentence Encoder's output. This change ensures consistency between data insertion, storage, and search operations, eliminating potential dimension mismatch issues. See VECTOR_DIMENSION_STANDARDIZATION.md for details and migration instructions.
  • Updated all tests to expect 512 dimensions and use 512-dimensional test vectors. This includes tests in core.test.ts, vector-operations.test.ts, environment.browser.test.ts, environment.node.test.ts, and statistics.test.ts.

2025-07-23

Fixed

  • Fixed an issue in the web service where tests were failing with "Cannot read properties of undefined (reading 'join')" error. The problem was a race condition in the FileSystemStorage constructor, where the path module was being used before it was fully loaded. The fix ensures that the path module is properly imported and initialized before creating the FileSystemStorage instance.

Previous Changes

(Previous changes would be listed here)