feat(core, tests): add standalone getStatistics function and improve storage configuration
- **Core**: Introduced a new `getStatistics` utility function in `statistics.ts` for fetching database statistics at the root level of the library. Enhanced `BrainyData` methods to ensure metadata includes `id` field and refined statistics calculations, excluding verbs from the noun count.
- **Tests**: Added comprehensive test coverage in `statistics.test.ts` for the new utility function, validating proper error handling, statistics accuracy, and consistent results between instance methods and standalone function.
- **Storage Config**: Enabled dynamic support for AWS S3, Cloudflare R2, and Google Cloud Storage in web service configuration, utilizing environment variables for adapter setup. Addressed a race condition in `FileSystemStorage` initialization by deferring path module imports.
**Purpose**: Enhance database analytics by introducing a reusable `getStatistics` function, improve flexibility in storage configuration, and ensure robust testing for reliability and accuracy.
2025-07-23 16:26:59 -07:00
|
|
|
# Brainy Changes Log
|
feat(tests): add robust mock implementations and expand test coverage for S3 and OPFS storage
- Introduced comprehensive mock for the Origin Private File System (OPFS) in `tests/mocks/opfs-mock.ts`, simulating environment for detailed storage testing.
- Added `tests/opfs-storage.test.ts`, containing extensive test cases for `OPFSStorage` operations including metadata, nouns, verbs, and storage status.
- Improved S3 mock implementation in `tests/mocks/s3-mock.ts` with better object persistence, validation, and logging to emulate real S3 behavior.
- Resolved issues related to metadata, nouns, verbs, and storage usage inconsistencies in mock storage adapters.
- Enhanced logging and error handling to aid in debugging and test reliability.
Purpose: Improve test completeness and reliability by introducing detailed mocks and extended test cases for S3 and OPFS storage systems.
2025-07-21 13:42:10 -07:00
|
|
|
|
**feat(core, migration, docs): introduce dimension mismatch resolution tools and migration guide**
- **Core**:
- Added `check-database.js` to verify database status and validate search functionality.
- Created `fix-dimension-mismatch.js` to handle re-embedding of existing data to resolve dimension mismatch from 3 to 512.
- Improved test cases by updating vector operations to support 512 dimensions, replacing previously hardcoded dimensions.
- **Migration**:
- Developed `DIMENSION_MISMATCH_SUMMARY.md`, detailing the root cause, solution, and preventive strategies for dimension mismatch issues.
- Added `production-migration-guide.md` for structured production migration with detailed steps on re-embedding strategies, batching, and error handling.
- **Tests**:
- Enhanced test coverage with 512-dimensional vector validation.
- Introduced helper functions for consistent vector testing behavior and streamlined search test cases.
- **Documentation**:
- Updated project documentation to highlight the resolution process for dimension mismatches, emphasizing preventive mechanisms such as auto-migration and version tracking.
**Purpose**: Address critical dimension mismatch issues caused by embedding changes, restore functionality, and provide a roadmap for robust prevention strategies and migration processes.
2025-07-25 13:38:56 -07:00
|
|
|
## 2025-07-25
|
|
|
|
|
|
|
|
|
|
### Changed
|
|
|
|
|
|
|
|
|
|
- **BREAKING CHANGE**: Standardized vector dimensions to 512 throughout the codebase. The `dimensions` configuration
|
|
|
|
|
option has been removed from `BrainyDataConfig`. All vectors are now required to have exactly 512 dimensions, matching
|
|
|
|
|
the Universal Sentence Encoder's output. This change ensures consistency between data insertion, storage, and search
|
|
|
|
|
operations, eliminating potential dimension mismatch issues. See `VECTOR_DIMENSION_STANDARDIZATION.md` for details and
|
|
|
|
|
migration instructions.
|
|
|
|
|
- Updated all tests to expect 512 dimensions and use 512-dimensional test vectors. This includes tests in core.test.ts,
|
|
|
|
|
vector-operations.test.ts, environment.browser.test.ts, environment.node.test.ts, and statistics.test.ts.
|
|
|
|
|
|
feat(core, tests): add standalone getStatistics function and improve storage configuration
- **Core**: Introduced a new `getStatistics` utility function in `statistics.ts` for fetching database statistics at the root level of the library. Enhanced `BrainyData` methods to ensure metadata includes `id` field and refined statistics calculations, excluding verbs from the noun count.
- **Tests**: Added comprehensive test coverage in `statistics.test.ts` for the new utility function, validating proper error handling, statistics accuracy, and consistent results between instance methods and standalone function.
- **Storage Config**: Enabled dynamic support for AWS S3, Cloudflare R2, and Google Cloud Storage in web service configuration, utilizing environment variables for adapter setup. Addressed a race condition in `FileSystemStorage` initialization by deferring path module imports.
**Purpose**: Enhance database analytics by introducing a reusable `getStatistics` function, improve flexibility in storage configuration, and ensure robust testing for reliability and accuracy.
2025-07-23 16:26:59 -07:00
|
|
|
## 2025-07-23
|
feat(tests): add robust mock implementations and expand test coverage for S3 and OPFS storage
- Introduced comprehensive mock for the Origin Private File System (OPFS) in `tests/mocks/opfs-mock.ts`, simulating environment for detailed storage testing.
- Added `tests/opfs-storage.test.ts`, containing extensive test cases for `OPFSStorage` operations including metadata, nouns, verbs, and storage status.
- Improved S3 mock implementation in `tests/mocks/s3-mock.ts` with better object persistence, validation, and logging to emulate real S3 behavior.
- Resolved issues related to metadata, nouns, verbs, and storage usage inconsistencies in mock storage adapters.
- Enhanced logging and error handling to aid in debugging and test reliability.
Purpose: Improve test completeness and reliability by introducing detailed mocks and extended test cases for S3 and OPFS storage systems.
2025-07-21 13:42:10 -07:00
|
|
|
|
feat(core, tests): add standalone getStatistics function and improve storage configuration
- **Core**: Introduced a new `getStatistics` utility function in `statistics.ts` for fetching database statistics at the root level of the library. Enhanced `BrainyData` methods to ensure metadata includes `id` field and refined statistics calculations, excluding verbs from the noun count.
- **Tests**: Added comprehensive test coverage in `statistics.test.ts` for the new utility function, validating proper error handling, statistics accuracy, and consistent results between instance methods and standalone function.
- **Storage Config**: Enabled dynamic support for AWS S3, Cloudflare R2, and Google Cloud Storage in web service configuration, utilizing environment variables for adapter setup. Addressed a race condition in `FileSystemStorage` initialization by deferring path module imports.
**Purpose**: Enhance database analytics by introducing a reusable `getStatistics` function, improve flexibility in storage configuration, and ensure robust testing for reliability and accuracy.
2025-07-23 16:26:59 -07:00
|
|
|
### Fixed
|
feat(tests): add robust mock implementations and expand test coverage for S3 and OPFS storage
- Introduced comprehensive mock for the Origin Private File System (OPFS) in `tests/mocks/opfs-mock.ts`, simulating environment for detailed storage testing.
- Added `tests/opfs-storage.test.ts`, containing extensive test cases for `OPFSStorage` operations including metadata, nouns, verbs, and storage status.
- Improved S3 mock implementation in `tests/mocks/s3-mock.ts` with better object persistence, validation, and logging to emulate real S3 behavior.
- Resolved issues related to metadata, nouns, verbs, and storage usage inconsistencies in mock storage adapters.
- Enhanced logging and error handling to aid in debugging and test reliability.
Purpose: Improve test completeness and reliability by introducing detailed mocks and extended test cases for S3 and OPFS storage systems.
2025-07-21 13:42:10 -07:00
|
|
|
|
**feat(core, migration, docs): introduce dimension mismatch resolution tools and migration guide**
- **Core**:
- Added `check-database.js` to verify database status and validate search functionality.
- Created `fix-dimension-mismatch.js` to handle re-embedding of existing data to resolve dimension mismatch from 3 to 512.
- Improved test cases by updating vector operations to support 512 dimensions, replacing previously hardcoded dimensions.
- **Migration**:
- Developed `DIMENSION_MISMATCH_SUMMARY.md`, detailing the root cause, solution, and preventive strategies for dimension mismatch issues.
- Added `production-migration-guide.md` for structured production migration with detailed steps on re-embedding strategies, batching, and error handling.
- **Tests**:
- Enhanced test coverage with 512-dimensional vector validation.
- Introduced helper functions for consistent vector testing behavior and streamlined search test cases.
- **Documentation**:
- Updated project documentation to highlight the resolution process for dimension mismatches, emphasizing preventive mechanisms such as auto-migration and version tracking.
**Purpose**: Address critical dimension mismatch issues caused by embedding changes, restore functionality, and provide a roadmap for robust prevention strategies and migration processes.
2025-07-25 13:38:56 -07:00
|
|
|
- Fixed an issue in the web service where tests were failing with "Cannot read properties of undefined (reading 'join')"
|
|
|
|
|
error. The problem was a race condition in the FileSystemStorage constructor, where the path module was being used
|
|
|
|
|
before it was fully loaded. The fix ensures that the path module is properly imported and initialized before creating
|
|
|
|
|
the FileSystemStorage instance.
|
feat(tests): add robust mock implementations and expand test coverage for S3 and OPFS storage
- Introduced comprehensive mock for the Origin Private File System (OPFS) in `tests/mocks/opfs-mock.ts`, simulating environment for detailed storage testing.
- Added `tests/opfs-storage.test.ts`, containing extensive test cases for `OPFSStorage` operations including metadata, nouns, verbs, and storage status.
- Improved S3 mock implementation in `tests/mocks/s3-mock.ts` with better object persistence, validation, and logging to emulate real S3 behavior.
- Resolved issues related to metadata, nouns, verbs, and storage usage inconsistencies in mock storage adapters.
- Enhanced logging and error handling to aid in debugging and test reliability.
Purpose: Improve test completeness and reliability by introducing detailed mocks and extended test cases for S3 and OPFS storage systems.
2025-07-21 13:42:10 -07:00
|
|
|
|
feat(core, tests): add standalone getStatistics function and improve storage configuration
- **Core**: Introduced a new `getStatistics` utility function in `statistics.ts` for fetching database statistics at the root level of the library. Enhanced `BrainyData` methods to ensure metadata includes `id` field and refined statistics calculations, excluding verbs from the noun count.
- **Tests**: Added comprehensive test coverage in `statistics.test.ts` for the new utility function, validating proper error handling, statistics accuracy, and consistent results between instance methods and standalone function.
- **Storage Config**: Enabled dynamic support for AWS S3, Cloudflare R2, and Google Cloud Storage in web service configuration, utilizing environment variables for adapter setup. Addressed a race condition in `FileSystemStorage` initialization by deferring path module imports.
**Purpose**: Enhance database analytics by introducing a reusable `getStatistics` function, improve flexibility in storage configuration, and ensure robust testing for reliability and accuracy.
2025-07-23 16:26:59 -07:00
|
|
|
## Previous Changes
|
feat(tests): add robust mock implementations and expand test coverage for S3 and OPFS storage
- Introduced comprehensive mock for the Origin Private File System (OPFS) in `tests/mocks/opfs-mock.ts`, simulating environment for detailed storage testing.
- Added `tests/opfs-storage.test.ts`, containing extensive test cases for `OPFSStorage` operations including metadata, nouns, verbs, and storage status.
- Improved S3 mock implementation in `tests/mocks/s3-mock.ts` with better object persistence, validation, and logging to emulate real S3 behavior.
- Resolved issues related to metadata, nouns, verbs, and storage usage inconsistencies in mock storage adapters.
- Enhanced logging and error handling to aid in debugging and test reliability.
Purpose: Improve test completeness and reliability by introducing detailed mocks and extended test cases for S3 and OPFS storage systems.
2025-07-21 13:42:10 -07:00
|
|
|
|
feat(core, tests): add standalone getStatistics function and improve storage configuration
- **Core**: Introduced a new `getStatistics` utility function in `statistics.ts` for fetching database statistics at the root level of the library. Enhanced `BrainyData` methods to ensure metadata includes `id` field and refined statistics calculations, excluding verbs from the noun count.
- **Tests**: Added comprehensive test coverage in `statistics.test.ts` for the new utility function, validating proper error handling, statistics accuracy, and consistent results between instance methods and standalone function.
- **Storage Config**: Enabled dynamic support for AWS S3, Cloudflare R2, and Google Cloud Storage in web service configuration, utilizing environment variables for adapter setup. Addressed a race condition in `FileSystemStorage` initialization by deferring path module imports.
**Purpose**: Enhance database analytics by introducing a reusable `getStatistics` function, improve flexibility in storage configuration, and ensure robust testing for reliability and accuracy.
2025-07-23 16:26:59 -07:00
|
|
|
(Previous changes would be listed here)
|