- Introduced comprehensive mock for the Origin Private File System (OPFS) in `tests/mocks/opfs-mock.ts`, simulating environment for detailed storage testing.
- Added `tests/opfs-storage.test.ts`, containing extensive test cases for `OPFSStorage` operations including metadata, nouns, verbs, and storage status.
- Improved S3 mock implementation in `tests/mocks/s3-mock.ts` with better object persistence, validation, and logging to emulate real S3 behavior.
- Resolved issues related to metadata, nouns, verbs, and storage usage inconsistencies in mock storage adapters.
- Enhanced logging and error handling to aid in debugging and test reliability.
Purpose: Improve test completeness and reliability by introducing detailed mocks and extended test cases for S3 and OPFS storage systems.
- Added `getFile` method to `FileSystemHandle` interface for enhanced TypeScript compatibility with File System Access API.
- Improved maintainability by reformatting and aligning imports in `brainyData.ts`.
- Standardized spacing and indentation across structured comments and interface fields.
Purpose: Improve TypeScript support for file system operations and enhance code readability with consistent formatting and import management.
- Added `BaseStorage` abstract class to provide common functionality for storage adapters.
- Implemented `FileSystemStorage` adapter for Node.js environments, supporting persistent file-based storage.
- Added directory structure management methods to `FileSystemStorage` (`ensureDirectoryExists`, `clear`, etc.).
- Implemented `MemoryStorage` adapter for in-memory storage suitable for non-persistent environments.
- Supported CRUD operations for nouns, verbs, and metadata in both `FileSystemStorage` and `MemoryStorage`.
- Included utility methods like `mapToObject` for serialization and deserialization of complex data structures.
Purpose: Provide extensible, reusable storage solutions with both persistent and non-persistent options for diverse application needs.
- Updated `VERSION` in `version.ts` to `0.14.0`.
- Updated `npm` badge in `README.md` to reflect version `0.14.0`.
- Updated `version` field and dependency reference in `cli-package/package.json` to `0.14.0`.
Purpose: Prepare for release by synchronizing version across source files and documentation.
- Added a check to avoid appending `ROOT_DIR` if the `rootDir` already ends with it.
- Ensures proper directory paths setup without redundant path segments.
- Refactor embedding functions to support customizable verbosity through new helper methods (`getDefaultEmbeddingFunction`, `getDefaultBatchEmbeddingFunction`).
- Ensure metadata initialization with default values when null during search results.
- **Logging Cleanup**:
- Removed redundant `console.log` statements in `brainyData.ts` and `embedding.ts` related to Universal Sentence Encoder initialization and load function detection.
- Replaced detailed logs with concise comments to streamline debugging and reduce noisy outputs.
- **Purpose**:
- Improves code readability, reduces runtime logging noise, and aligns logging verbosity with the project's streamlined debugging practices.
- **Core Improvements**:
- Refactored logging functions into a unified `logger` method for consistent output across the library.
- Enabled the `forceMemoryStorage` option in `BrainyData` initialization for improved storage flexibility in tests and specific use cases.
- **TensorFlow.js and Environment Updates**:
- Clarified the dependency structure in `README.md` to emphasize bundled dependencies and remove legacy peer dependency instructions.
- Simplified and reformatted environment detection logic for better maintainability and readability.
- **Testing Enhancements**:
- Added `tests/package-size-limit.test.ts` to monitor and validate npm package size against defined thresholds.
- Updated `tests/environment.node.test.ts` and core tests to leverage `forceMemoryStorage` for better test setup standardization.
- Improved test isolation with expanded `globalThis` utility definitions and cleanup logic.
- **Documentation**:
- Added detailed best practices for debugging and organizing tests in `DEVELOPERS.md`.
- Removed outdated installation hints from `package.json` and streamlined scripts by including `test:size` for package size validation.
**Purpose**: These changes unify core logging mechanisms, expand configurability of storage options, and improve testing reliability and coverage. Documentation and clarity are enhanced to align with updated functionality and best practices.
- Updated the version in `README.md`, `package.json`, and `version.ts` to `0.12.0` to reflect the new release.
- Adjusted dependency reference for `@soulcraft/brainy` in `cli-package/package.json` to use version `0.12.0`.
**Purpose**: Prepares for the 0.12.0 release, ensuring all version references are consistent across documentation and configuration files.
- **Vector Handling Updates**:
- Added a `dimensions` property to `BrainyDataConfig` for specifying vector dimensions.
- Introduced validation for vector dimensions during database creation and insertion to ensure consistency.
- Enhanced error handling and logging for dimension mismatches.
- **Model Loading Improvements**:
- Implemented retry logic for Universal Sentence Encoder model loading to handle network instability and JSON parsing errors gracefully.
- Improved logging and debugging support for failures during model initialization and embedding operations.
- **Compatibility Enhancements**:
- Updated polyfills to support TensorFlow.js compatibility across diverse server environments (Node.js, serverless, etc.).
- Introduced and refactored global `TextEncoder`/`TextDecoder` definitions for seamless operation in non-browser environments.
- Simplified TensorFlow.js backend setup with streamlined imports and logging for GPU/WebGL fallback.
- **Purpose**:
- These updates improve BrainyData's robustness, enforce correct vector usage, and extend compatibility with varied runtime environments. The changes enhance the usability, reliability, and cross-platform readiness of core functionalities.
- Simplified explanation of server functionality (`WebSocket` and `REST`) to emphasize its exclusion from the main Brainy package for maintaining lightweight browser bundles.
- Removed detailed Node.js-specific setup examples, configuration options, and cloud-wrapper deployment instructions.
- Retained relevant core functionality usage details for broader accessibility.
This update aligns the documentation with recent repository changes, focusing on clarity, maintainability, and providing guidance only on actively supported features.
- Removed obsolete scripts: `test-all-environments.js`, `test-fallback-function.js`, `test-fallback-simple.js`, `test-fix.js`, `test-tensorflow-textencoder.js`, `test-unified-encoding.js`, and `test-worker-utils.js`.
- Introduced `scripts/comprehensive-test.js` as a unified testing script covering all environments: Browser, Node.js, and CLI.
- Added `examples/cli-wrapper-example.js` to demonstrate a proper CLI implementation with TensorFlow.js initialization.
This refactor simplifies the testing structure by consolidating redundant scripts into a single comprehensive script while ensuring robust cross-environment coverage.
- Updated version to `0.11.0` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.11.0` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
This release ensures consistent versioning across the project and prepares for the distribution of changes in version `0.11.0`.
- Refactored `applyTensorFlowPatch` in `textEncoding.ts` to introduce a unified `Platform` class supporting both Node.js and browser environments with native `TextEncoder`/`TextDecoder`.
- Modified global object handling to target `PlatformNode` in Node.js and `PlatformBrowser` in browser environments.
- Added `getTextEncoder` and `getTextDecoder` utility functions to simplify text encoding/decoding across platforms.
- Introduced `setup.ts` to apply environment patches before other modules load.
- Updated CLI entry points (`cli.ts`, `rollup.config.js`) to ensure patching precedes TensorFlow.js usage.
- Enhanced test coverage for TextEncoder compatibility with `test-fix.js`.
- Adjusted imports in `test-tensorflow-textencoder.ts` to align with the updated `./utils/textEncoding.js`.
- Streamlined constructor definitions in `Platform` for improved maintainability.
This update ensures robust compatibility for TensorFlow.js by patching and standardizing text encoding/decoding functionality across environments.
- Introduced `test-fallback-function.js` and `test-fallback-simple.js` to validate `executeInThread` fallback functionality with both named and anonymous compute-intensive functions.
- Added `test-tensorflow-textencoder.js` for TensorFlow.js and TextEncoder tests in a Node.js environment.
- Created `test-tensorflow-textencoder.html` for browser-based TensorFlow.js and TextEncoder tests.
- Implemented cross-environment test support in `cli-package/src/test-tensorflow-textencoder.ts` for CLI functionality.
- Enhanced `src/utils/embedding.ts`, `textEncoding.ts`, and `brainy-wrapper.js` to include updated global `TextEncoder` and `TextDecoder` utilities for compatibility and worker improvements.
- Standardized and expanded utility methods in `PlatformNode` for broader support, including `isFloat32Array` and `isTypedArray` checks.
- Updated Node.js requirement to `>= 24.4.0` across documentation and configuration files for compatibility improvements.
This update introduces comprehensive testing for fallback mechanisms, TensorFlow.js, and TextEncoder across multiple environments, ensuring robustness and compatibility.
- Updated version to `0.10.0` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Removed `CHANGELOG.md` from the root and `cli-package`, as GitHub auto-generates release notes.
- Refactored `brainy-wrapper.js` to simplify utility methods (`isFloat32Array` and `isTypedArray`) by moving them to instance-level definitions.
- Standardized Node.js version requirement to `>= 24.3.0` across documentation and configuration files.
- Improved `textEncoding.ts` by introducing a unified `Platform` class for cross-environment compatibility (Node.js and browser).
- Restructured scripts:
- Replaced outdated commands (e.g., `test-cli` -> `test:cli` and `test-all` -> `test:all`) for consistency.
- Removed changelog updates from `scripts/create-github-release.js` as part of simplification.
- Enhanced `patch-textencoder.js` to also patch `worker.js` for both `TextEncoder` and `TextDecoder`.
This release improves compatibility with Node.js 24, simplifies the deployment/changelog process, and refines text encoding and worker functionalities.
- Updated version to `0.9.37` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.37` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
- Added release details for version `0.9.37` in `CHANGELOG.md`.
This update ensures consistent versioning across the project and documents the latest release.
- Added `cli-package/brainy-wrapper.js` to patch global `TextEncoder` and `TextDecoder` for Node.js environments, ensuring compatibility with TensorFlow.js.
- Implemented unified text encoding utilities in `cli-package/src/utils/textEncoding.ts` for cross-environment consistency.
- Enhanced `src/worker.js` and introduced `src/worker.ts` to improve worker execution with better function serialization and error handling.
- Updated `package.json`:
- Modified the `build` script to include a patch for `TextEncoder`.
- Added `test-all` script for multi-environment testing.
- Introduced the Puppeteer dependency for browser testing.
- Created a favicon generation script `scripts/create-favicon.js` using a base64-encoded icon.
- Added `scripts/test-all-environments.js` to run automated tests across Node.js, browser, and CLI environments.
- Enhanced `src/utils/workerUtils.ts` to support improved fallback mechanisms and robust worker pooling.
This update improves the project's compatibility across environments, refines worker functionalities, and adds comprehensive testing support for reliability.
- Updated version to `0.9.36` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.36` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
- Added release details for version `0.9.36` in `CHANGELOG.md`.
This update ensures consistent versioning across the project and documents the latest release.
- Updated version to `0.9.35` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.35` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
- Enhanced GitHub release script (`scripts/create-github-release.js`) to check and push tags to remote if they exist locally.
This update ensures consistent versioning across the project and improves release automation with better tag management.
- Updated Node.js version requirement to `>=24.3.0` in `README.md` and `package-lock.json`.
- Upgraded `TypeScript` to version `5.4.5` and synced references.
- Updated multiple dependencies, including:
- `@aws-sdk/client-s3` to `^3.540.0`.
- `uuid` to `^9.0.1`.
- Development tools such as `eslint` to `^8.57.0`, `typescript-eslint` plugins to `^7.4.0`, and more.
- Refreshed transitive dependencies with up-to-date versions for consistency and performance.
This update ensures compatibility with the latest Node.js features, aligns TypeScript references, and keeps the project dependencies up-to-date for improved stability and functionality.
- Updated version to `0.9.34` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.34` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
This update ensures consistent versioning across the project and aligns dependencies with the latest release.
- Updated version to `0.9.33` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.33` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
- Refactored `TextEncoder` and `TextDecoder` implementations in `src/utils/tensorflowUtils.ts`:
- Added dynamic environment-aware loading for improved Node.js and browser compatibility.
- Enhanced fallback mechanisms for environments without global support.
This update ensures consistent versioning across the project and improves the robustness of utility methods for broader environment compatibility.
- Updated version to `0.9.32` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.32` in the CLI package for consistency.
- Renamed npm scripts `publish` to `deploy` and `publish:cli` to `deploy:cli`, aligning terminology with deployment workflows.
- Updated npm version badge in `README.md` to reflect the new version.
This update ensures consistent versioning across the project, aligns CLI and main package references, and enhances clarity in deployment script naming.
- Updated version to `0.9.31` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.31` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
This update ensures consistent versioning across the project and aligns references between main and CLI packages.
- Refactored `TextEncoder` and `TextDecoder` in `src/utils/tensorflowUtils.ts` to handle environments without global availability, logging warnings and providing fallback implementations.
- Enhanced threading utilities in `src/utils/environment.ts`:
- Introduced `areWorkerThreadsAvailableSync()` and `isThreadingAvailableAsync()` for synchronous and asynchronous checks.
- Improved compatibility across browser and Node.js environments.
- Migrated `fileSystemStorage.ts` to use dynamic imports for Node.js modules, removing synchronous loading mechanisms to align with ES module standards.
- Updated `cli-package/package.json` keywords to include `browser` and `container`, improving discoverability.
- Added `worker_threads` as an external dependency in `cli-package/rollup.config.js` and `rollup.config.js`.
- Updated `.gitignore` to ignore all `tgz` files dynamically under main and CLI package.
This update improves threading compatibility, enhances environment-specific support, and aligns with modern module loading practices.
- Updated version to `0.9.30` in `package.json`, `cli-package/package.json`, `src/utils/version.ts`, and `README.md`.
- Synced `@soulcraft/brainy` dependency version to `0.9.30` for CLI package consistency.
- Modified npm version badge in `README.md` to reflect the new version.
- Added detailed publishing instructions in `docs/publishing-cli.md` to guide users on publishing both, main, or CLI packages.
- Introduced new npm scripts `publish:cli` and `build:cli` for streamlined package-specific operations.
This update ensures cohesive versioning, provides clearer publishing guidelines, and improves maintainability with new build/publish scripts.
- Updated version to `0.9.29` in `package.json`, `cli-package/package.json`, `README.md`, and `src/utils/version.ts`.
- Synchronized `@soulcraft/brainy` dependency version to `0.9.29` in CLI package for consistency.
- Updated npm version badge in `README.md` to reflect `0.9.29`.
This update ensures consistent versioning across the project and maintains alignment between main and CLI package dependencies.
- Updated version to `0.9.28` in `package.json`, `cli-package/package.json`, `README.md`, and `src/utils/version.ts`.
- Synchronized `@soulcraft/brainy` dependency version to `0.9.28` in CLI package for consistency.
- Updated npm version badge in `README.md` to reflect `0.9.28`.
This update ensures consistent versioning across the project and maintains alignment between main and CLI package dependencies.
- Updated version to `0.9.27` in `package.json`, `cli-package/package.json`, `README.md`, and `src/utils/version.ts`.
- Synchronized `@soulcraft/brainy` dependency version to `0.9.27` in CLI package for consistency.
- Updated npm version badge in `README.md` to reflect `0.9.27`.
This update ensures accurate versioning across the project and maintains alignment between main and CLI package dependencies.
- Removed outdated Node.js v24 compatibility patches for `TextEncoder` and `TextDecoder` in `cli-wrapper.js` and other modules since TensorFlow.js now supports Node.js v24+ natively.
- Introduced a utility module `src/utils/tensorflowUtils.ts` for TensorFlow.js compatibility, defining global `PlatformNode` and related utilities.
- Enhanced `fileSystemStorage.ts` with improved Node.js module loading mechanisms for better compatibility across environments.
- Simplified dependency requirements by reducing the minimum Node.js version to `>=23.0.0`.
- Updated `package.json`, `cli-package/package.json`, and `README.md` versions to `0.9.25`.
- Harmonized `cli-package` and main package dependencies to ensure version alignment.
- Improved global compatibility with TensorFlow.js in mixed Node.js environments.
This update modernizes TensorFlow.js integration, reduces maintenance efforts, and aligns compatibility with current Node.js and TensorFlow.js capabilities.
- Introduced `scripts/test-cli-locally.js` to streamline local testing of the CLI package before publishing.
- Builds main and CLI packages, creates local tarballs, and installs the CLI package globally for testing.
- Ensures version synchronization between main and CLI packages during testing.
- Updated `cli-package/package.json` version to `0.9.23` and synchronized `@soulcraft/brainy` dependency to `0.9.22`.
- Patched `TextEncoder` fallback for Node.js v24 in `cli-wrapper.js` to ensure overall CLI compatibility.
- Added new `test-cli` npm script to `package.json`.
This update simplifies the CLI local testing process, maintains version alignment, and improves readiness for Node.js v24.
- Updated version to `0.9.21` in `README.md`, `package.json`, `cli-package/package.json`, and `src/utils/version.ts`.
- Updated npm version badge in `README.md` to reflect `0.9.21`.
- Synchronized `@soulcraft/brainy` dependency version to `0.9.21` in CLI package for consistency.
This release ensures accurate versioning across the project and aligns dependencies with the latest version.
- Updated version to `0.9.20` across `package.json`, `cli-package/package.json`, `src/utils/version.ts`, and `cli-package/package-lock.json`.
- Updated npm version badge in `README.md` to reflect `0.9.20`.
- Adjusted `package.json` to include additional files in the distribution bundle, such as `dist/unified.js`, `dist/brainy.min.js`, and other key directories.
- Updated `@soulcraft/brainy` dependency to `0.9.20` in CLI package to align dependencies with the latest version.
- Resolved dependency integrity updates in `package-lock.json`.
- Improved dependency management for better version consistency and distribution.
This release ensures up-to-date versioning, improved distribution content management, and enhanced dependency alignment.
- Introduced `package-lock.json` for the `@soulcraft/brainy-cli` package to manage dependencies explicitly.
- Ensures reproducible installs and aligns the CLI with the latest project dependencies.
- Supports dependency resolution, locking versions, and maintaining consistency across development and CI/CD environments.
- Updated version to `0.9.18` in `package.json`, `cli-package/package.json`, and `src/utils/version.ts`.
- Updated npm badge in `README.md` to reflect version `0.9.18`.
- Adjusted `publish-cli.js` to synchronize publishing of the main and CLI packages with enhanced build steps.
- Introduced `tsconfig.json` updates to include type paths (`types.d.ts`) in CLI for better type handling.
- Refined CLI scripts for improved TypeScript annotations and better code structure.
- Removed `@types/omelette` and `omelette` dependencies along with associated CLI configurations to simplify the codebase.
- Updated `.gitignore` to exclude CLI `dist` directory for cleaner repository management.
- Added `deploy:both` npm script for combined publishing of main and CLI packages.
This release ensures version consistency, improves CLI workflows, and enhances type handling while decluttering unused dependencies.
- Deleted the entire `src/cli.ts` file, effectively removing the Brainy CLI component.
- The change streamlines the codebase by eliminating unused and redundant code related to the CLI.
- This refactor supports maintaining a cleaner and more focused project architecture.
- Added a new `@soulcraft/brainy-cli` package to provide a dedicated CLI interface for the Brainy database.
- Implemented a CLI wrapper script (`cli-wrapper.js`) to manage global package execution and version handling.
- Introduced `scripts/publish-cli.js` to automate synchronized publishing of the main package and CLI, ensuring version consistency.
- Created documentation for CLI usage and publishing workflow (`publishing-cli.md`).
- Updated `package.json` to include CLI-related build and deploy scripts.
- Adjusted `README.md` to document the CLI package, usage instructions, and build separation.
This feature introduces a standalone CLI package, improves deployment automation, and enhances documentation to align with the updated architecture.
- Updated `npm` badge in `README.md` to reflect version `0.9.15`.
- Updated `VERSION` constant in `src/utils/version.ts` to `0.9.15`.
These changes ensure version consistency in preparation for the new release.
- Updated `npm` badge in `README.md` to reflect version `0.9.14`.
- Updated `VERSION` constant in `src/utils/version.ts` to `0.9.14`.
- Replaced `build:all` script references in documentation with separate `build` and `build:browser` commands for better clarity.
- Adjusted `demo` script to build both Node.js and browser versions before running the demo.
- Updated prerequisites in `README.demo.md` to align with the new build workflow.
- Refined `package.json` `files` field to clean up package distribution artifacts.
These changes ensure version consistency, clarify build processes, and improve documentation to streamline the developer experience.
- Updated `npm` badge in `README.md` to reflect version `0.9.13`.
- Updated `VERSION` constant in `src/utils/version.ts` to `0.9.13`.
- Refined `rollup.config.js` to conditionally export configurations based on build type (`cli` or `main`).
- Added `build:cli` script in `package.json` for CLI-specific builds.
- Updated `cli` script to include CLI build step for executable generation.
- Adjusted `package.json` `files` field to exclude CLI build artifacts in package distribution.
These changes ensure version consistency, improve CLI build workflows, and align the package distribution with project requirements.
- Added `test-browser-worker.html` to demonstrate threading with Web Workers in browser environments.
- Added `test-fallback.html` to verify fallback functionality when threading is unavailable.
- Created `THREADING.md` to document unified threading implementation, including Node.js Worker Threads, Web Workers, and fallback mechanisms.
- Updated Node.js version requirement in `README.md` and `package.json` to `>=24.0.0` for compatibility with improved Worker Threads API.
- Enhanced `workerUtils.ts` to implement threading with a worker pool for Node.js and added Web Worker execution logic for browsers.
- Enabled environment-aware threading availability in `environment.ts`.
- Updated `rollup.config.js` to include CLI configurations for streamlined builds.
- Adjusted `.gitignore` to exclude test artifacts and package files.
These changes provide comprehensive testing and documentation of threading functionality, improve cross-environment compatibility, and enhance developer workflows.
- Updated `npm` badge in `README.md` to reflect version `0.9.12`.
- Updated `VERSION` constant in `src/utils/version.ts` to `0.9.12`.
These changes ensure version consistency across documentation and codebase in preparation for release.
- Refactored function signature to include line breaks for enhanced readability and adherence to style guidelines.
- No functional changes introduced, ensuring consistent behavior.
These adjustments improve code clarity and maintain alignment with project formatting standards.
- Eliminated redundant line break to ensure consistent code formatting.
- Aligns the file with project coding standards for better readability and maintainability.
- Updated `findUSELoadFunction` return type for better type safety and clarity.
- Enhanced load function detection logic with improved readability and consistency.
- Applied formatting adjustments, such as line breaks and removal of redundant semicolons, to adhere to coding standards.
These changes ensure stronger type enforcement, align with project style guidelines, and improve maintainability of Universal Sentence Encoder initialization logic.
- Applied consistent formatting improvements, including line breaks, parentheses usage, and object destructuring, to enhance code readability and maintainability.
- Enhanced the fallback mechanism during Universal Sentence Encoder initialization by implementing a retry approach with error handling.
- Refactored `addBatch` processing for both vector and text items to improve clarity and adhere to project coding standards.
- Optimized initialization safeguards with structured retry implementations, ensuring robust error resiliency.
These changes align the codebase with established formatting guidelines and improve the reliability of embedding initialization processes.
- Updated `npm` shield in `README.md` to reflect the new version (v0.9.11).
- Incremented `VERSION` constant in `src/utils/version.ts` to align with the release update.
These changes ensure documentation and source files remain consistent with the latest release version.
