- **New Web Service Package**:
- Introduced `@soulcraft/brainy-web-service`, a REST API wrapper for the Brainy vector graph database.
- Added documentation and features to support secure, read-only search and retrieval operations.
- **Deployment Support**:
- Included comprehensive deployment instructions in the `web-service-package/README.md`:
- Options for Docker, serverless platforms, and cloud providers (AWS, GCP, Azure, Cloudflare).
- Example configurations for systemd, Nginx, and Docker Compose.
- **Scripts and Documentation Updates**:
- Added `deploy:web-service` script to `package.json` for streamlined build and publishing.
- Enhanced `README.md` to reflect the introduction of the web service package and its capabilities.
- **Purpose**:
- This update extends Brainy’s functionality by providing a production-ready, easy-to-deploy REST API for search operations. It ensures flexibility for diverse deployment scenarios while maintaining security and high performance.
- **Logging Cleanup**:
- Removed redundant `console.log` statements in `brainyData.ts` and `embedding.ts` related to Universal Sentence Encoder initialization and load function detection.
- Replaced detailed logs with concise comments to streamline debugging and reduce noisy outputs.
- **Purpose**:
- Improves code readability, reduces runtime logging noise, and aligns logging verbosity with the project's streamlined debugging practices.
- **Core Improvements**:
- Refactored logging functions into a unified `logger` method for consistent output across the library.
- Enabled the `forceMemoryStorage` option in `BrainyData` initialization for improved storage flexibility in tests and specific use cases.
- **TensorFlow.js and Environment Updates**:
- Clarified the dependency structure in `README.md` to emphasize bundled dependencies and remove legacy peer dependency instructions.
- Simplified and reformatted environment detection logic for better maintainability and readability.
- **Testing Enhancements**:
- Added `tests/package-size-limit.test.ts` to monitor and validate npm package size against defined thresholds.
- Updated `tests/environment.node.test.ts` and core tests to leverage `forceMemoryStorage` for better test setup standardization.
- Improved test isolation with expanded `globalThis` utility definitions and cleanup logic.
- **Documentation**:
- Added detailed best practices for debugging and organizing tests in `DEVELOPERS.md`.
- Removed outdated installation hints from `package.json` and streamlined scripts by including `test:size` for package size validation.
**Purpose**: These changes unify core logging mechanisms, expand configurability of storage options, and improve testing reliability and coverage. Documentation and clarity are enhanced to align with updated functionality and best practices.
- **CLI Documentation Updates**:
- Updated `README.md` to reflect the separation of CLI functionality into a new package: `@soulcraft/brainy-cli`.
- Replaced inline CLI imports with instructions for installing and using `@soulcraft/brainy-cli`.
- Clarified notes on keeping the main package lightweight by decoupling CLI functionality.
- **Script and Command Adjustments**:
- Modified build instructions in `README.demo.md` to use `npm run build && npm run build:browser`.
- Updated demo scripts in `package.json` to point to `/demo/index.html` instead of `/index.html`.
- Revised style-check script in `CONTRIBUTING.md` to use `npm run check:style`.
- **Issues and Contribution Guidelines**:
- Improved issue reporting section in `DEVELOPERS.md` with clearer guidelines and examples for bug and feature requests.
**Purpose**: These changes enhance documentation accuracy, streamline developer guidance, and ensure commands/scripts are up-to-date with current usage patterns.
- Updated `version` of `@soulcraft/brainy` from `0.11.0` to `0.12.0` in `cli-package/package-lock.json`.
- Adjusted resolved package metadata (e.g., `integrity` values) to match the new release version.
**Purpose**: Ensures `package-lock.json` reflects the updated dependency version for consistency with the 0.12.0 release.
- Updated the version in `README.md`, `package.json`, and `version.ts` to `0.12.0` to reflect the new release.
- Adjusted dependency reference for `@soulcraft/brainy` in `cli-package/package.json` to use version `0.12.0`.
**Purpose**: Prepares for the 0.12.0 release, ensuring all version references are consistent across documentation and configuration files.
- **Vector Handling Updates**:
- Added a `dimensions` property to `BrainyDataConfig` for specifying vector dimensions.
- Introduced validation for vector dimensions during database creation and insertion to ensure consistency.
- Enhanced error handling and logging for dimension mismatches.
- **Model Loading Improvements**:
- Implemented retry logic for Universal Sentence Encoder model loading to handle network instability and JSON parsing errors gracefully.
- Improved logging and debugging support for failures during model initialization and embedding operations.
- **Compatibility Enhancements**:
- Updated polyfills to support TensorFlow.js compatibility across diverse server environments (Node.js, serverless, etc.).
- Introduced and refactored global `TextEncoder`/`TextDecoder` definitions for seamless operation in non-browser environments.
- Simplified TensorFlow.js backend setup with streamlined imports and logging for GPU/WebGL fallback.
- **Purpose**:
- These updates improve BrainyData's robustness, enforce correct vector usage, and extend compatibility with varied runtime environments. The changes enhance the usability, reliability, and cross-platform readiness of core functionalities.
- **Test Enhancements**:
- Refactored test setup in `core.test.ts` for better isolation and clarity:
- Added explicit `.clear()` calls to ensure clean state between tests.
- Replaced old vector addition logic with simplified data insertion methods.
- Updated search operations to reflect current functionality and debug-friendly output.
- Introduced the `@vitest-environment jsdom` annotation in `environment.browser.test.ts` for accurate browser environment emulation.
- **Configuration Updates**:
- Enhanced `vitest.config.ts`:
- Introduced custom `reporters` for cleaner and focused test result presentation.
- Expanded console log filtering with additional patterns for reducing noise from TensorFlow.js and setup processes.
- **Purpose**:
- These updates improve test clarity, consistency, and robustness while streamlining the configuration to minimize distractions in test outputs.
- **Configuration Updates**:
- Set default test environment to `node` for consistency.
- Reduced noise in test outputs by configuring options such as `silent`, `bail`, `hideSkippedTests`, `printConsoleTrace`, and `onConsoleLog`.
- Disabled test coverage reports by default for reduced verbosity.
- Added functionality to filter out noisy console logs more aggressively.
- **Purpose**:
- Improves the clarity and readability of test output, reduces unnecessary verbosity, and ensures consistent behavior in a `node` environment. Encourages focus on debugging failed tests by hiding irrelevant details.
- **Added Dependencies**:
- Introduced `@vitest/coverage-v8` for enhanced test coverage reporting.
- Added various dev dependencies such as `@ampproject/remapping`, `@babel/parser`, `@babel/types`, and `ast-v8-to-istanbul` to support improved source mapping and code coverage analysis.
- Included modules such as `html-escaper`, `path-scurry`, and `istanbul-lib-coverage` to align with updated testing and utility requirements.
- **Purpose**:
- This update ensures compatibility with modern testing tools, facilitates advanced debugging, and supports enhanced test coverage capabilities. It also updates metadata for package resolutions.
- **Test Improvements**:
- Introduced data-clearing steps (`.clear()`) across critical test cases for ensuring better test isolation and preventing state leakage.
- Extended support for overriding global utilities (`testUtils`) and added fallback behaviors for test vector creation.
- **Configuration Updates**:
- Added support for `distanceFunction` as an alternative to `metric` in vector operations for consistency.
- Adjusted and unified asynchronous `timeout` handling across test suites for predictability.
- **Purpose**:
- These updates improve reliability, maintainability, and clarity in test cases while ensuring compatibility across diverse test environments.
- **New Tests Added**:
- Introduced multiple test suites covering core functionalities (`core.test.ts`), vector operations (`vector-operations.test.ts`), Node.js environment (`environment.node.test.ts`), browser setup (`environment.browser.test.ts`), and TensorFlow.js-specific behaviors (`tensorflow-patch.test.ts`).
- Added performance, scalability, and error-handling tests to ensure robust validation of vector addition, search, and text embedding functionalities.
- Introduced setup utilities (`tests/setup.ts`) and standardized test utilities for creating predictable test cases.
- **Configuration**:
- Created `vitest.config.ts` for custom test configurations, including support for modern test environments (`jsdom`, `happy-dom`) and extended timeouts for asynchronous operations.
- **Validation**:
- Includes compatibility checks for TensorFlow.js imports and ensures proper handling of `TextEncoder`/`TextDecoder` in Node.js environments.
This commit significantly enhances the testing coverage and structure, ensuring Brainy functionality is robust, cross-platform, and aligned with evolving reliability standards.
- Removed extraneous semicolons across `cli-wrapper-example.js` for cleaner, consistent code style.
- Adjusted indentation and spacing for improved readability.
- Updated console statements to be consistent, ensuring uniform logging behavior across the example.
This change aligns the example file with the project's coding standards, improving code clarity and maintainability.
- **Documentation Updates**:
- Simplified server functionality details in `README.md`, removing cloud-specific deployment sections and focusing on core features for broader usability.
- **Shim Enhancements**:
- Improved `util` shim in `rollup.config.js` to include `TextEncoder`/`TextDecoder` support and enhanced `types` checks (e.g., `isFloat32Array`, `isUint8Array`).
- **Dependency Updates**:
- Added new dev dependencies (`@vitest/ui`, `jsdom`, `happy-dom`).
- Updated `package-lock.json` with additional dependency resolutions and metadata for better compatibility and maintainability.
This commit aligns documentation with current project focus, enhances cross-platform support via refined shims, and ensures updated dependencies for development and testing environments.
- Removed the `test-tensorflow-textencoder` command from `cli.ts` as part of deprecating redundant tests and unused CLI functionality.
- Simplified `TextEncoder` and `TextDecoder` definitions in `brainy-wrapper.js` by replacing direct constructor usage with shorthand properties within a utility object.
This refactor reduces unnecessary complexity in CLI commands and improves maintainability by aligning with current project goals and streamlined test structure.
- Simplified explanation of server functionality (`WebSocket` and `REST`) to emphasize its exclusion from the main Brainy package for maintaining lightweight browser bundles.
- Removed detailed Node.js-specific setup examples, configuration options, and cloud-wrapper deployment instructions.
- Retained relevant core functionality usage details for broader accessibility.
This update aligns the documentation with recent repository changes, focusing on clarity, maintainability, and providing guidance only on actively supported features.
- Removed the `scripts/comprehensive-test.js` script and its auxiliary files (`node-test.js`, `TENSORFLOW_NODEJS.md`, and `scripts/test-cli-locally.js`).
- These files encompassed unified testing across environments (Browser, Node.js, and CLI) and TensorFlow.js compatibility.
- Reflects a strategic decision to deprecate integrated testing workflow in favor of modular testing approaches and project simplification.
This cleanup aligns with the ongoing effort to streamline the repository and remove redundant or outdated test infrastructure.
- Removed entire `cloud-wrapper` directory and its contents, including:
- Obsolete deployment scripts: `deploy-aws.js`, `deploy-cloudflare.js`, `deploy-gcp.js`.
- Related service implementations: `brainyService.ts`, `mcpService.ts`, and `index.ts`.
- Legacy health-check endpoints, WebSocket handlers, and MCP service initialization.
- The removed code handled cloud-specific deployment and configuration for Brainy services, which is no longer relevant.
This cleanup removes unsupported and outdated code, improving repository maintainability and aligning it with current project goals.
- Removed obsolete scripts: `test-all-environments.js`, `test-fallback-function.js`, `test-fallback-simple.js`, `test-fix.js`, `test-tensorflow-textencoder.js`, `test-unified-encoding.js`, and `test-worker-utils.js`.
- Introduced `scripts/comprehensive-test.js` as a unified testing script covering all environments: Browser, Node.js, and CLI.
- Added `examples/cli-wrapper-example.js` to demonstrate a proper CLI implementation with TensorFlow.js initialization.
This refactor simplifies the testing structure by consolidating redundant scripts into a single comprehensive script while ensuring robust cross-environment coverage.
- Updated `@soulcraft/brainy` dependency version to `0.11.0` in `cli-package/package-lock.json`.
- Adjusted resolved version, integrity hash, and related metadata for `@soulcraft/brainy` to reflect the `0.11.0` release.
- Synced CLI package lock file version with the latest release for consistency and reliability.
This ensures alignment with the updated dependency versions across the project.
- Updated version to `0.11.0` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.11.0` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
This release ensures consistent versioning across the project and prepares for the distribution of changes in version `0.11.0`.
- Refactored `applyTensorFlowPatch` in `textEncoding.ts` to introduce a unified `Platform` class supporting both Node.js and browser environments with native `TextEncoder`/`TextDecoder`.
- Modified global object handling to target `PlatformNode` in Node.js and `PlatformBrowser` in browser environments.
- Added `getTextEncoder` and `getTextDecoder` utility functions to simplify text encoding/decoding across platforms.
- Introduced `setup.ts` to apply environment patches before other modules load.
- Updated CLI entry points (`cli.ts`, `rollup.config.js`) to ensure patching precedes TensorFlow.js usage.
- Enhanced test coverage for TextEncoder compatibility with `test-fix.js`.
- Adjusted imports in `test-tensorflow-textencoder.ts` to align with the updated `./utils/textEncoding.js`.
- Streamlined constructor definitions in `Platform` for improved maintainability.
This update ensures robust compatibility for TensorFlow.js by patching and standardizing text encoding/decoding functionality across environments.
- Introduced `test-fallback-function.js` and `test-fallback-simple.js` to validate `executeInThread` fallback functionality with both named and anonymous compute-intensive functions.
- Added `test-tensorflow-textencoder.js` for TensorFlow.js and TextEncoder tests in a Node.js environment.
- Created `test-tensorflow-textencoder.html` for browser-based TensorFlow.js and TextEncoder tests.
- Implemented cross-environment test support in `cli-package/src/test-tensorflow-textencoder.ts` for CLI functionality.
- Enhanced `src/utils/embedding.ts`, `textEncoding.ts`, and `brainy-wrapper.js` to include updated global `TextEncoder` and `TextDecoder` utilities for compatibility and worker improvements.
- Standardized and expanded utility methods in `PlatformNode` for broader support, including `isFloat32Array` and `isTypedArray` checks.
- Updated Node.js requirement to `>= 24.4.0` across documentation and configuration files for compatibility improvements.
This update introduces comprehensive testing for fallback mechanisms, TensorFlow.js, and TextEncoder across multiple environments, ensuring robustness and compatibility.
- Updated version to `0.10.0` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Removed `CHANGELOG.md` from the root and `cli-package`, as GitHub auto-generates release notes.
- Refactored `brainy-wrapper.js` to simplify utility methods (`isFloat32Array` and `isTypedArray`) by moving them to instance-level definitions.
- Standardized Node.js version requirement to `>= 24.3.0` across documentation and configuration files.
- Improved `textEncoding.ts` by introducing a unified `Platform` class for cross-environment compatibility (Node.js and browser).
- Restructured scripts:
- Replaced outdated commands (e.g., `test-cli` -> `test:cli` and `test-all` -> `test:all`) for consistency.
- Removed changelog updates from `scripts/create-github-release.js` as part of simplification.
- Enhanced `patch-textencoder.js` to also patch `worker.js` for both `TextEncoder` and `TextDecoder`.
This release improves compatibility with Node.js 24, simplifies the deployment/changelog process, and refines text encoding and worker functionalities.
- Updated version to `0.9.37` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.37` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
- Added release details for version `0.9.37` in `CHANGELOG.md`.
This update ensures consistent versioning across the project and documents the latest release.
- Added `cli-package/brainy-wrapper.js` to patch global `TextEncoder` and `TextDecoder` for Node.js environments, ensuring compatibility with TensorFlow.js.
- Implemented unified text encoding utilities in `cli-package/src/utils/textEncoding.ts` for cross-environment consistency.
- Enhanced `src/worker.js` and introduced `src/worker.ts` to improve worker execution with better function serialization and error handling.
- Updated `package.json`:
- Modified the `build` script to include a patch for `TextEncoder`.
- Added `test-all` script for multi-environment testing.
- Introduced the Puppeteer dependency for browser testing.
- Created a favicon generation script `scripts/create-favicon.js` using a base64-encoded icon.
- Added `scripts/test-all-environments.js` to run automated tests across Node.js, browser, and CLI environments.
- Enhanced `src/utils/workerUtils.ts` to support improved fallback mechanisms and robust worker pooling.
This update improves the project's compatibility across environments, refines worker functionalities, and adds comprehensive testing support for reliability.
- Updated `package.json` to modify the `deploy:cli` script by removing the call to `scripts/create-github-release.js`.
- Simplified the deployment process for the CLI package by decoupling GitHub release creation.
This change aims to streamline the CLI deployment workflow, allowing more flexibility in managing GitHub releases.
- Updated version to `0.9.36` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.36` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
- Added release details for version `0.9.36` in `CHANGELOG.md`.
This update ensures consistent versioning across the project and documents the latest release.
- Updated version to `0.9.35` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.35` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
- Enhanced GitHub release script (`scripts/create-github-release.js`) to check and push tags to remote if they exist locally.
This update ensures consistent versioning across the project and improves release automation with better tag management.
- Added functionality to auto-generate GitHub releases:
- Included `scripts/create-github-release.js` to automate release creation with notes.
- Modified `scripts/publish-cli.js` to create GitHub releases after publishing packages.
- Updated `package.json` to call the GitHub release script during `deploy` and `deploy:cli` commands.
- Introduced `CHANGELOG.md` for both main and CLI packages:
- Enabled auto-updating of CHANGELOG.md from GitHub release notes.
- Added a new `scripts/update-changelog.js` script to synchronize release details with changelogs.
- Ensured `CHANGELOG.md` files are included in the published packages.
- Improved documentation visibility by displaying release notes on npmjs.com.
This update streamlines the release process, ensuring consistent documentation, and aligns npm releases with corresponding GitHub releases for better transparency.
- Updated Node.js version requirement to `>=24.3.0` in `README.md` and `package-lock.json`.
- Upgraded `TypeScript` to version `5.4.5` and synced references.
- Updated multiple dependencies, including:
- `@aws-sdk/client-s3` to `^3.540.0`.
- `uuid` to `^9.0.1`.
- Development tools such as `eslint` to `^8.57.0`, `typescript-eslint` plugins to `^7.4.0`, and more.
- Refreshed transitive dependencies with up-to-date versions for consistency and performance.
This update ensures compatibility with the latest Node.js features, aligns TypeScript references, and keeps the project dependencies up-to-date for improved stability and functionality.
- Updated version to `0.9.34` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.34` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
This update ensures consistent versioning across the project and aligns dependencies with the latest release.
- Updated version to `0.9.33` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.33` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
- Refactored `TextEncoder` and `TextDecoder` implementations in `src/utils/tensorflowUtils.ts`:
- Added dynamic environment-aware loading for improved Node.js and browser compatibility.
- Enhanced fallback mechanisms for environments without global support.
This update ensures consistent versioning across the project and improves the robustness of utility methods for broader environment compatibility.
- Updated version to `0.9.32` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.32` in the CLI package for consistency.
- Renamed npm scripts `publish` to `deploy` and `publish:cli` to `deploy:cli`, aligning terminology with deployment workflows.
- Updated npm version badge in `README.md` to reflect the new version.
This update ensures consistent versioning across the project, aligns CLI and main package references, and enhances clarity in deployment script naming.
- Updated version to `0.9.31` in `README.md`, `cli-package/package.json`, and `src/utils/version.ts`.
- Synced `@soulcraft/brainy` dependency version to `0.9.31` in the CLI package for consistency.
- Updated npm version badge in `README.md` to reflect the new version.
This update ensures consistent versioning across the project and aligns references between main and CLI packages.
- Refactored `TextEncoder` and `TextDecoder` in `src/utils/tensorflowUtils.ts` to handle environments without global availability, logging warnings and providing fallback implementations.
- Enhanced threading utilities in `src/utils/environment.ts`:
- Introduced `areWorkerThreadsAvailableSync()` and `isThreadingAvailableAsync()` for synchronous and asynchronous checks.
- Improved compatibility across browser and Node.js environments.
- Migrated `fileSystemStorage.ts` to use dynamic imports for Node.js modules, removing synchronous loading mechanisms to align with ES module standards.
- Updated `cli-package/package.json` keywords to include `browser` and `container`, improving discoverability.
- Added `worker_threads` as an external dependency in `cli-package/rollup.config.js` and `rollup.config.js`.
- Updated `.gitignore` to ignore all `tgz` files dynamically under main and CLI package.
This update improves threading compatibility, enhances environment-specific support, and aligns with modern module loading practices.
