- Fixed missing setup.js issue by updating files field in package.json
- Changed from selective file inclusion to including all JS/TS files
- Excluded large framework bundles to keep package size reasonable
- Updated package size test thresholds to match new structure
- Package now correctly includes all necessary modules for installation
- Remove Rollup bundling in favor of direct TypeScript compilation
- Move from bundled models to dynamic model loading with configurable paths
- Add Docker deployment examples and documentation
- Implement robust model loader with fallback mechanisms
- Update storage adapters for better cross-environment compatibility
- Add comprehensive tests for model loading and package installation
- Simplify package.json scripts and remove complex build configurations
- Clean up deprecated demo files and old bundling scripts
BREAKING CHANGE: Models are no longer bundled with the package. They are now loaded dynamically from CDN or custom paths.
Add @soulcraft/brainy-models as optional dependency for zero-config offline reliability. Enhance robustModelLoader with hierarchical loading strategy (local → online → fail). Add comprehensive production deployment documentation and update README with clear benefits.
This solves critical production issues where Universal Sentence Encoder fails to load in Docker/Cloud Run environments due to network timeouts or blocked URLs. The solution provides 100% offline reliability while maintaining backward compatibility and requires no code changes from users.
- Add npm overrides to force form-data version 4.0.4 or higher
- Fixes GitHub security advisory GHSA-fjxv-7rqg-78g4
- Vulnerability was in transitive dependency via TensorFlow.js
- No functionality changes, all tests passing
- **Documentation Additions**:
- Created `brainy_architecture_diagram.md` to detail Brainy's architecture using diagrams and structured descriptions:
- Added overviews of the system, core architecture, and augmentation pipeline.
- Defined data models, graph structures, storage architecture, and performance optimizations.
- Explained vector search engine design, HNSW index structure, and usage flow examples.
- Developed `brainy_architecture_visual.md` to complement the architecture with visual aids in Mermaid.js:
- Provided detailed flowcharts, mind maps, and sequence diagrams for system components and data flow.
- **Purpose**:
- Provide in-depth technical insights into Brainy's architecture for developers and stakeholders.
- Enhance understanding of the system's core design principles with easy-to-follow diagrams and examples.
- Updated `package.json` in `web-service-package`, `cli-package`, and the root project to prefix internal NPM scripts with an underscore (`_`), such as `_version`, `_deploy`, `_dry-run`, etc.
- Adjusted all relevant build, versioning, deployment, and test scripts to follow this convention.
- Updated `.gitignore` to include `/brainy-models-package/node_modules/` for effective exclusion.
**Purpose**: Standardize the naming of internal scripts to better differentiate them from user-facing commands and maintain consistency across packages.
This change implements standard-version for versioning and changelog management:
- Adds standard-version as a dev dependency
- Updates package.json scripts to use standard-version
- Creates .versionrc.json configuration file
- Updates CHANGELOG.md with instructions for conventional commits
- Removes obsolete manual changelog update script
- Deleted `CHANGES.md`, `CHANGES_SUMMARY.md`, `CONCURRENCY_ANALYSIS.md`, `CONCURRENCY_IMPLEMENTATION_SUMMARY.md`, and related developer documentation files.
- Removed redundant or legacy content no longer aligned with the current codebase and workflows.
- Updated repository to reflect streamlined documentation approach, reducing clutter and improving maintainability.
**Purpose**: Simplify and declutter repository by removing obsolete documentation files, ensuring it remains focused and relevant.
- **Removed Files**:
- Deleted outdated statistics documentation files (`statistics.md`, `statistics-flush-solution.md`, `statistics-summary.md`) to clean up the repository and avoid confusion.
- **Added Standards**:
- Introduced `DOCUMENTATION_STANDARDS.md` to outline naming conventions and troubleshooting practices for more consistent and maintainable project documentation.
- **Tests**:
- Added a new test file `edge-cases.test.ts` to verify handling of edge cases, ensuring robust behavior against boundary values and invalid inputs.
**Purpose**: Cleans up deprecated documentation while introducing concrete standards for maintaining and updating documentation. Enhances test coverage for unusual or boundary inputs, improving overall system resilience.
- **Integration Tests**:
- Introduced `api-integration.test.ts` to validate API functionality:
- Verifies text insertion, vector embedding generation, and search operations.
- Confirms HNSW index correctness for vector similarity search.
- Ensures no dimensional mismatches in embeddings.
- **Test Server**:
- Added test server utilizing Express for endpoint simulation (`/insert` and `/search/text`).
- **Dependencies**:
- Introduced `express` and `node-fetch` as new dependencies for testing purposes.
- **Vitest Fix**:
- Updated `vitest.config.ts` to resolve the `process.memoryUsage` error by setting `logHeapUsage: false`.
- **Package Updates**:
- Modified `package-lock.json` to include newly added dependencies and updates.
**Purpose**: Guarantees the stability of core API endpoints and vector-related functionality, ensuring reliable behavior for end-to-end scenarios.
- **Version Updates**:
- Updated `cli-package/package.json` and `package-lock.json` from `0.15.0` to `0.16.0`.
- **NPM Script Enhancements**:
- Added version-related scripts (`version`, `version:patch`, `version:minor`, `version:major`) to `web-service-package` and `cli-package`.
- Introduced `deploy` and `dry-run` scripts to streamline build and deployment workflows.
- **Script Cleanup**:
- Removed redundant `build:cli`, `deploy:cli`, and `deploy:web-service` scripts from the root `package.json`.
**Purpose**: Prepares for release 0.16.0 while introducing reusable and optimized build/release scripts, improving the development and deployment experience.
- **Removal of `generate-version.js`**:
- Deleted `scripts/generate-version.js` as it is no longer required for version handling.
- Removed associated output files, including `src/utils/version.ts`.
- **Prebuild and Version Process Updates**:
- Updated `prebuild` and `version` scripts in `package.json` to reflect the new workflow:
- Replaced `generate-version.js` with simplified echo commands for version updates and build steps.
- Adjusted deployment scripts (`deploy:cli` and `deploy:web-service`) to align with the removal of the script.
- **Documentation and Config Updates**:
- Cleaned up `README.md` by removing unused version badge references.
- Updated `cli-package/package.json` to specify a version range (`^0.15.0`) for `@soulcraft/brainy`.
**Purpose**: This refactor simplifies the version handling process by removing outdated scripts, reducing maintenance overhead, and aligning workflows with project conventions. Improves clarity and consistency in both configuration and documentation.
- **New Web Service Package**:
- Introduced `@soulcraft/brainy-web-service`, a REST API wrapper for the Brainy vector graph database.
- Added documentation and features to support secure, read-only search and retrieval operations.
- **Deployment Support**:
- Included comprehensive deployment instructions in the `web-service-package/README.md`:
- Options for Docker, serverless platforms, and cloud providers (AWS, GCP, Azure, Cloudflare).
- Example configurations for systemd, Nginx, and Docker Compose.
- **Scripts and Documentation Updates**:
- Added `deploy:web-service` script to `package.json` for streamlined build and publishing.
- Enhanced `README.md` to reflect the introduction of the web service package and its capabilities.
- **Purpose**:
- This update extends Brainy’s functionality by providing a production-ready, easy-to-deploy REST API for search operations. It ensures flexibility for diverse deployment scenarios while maintaining security and high performance.
- **Core Improvements**:
- Refactored logging functions into a unified `logger` method for consistent output across the library.
- Enabled the `forceMemoryStorage` option in `BrainyData` initialization for improved storage flexibility in tests and specific use cases.
- **TensorFlow.js and Environment Updates**:
- Clarified the dependency structure in `README.md` to emphasize bundled dependencies and remove legacy peer dependency instructions.
- Simplified and reformatted environment detection logic for better maintainability and readability.
- **Testing Enhancements**:
- Added `tests/package-size-limit.test.ts` to monitor and validate npm package size against defined thresholds.
- Updated `tests/environment.node.test.ts` and core tests to leverage `forceMemoryStorage` for better test setup standardization.
- Improved test isolation with expanded `globalThis` utility definitions and cleanup logic.
- **Documentation**:
- Added detailed best practices for debugging and organizing tests in `DEVELOPERS.md`.
- Removed outdated installation hints from `package.json` and streamlined scripts by including `test:size` for package size validation.
**Purpose**: These changes unify core logging mechanisms, expand configurability of storage options, and improve testing reliability and coverage. Documentation and clarity are enhanced to align with updated functionality and best practices.
- **CLI Documentation Updates**:
- Updated `README.md` to reflect the separation of CLI functionality into a new package: `@soulcraft/brainy-cli`.
- Replaced inline CLI imports with instructions for installing and using `@soulcraft/brainy-cli`.
- Clarified notes on keeping the main package lightweight by decoupling CLI functionality.
- **Script and Command Adjustments**:
- Modified build instructions in `README.demo.md` to use `npm run build && npm run build:browser`.
- Updated demo scripts in `package.json` to point to `/demo/index.html` instead of `/index.html`.
- Revised style-check script in `CONTRIBUTING.md` to use `npm run check:style`.
- **Issues and Contribution Guidelines**:
- Improved issue reporting section in `DEVELOPERS.md` with clearer guidelines and examples for bug and feature requests.
**Purpose**: These changes enhance documentation accuracy, streamline developer guidance, and ensure commands/scripts are up-to-date with current usage patterns.
- **Added Dependencies**:
- Introduced `@vitest/coverage-v8` for enhanced test coverage reporting.
- Added various dev dependencies such as `@ampproject/remapping`, `@babel/parser`, `@babel/types`, and `ast-v8-to-istanbul` to support improved source mapping and code coverage analysis.
- Included modules such as `html-escaper`, `path-scurry`, and `istanbul-lib-coverage` to align with updated testing and utility requirements.
- **Purpose**:
- This update ensures compatibility with modern testing tools, facilitates advanced debugging, and supports enhanced test coverage capabilities. It also updates metadata for package resolutions.
- **Documentation Updates**:
- Simplified server functionality details in `README.md`, removing cloud-specific deployment sections and focusing on core features for broader usability.
- **Shim Enhancements**:
- Improved `util` shim in `rollup.config.js` to include `TextEncoder`/`TextDecoder` support and enhanced `types` checks (e.g., `isFloat32Array`, `isUint8Array`).
- **Dependency Updates**:
- Added new dev dependencies (`@vitest/ui`, `jsdom`, `happy-dom`).
- Updated `package-lock.json` with additional dependency resolutions and metadata for better compatibility and maintainability.
This commit aligns documentation with current project focus, enhances cross-platform support via refined shims, and ensures updated dependencies for development and testing environments.
