### Changes:
- **`demo/simplified-augmentations-gh-pages.js`**:
- Added a new demo script showcasing simplified augmentations for GitHub Pages.
- Includes examples for static and streaming data processing, WebSocket support, and dynamic augmentation loading.
- **`README.md`**:
- Updated the demo link to point directly to `https://soulcraft-research.github.io/brainy/`.
- **`.github/workflows/deploy-demo.yml`**:
- Enhanced the deployment workflow to include the `simplified-augmentations-gh-pages.js` file as the GitHub Pages demo entry point.
- Added steps to copy and prepare required `dist` files during the build process.
- **`demo/index.html`**:
- Dynamically adjusted the import path for the Brainy library to support both local and CDN environments.
- Reformatted code for readability and alignment with deployment needs.
### Purpose:
Introduced a tailored demonstration script for GitHub Pages to highlight Brainy's streamlined augmentation features. Enhanced the deployment pipeline for better integration and seamless live demo experience.
### Changes:
- **README.md**:
- Updated `Brainy Logo` image URL to reference a specific commit for consistent display.
- Simplified the description by removing "lightweight" for a more concise statement.
- **package.json**:
- Added `brainy.png` to the `files` array to include it in published packages.
- **src/utils/version.ts**:
- Updated `VERSION` constant from `0.9.2` to `0.9.3`.
### Purpose:
Aligned the documentation with a specific asset version for consistent display. Improved package completeness by including the project logo. Incremented the version to `0.9.3` to reflect the latest changes.
### Changes:
- **src/cli.ts**:
- Reformatted multi-line logging statements using consistent indentation for better readability.
- Reformatted `import-sparse` command description for consistent line wrapping.
- Adjusted function definitions and options configurations for clarity.
- Improved inline comments (`examples` → `demo`) for accurate context.
### Purpose:
Enhanced code readability and consistency by reformatting logging, descriptions, and comments. Improved maintainability through structured formatting and updated context-specific references.
### Changes:
- **README.md**:
- Replaced local image path for `Brainy Logo` with a GitHub raw URL to ensure accessibility in remote contexts.
- Updated links to reference `demo` folder structure (`examples/simplified-augmentations.js` → `demo/simplified-augmentations.js`, `examples/demo.html` → `demo/index.html`).
### Purpose:
Enhanced documentation by ensuring that assets and links are accessible remotely and aligned with the recent folder structure updates. Improved clarity and consistency with the updated demo setup.
### Changes:
- **simplified-augmentations.js**:
- Updated comments and console logs to replace "examples" with "demo" for consistent terminology.
- Adjusted function description and logging messages to align with the recent folder structure changes.
- **Renamed**:
- `examples/simplified-augmentations.js` → `demo/simplified-augmentations.js`.
- `examples/demo.html` → `demo/index.html`.
### Purpose:
Revised terminology and logging outputs to reflect the updated folder structure and naming conventions. Improved clarity and consistency across file comments and logs.
### Changes:
- **mcpService.ts**:
- Updated `type` in `/mcp/tools` route to use `MCPRequestType.SYSTEM_INFO` enum instead of hardcoded string `'SYSTEM_INFO'`.
- Removed redundant `infoType: 'availableTools'` from the MCP request payload.
- Added `MCPRequestType` import from `@soulcraft/brainy`.
### Purpose:
Replaced hardcoded request type string with an enum for improved type safety and maintainability. Removed unnecessary `infoType` parameter, aligning the request payload with updated requirements.
### Changes:
- **rollup.config.js**:
- Merged and reformatted `nodeBuiltins` definition for cleaner code.
- Updated `input` path from `examples/browser_compatible_exports.ts` to `demo/browser_compatible_exports.ts`.
- Reformatted `replace` plugin configuration for better readability.
- **.github/workflows/deploy-demo.yml**:
- Updated deployment folder from `examples` to `demo`.
- **Renamed**:
- `examples/browser_compatible_exports.ts` → `demo/browser_compatible_exports.ts`.
- **scripts/generate-version.js**:
- Removed unnecessary semicolons for consistent formatting.
### Purpose:
Aligned folder structure by migrating from `examples` to `demo` for uniformity and better organization. Improved code readability and formatting across configurations and scripts.
### Changes:
- **tsconfig.browser.json**:
- Replaced `examples/**/*` with `demo/**/*` in the `include` paths.
- **src/utils/version.ts**:
- Updated `VERSION` constant from `0.9.1` to `0.9.2`.
### Purpose:
Aligned the TypeScript configuration with the recent folder structure update for better organization. Incremented the version to `0.9.2` to reflect the latest improvements.
### Changes:
- **package.json** & **package-lock.json**:
- Upgraded `@soulcraft/brainy` dependency from `^0.7.5` to `^0.9.2`.
- Updated dependencies within `@soulcraft/brainy`, including:
- Added `@tensorflow/tfjs-converter` and `buffer` as new dependencies.
- Adjusted peer dependencies for `@tensorflow/tfjs-core`.
- Included `base64-js` and `ieee754` packages.
- **package.json**:
- Refined `demo` script by replacing `demo.html` with `index.html` under `/demo/`.
### Purpose:
Upgraded `@soulcraft/brainy` to the latest version, ensuring compatibility with the new features and performance enhancements. Adjusted related dependencies to align with the updated version. Improved demo path structure for better organization.
### Changes:
- **examples/index.html**:
- Deleted the redirect HTML file that directed users to `demo.html`.
### Purpose:
Cleaned up unused redirect page as part of
### Changes:
- **README.md**:
- Reformatted "Live Demo" and "Demo" sections for improved readability.
- Adjusted line wrapping to enhance clarity and maintain consistent formatting.
- **.npmignore**:
- Allowed inclusion of `brainy.png` while preserving exclusions for other `.png` files.
- **src/utils/version.ts**:
- Bumped version from `0.9.0` to `0.9.1`.
### Purpose:
Enhanced documentation formatting for better user experience and easier navigation. Updated `.npmignore` to refine file inclusion for package distribution. Incremented version to `0.9.1` to reflect the latest changes.
### Changes:
- **verify-package-size.js**:
- Added a CLI script to calculate and display the estimated package size.
- Integrated support for parsing `.npmignore` and `package.json` `files` patterns.
- Outputs total package size and lists the largest files included.
- **.npmignore**:
- Introduced an `.npmignore` file to exclude unnecessary development, configuration, and large files from the package.
- Added patterns for source maps, development files, configuration files, and artifacts.
- **package.json**:
- Updated the `files` array to include new patterns for distribution files (`.js`, `.d.ts`, `types`).
- Improved package inclusion accuracy with precise file patterns.
### Purpose:
Introduced a new utility to provide quick insights into the package size for better maintenance and optimization. Optimized `.npmignore` and `files` configurations in `package.json` to exclude irrelevant files, ensuring a smaller, cleaner package distribution.
### Changes:
- **examples/index.html**:
- Introduced a new HTML file that redirects users to `demo.html` using both meta tag and JavaScript.
- **README.md**:
- Updated live demo link to point to the new `examples/` directory and clarified access to both the root directory and `demo.html`.
### Purpose:
Improved user experience by creating a smoother navigation to the demo page and updated documentation to reflect the new entry point of the live demo. Removed redundant `CNAME` file as part of cleanup.
### Changes:
- **deploy-demo.yml**:
- Added `contents: write` permissions to ensure the workflow can access repository content during operations.
- Used `${{ secrets.GITHUB_TOKEN }}` for authentication in the deployment step.
- **package.json**:
- Removed the `deploy:demo` script and its `gh-pages` dependency to clean up unused scripts and dependencies.
### Purpose:
Enhanced the deploy workflow by explicitly defining required permissions and securely managing authentication. Simplified the configuration by removing outdated scripts and dependencies.
### Changes:
- **deploy-demo.yml**: Removed an unnecessary blank line in the dependency installation step.
### Purpose:
Refined the workflow file to improve readability and maintain consistency in formatting. This is a minor housekeeping change with no impact on functionality.
### Changes:
- **README.md**: Added a "Live Demo" section with a direct link to the interactive GitHub Pages demo.
- **deploy-demo.yml**: Modified dependency installation step to use `npm install --legacy-peer-deps` for compatibility.
### Purpose:
Improved documentation by providing direct access to the live demo for better user engagement. Updated the deploy workflow to ensure successful dependency resolution during the setup process.
### Changes:
- Introduced `deploy-demo.yml` workflow in `.github/workflows/` to automate deployment to GitHub Pages:
- Triggers on `main` branch pushes and manual workflow dispatch.
- Includes steps for project setup, build, and deployment using `JamesIves/github-pages-deploy-action`.
- Added `examples/CNAME` with custom domain `www.soulcraft.com`.
### Purpose:
Streamlined and automated the deployment of the demo to GitHub Pages, ensuring efficient updates. Configured a custom domain for better branding and accessibility of the live demo.
### Changes:
- Enhanced deployment section in `README.md`:
- Added instructions for automatic deployment using the included GitHub Actions workflow (`.github/workflows/deploy-demo.yml`).
- Documented steps for configuring a custom domain:
- Pointing CNAME or A records to GitHub Pages.
- Adding a domain via repository settings.
### Purpose:
Improved documentation to guide users in automating GitHub Pages demo deployment and configuring custom domains, simplifying setup and enhancing accessibility for personalized deployments.
### Changes:
- Removed `github-pages.yml` GitHub Actions workflow file.
- Updated `README.md`:
- Added documentation for `npm run deploy:demo` to deploy the demo to GitHub Pages.
- Updated the live demo link to reflect the repository's namespace change.
- Added `gh-pages` dependency (v6.3.0) in `package.json`.
- Introduced `deploy:demo` script in `package.json` for demo deployment.
### Purpose:
Streamlined the demo deployment process by removing the dedicated GitHub Pages workflow and replacing it with a manual deployment script (`npm run deploy:demo`). This enhances developer control while maintaining simplicity in deploying updates to the live demo.
### Changes:
- **github-pages.yml**:
- Downgraded `actions/upload-pages-artifact` from `v3` to `v2`.
### Purpose:
Reverted to `v2` of the `upload-pages-artifact` action to maintain compatibility with existing workflows or address potential issues introduced in `v3`.
### Changes:
- Reformatted `README.md` for enhanced readability:
- Applied line wrapping to long paragraphs and bullet points.
- Adjusted spacing and indentation for consistent formatting.
- Resolved issues with inconsistent code snippet alignments and metadata formatting.
### Purpose:
Improved the readability and maintainability of the README by standardizing formatting and wrapping long lines. These changes ensure better clarity for users exploring the documentation.
### Changes:
- **github-pages.yml**:
- Updated `actions/upload-pages-artifact` from `v3` to `v4`.
### Purpose:
Ensured compatibility with the latest version of the `upload-pages-artifact` action to leverage improvements and maintain long-term support.
### Changes:
- **README.md**: Corrected the relative path for the live demo link to `examples/demo.html`.
- **github-pages.yml**:
- Updated GitHub Actions steps to their latest versions:
- `actions/checkout@v4`
- `actions/configure-pages@v4`
- `actions/upload-pages-artifact@v3`
- `actions/deploy-pages@v4`
- Ensured compatibility with the latest versions of the workflow actions.
### Purpose:
Fixed a broken link in the README for better user accessibility to the demo. Updated workflow actions in `github-pages.yml` to leverage the latest performances and features.
### Changes:
- Introduced a new `github-pages.yml` workflow in `.github/workflows/`.
- Configured the workflow to:
- Trigger on pushes to the `main` branch.
- Deploy files from the `./examples` directory to GitHub Pages.
- Utilize Actions such as `checkout`, `configure-pages`, `upload-pages-artifact`, and `deploy-pages`.
- Set up necessary permissions for deploying to GitHub Pages.
### Purpose:
Added an automated GitHub Actions workflow to streamline the deployment of content to GitHub Pages. This ensures a consistent and efficient process for updating the demo or documentation hosted online.
### Changes:
- Updated `README.md` to include a direct link to the interactive demo hosted on GitHub Pages.
- Improved demo instructions with clearer options for trying the live demo or running it locally.
- Refined descriptions of demo features such as environment compatibility, noun-verb model, and HNSW search.
### Purpose:
Provided a quick access link to the live demo for better discoverability and convenience. Enhanced clarity of instructions to guide users in exploring Brainy's capabilities effectively.
### Changes:
- **Favicon Update**: Replaced the relative favicon path with an absolute URL pointing to the GitHub-hosted file.
- **Header Redesign**: Improved header layout with a more modern and aligned flexbox structure. Adjusted logo size and added a left-aligned text section.
- **Styling Enhancements**:
- Introduced new styles for zoom controls, Google-like search results, and result clusters.
- Updated padding, alignment, and hover states for buttons and interactive elements.
- Refined visual hierarchy in sections with updated margins, sizes, and typography.
- **Added `What Can You Build?` Section**: Highlighted potential use cases for Brainy (e.g., semantic search, recommendation systems, adaptive experiences).
- **Improved Demo Steps**:
- Refactored interactive demo into guided "steps" for better usability.
- Added tooltips to clarify button functionality.
- Updated step titles and descriptions for clarity and conciseness.
- **Code Samples**:
- Simplified JavaScript import instructions for Node.js and browser environments.
- Highlighted CDN options for easier library access.
- **Advanced Features & New Controls**:
- Introduced new backup, restore, and additional operation buttons for database management.
- Added zoom-in, zoom-out, and zoom-to-fit controls for visualizations.
- Enhanced semantic search with dynamic filtering and guidance for queries.
### Purpose:
Updated the interactive demo to improve its usability, visual appeal, and feature coverage. These enhancements make the demo more user-friendly and better illustrate Brainy's capabilities, aiding developers in exploring and understanding the platform.
### Changes:
- Introduced two new relationship types in `VerbType`:
- `Follows`: Indicates a following relationship.
- `Likes`: Indicates a liking relationship.
### Purpose:
Enhanced the `VerbType` to support additional relationship semantics, expanding its applicability for modeling complex graph structures and fostering richer data representation.
### Changes:
- Introduced a `findSimilar` method in `src/brainyData.ts`:
- Enables searching for entities similar to a given entity ID.
- Supports options such as result limit, noun type filtering, verb inclusion, and search mode (local, remote, or combined).
- Filters out the original entity in results and returns limited output based on the specified options.
- Updated `brainyDataInterface` in `src/types/brainyDataInterface.ts` to include the new `findSimilar` method definition.
### Purpose:
Added the `findSimilar` method to enhance entity retrieval capabilities by enabling similarity-based searches. This feature improves data exploration and provides a flexible API for building advanced search and recommendation functionalities.
### Changes:
- Enhanced README with new feature highlights:
- Added the "Model Control Protocol (MCP)" feature to emphasize external AI model integration.
- Expanded "What Can You Build?" section with model-integrated systems use-case examples.
- Documented the new simplified augmentation system in detail:
- Provided examples for creating augmentations, adding WebSocket support, processing static data, and building reusable pipelines.
- Introduced a streamlined API for managing static and streaming data workflows, augmentations, and dynamic loading.
- Unified terminology by replacing `augmentationPipeline` references with `pipeline`.
### Purpose:
Updated the README to align with recent advancements in the augmentation system and provide users with actionable, hands-on guidance. These changes improve clarity, usability, and awareness of new framework capabilities such as MCP integration and the refined augmentation system.
### Changes:
- Introduced a new `examples/simplified-augmentations.js` file showcasing comprehensive usage examples:
- Example 1: Creating and using a simple memory augmentation.
- Example 2: Adding WebSocket support to a conduit augmentation.
- Example 3: Processing static data through a reusable pipeline.
- Example 4: Setting up and executing streaming data pipelines.
- Example 5: Dynamically loading augmentations from a module.
- Demonstrated augmentation factories, streamlined pipelines, dynamic loading, and reusable workflows through practical scenarios.
### Purpose:
Added detailed examples to assist developers in understanding and implementing augmentation factories and streamlined pipelines. These examples provide a hands-on reference for building static and streaming workflows, improving usability and accessibility for new and existing users.
### Changes:
- Updated `src/index.ts` exports to streamline pipeline and augmentation factory usage:
- Consolidated augmentation and streamlined pipeline exports under unified `Pipeline` module (e.g., `pipeline`, `createPipeline`, `executeStreamlined`).
- Added new augmentation factory exports:
- `createSenseAugmentation`, `addWebSocketSupport`, `executeAugmentation`, `loadAugmentationModule`.
- Enhanced backward compatibility with separate sequential pipeline exports.
- Introduced streamlined type exports, including `StreamlinedExecutionMode`, `StreamlinedPipelineOptions`, and `StreamlinedPipelineResult`.
### Purpose:
Standardized pipeline and augmentation factory exports for better modularity and backward compatibility. These changes improve developer experience by simplifying imports and aligning with the unified augmentation pipeline system.
### Changes:
- Added `Pipeline` class in `src/pipeline.ts` to unify primary and streamlined augmentation pipelines:
- Supports multiple execution modes: sequential, parallel, threaded, first success, and first result.
- Provides methods for registering, unregistering, and managing augmentations by type.
- Implements flexible execution options, including error handling, timeouts, and threading control.
- Introduced comprehensive augmentation registry for managing sense, dialog, memory, cognition, and WebSocket-based augmentations.
- Added global pipeline instance with legacy method support for backward compatibility.
- Enhanced augmentation execution capabilities:
- Support for static and streaming data processing.
- Pipeline creation for reusable augmentation workflows.
- Threaded execution for improved performance when supported.
- Added default pipeline integration with augmentation registry to resolve circular dependencies.
### Purpose:
Implemented a unified augmentation pipeline to streamline augmentation workflows, enhance flexibility, and simplify execution management. The changes improve performance, reduce complexity, and provide robust support for legacy methods.
### Changes:
- Updated the README to simplify descriptions and remove redundant details:
- Reduced verbosity in feature highlights, focusing on core functionality.
- Streamlined examples and usage sections for greater clarity.
- Simplified installation and quick start instructions.
- Removed overly casual or playful language to adopt a more professional tone.
- Consolidated streaming, environment detection, and build system explanations.
- Improved formatting consistency across sections:
- Aligned headings, examples, and lists for readability.
- Adjusted code indentation and structure in examples.
### Purpose:
Enhanced the README to provide a concise, focused, and professional presentation of Brainy. These changes improve readability, simplify onboarding for new users, and ensure consistency with recent terminology and documentation standards.
### Changes:
- Added a new `scalingStrategy.md` document explaining limitations and solutions for handling large-scale data with HNSW:
- Highlighted in-memory limitations of standard HNSW implementations.
- Detailed disk-based, distributed, and hybrid approaches for scaling to terabyte-scale datasets.
- Provided real-world examples from systems like Qdrant, Milvus, FAISS, and DiskANN.
- Emphasized trade-offs between memory usage, search accuracy, and performance when scaling.
### Purpose:
Documented approaches for managing large-scale data with HNSW to assist developers in implementing efficient, scalable solutions. The guide serves as a reference for overcoming memory limitations and adopting advanced techniques for handling terabyte-scale datasets.
### Changes:
- Introduced the `import-sparse` command in `src/cli.ts`:
- Allows importing sparse data from a JSON file into the database.
- Includes an option (`--clear`) to clear existing data before import.
- Displays counts of imported nouns and verbs upon successful completion.
- Updated CLI help and autocompletion configuration to support the new command.
### Purpose:
Enhanced the CLI by adding `import-sparse`, providing users with the ability to efficiently import sparse data into the database. This addition aligns with the application's functionality for managing sparse data operations.
### Changes:
- Introduced the `importSparseData` method in `src/brainyData.ts` to allow importing sparse data into the database.
- Supports embedding generation for nouns if vectors are missing.
- Includes options to clear existing data during import.
- Returns counts of imported nouns and verbs for tracking.
### Purpose:
Added a new method to manage sparse data imports, improving functionality and ensuring alignment with updated terminology across the application. This addition enhances flexibility in handling database operations.
### Changes:
- Introduced `initializeMCPService` in `cloud-wrapper/src/index.ts` to support initializing Model Control Protocol (MCP) services.
- Enabled configuration for WebSocket and REST APIs with customizable options like ports, authentication, rate limiting, and CORS.
- Exported MCP modules, types, and services (`BrainyMCPAdapter`, `BrainyMCPService`, etc.) in `src/index.ts`.
- Enhanced `unified.ts` to include global environment detection by populating `globalThis.__ENV__`.
- Added environment-specific logging for browser, Node.js, and serverless scenarios.
- Improved the structure of `environment` object and its global accessibility.
### Purpose:
Implemented MCP service integration to expand Brainy's external compatibility via WebSocket and REST interfaces. Enhanced environment detection provides better global access and debugging insights, ensuring robust operation across various runtime environments.
### Changes:
- Enhanced environment detection logic with a structured `environment` object (`isBrowser`, `isNode`, `isServerless`).
- Implemented fallback options for various environments:
- Added priority handling for storage types (e.g., MemoryStorage, OPFSStorage, FileSystemStorage).
- Improved dynamic imports for `s3CompatibleStorage.js` and `fileSystemStorage.js`.
- Unified and expanded cloud storage support by merging environment variables with provided options:
- Refined logic for R2, S3, and GCS storage configurations.
- Added detailed logging during storage type selection (`console.log` for selected storage).
- Introduced better error handling with warnings and defaults for unsupported environments.
- Removed redundant checks and reorganized storage selection logic for clarity.
### Purpose:
Streamlined environment detection and adaptive storage logic to ensure seamless operation across different platforms (browser, Node.js, serverless). These changes improve maintainability, enhance debugging, and simplify storage configuration while providing robust fallback mechanisms.
### Changes:
- Updated button label in `examples/demo.html` from "Restore Sparse Data" to "Import Sparse Data".
- Replaced references to `restoreSparseData` with `importSparseData` across code:
- Event listener for the sparse data button now uses `importSparseData`.
- Function name changed to `importSparseData` to reflect new terminology.
- Logging messages updated for consistency ("restored" -> "imported").
- Modified the database operation to use `importSparseData` for better alignment with the updated terminology.
### Purpose:
Improved clarity and consistency in terminology related to sparse data operations. The term "import" better aligns with its functionality and enhances user understanding across the application.
### Changes:
- **README Enhancements**:
- Added detection capabilities for browser, Node.js, and serverless environments with optimized configuration.
- Expanded details on adaptive storage strategies based on the detected environment.
- Included new information on the Model Control Protocol (MCP), including components like `BrainyMCPAdapter` and `MCPAugmentationToolset`.
- **Terminology Updates**:
- Replaced "restore sparse data" terminology with "import sparse data" in code samples and CLI commands for clarity (e.g., `importSparseData` method and `brainy import-sparse` command).
- Updated related examples for consistency with new term usage.
- **Cross-Environment Compatibility**:
- Highlighted dynamic imports and storage migration for seamless operation in different environments.
- Described augmented browser support and streamlined development practices.
- **MCP Documentation**:
- Detailed REST and WebSocket APIs for interacting with Brainy through MCP.
- Outlined configuration options like rate limiting and authentication for MCP services.
- **cloud-wrapper/README.md**:
- Added configuration details for MCP services, including specific usage of ports, API keys, and CORS settings.
- Expanded API scopes by introducing MCP-related endpoints for real-time and batch interactions.
### Purpose:
Enhanced documentation to reflect recent updates to environment detection, storage optimization, and MCP services. The changes improve clarity, usability, and developer understanding across environments and interaction protocols.
### Changes:
- Disabled the `@typescript-eslint/no-explicit-any` rule in `cloud-wrapper/package.json`.
- Reformatted ESLint rule configurations for `semi` and `@typescript-eslint/semi`.
- Reorganized dependencies in `package.json` for better readability:
- Moved `buffer`, `commander`, `omelette`, and `uuid` to the end of the list to match alphabetical order.
### Purpose:
Improved code clarity and maintainability by updating ESLint configurations and reordering dependencies in `package.json`. These adjustments ensure consistent formatting and easier navigation of dependencies.
### Changes:
- Introduced `global.d.ts` to define global type declarations for the `globalThis` interface.
- Added `__ENV__` object with the following properties:
- `isBrowser: boolean`
- `isNode: string | false`
- `isServerless: boolean`
- Included an export statement to ensure the file is treated as a module.
### Purpose:
Defined global environment properties to standardize and enhance environment detection across browser, Node.js, and serverless environments. This improves type safety and consistency in the codebase.
### Changes:
- Added `cloud-wrapper/dist` to ignore built output.
- Added `cloud-wrapper/node_modules` to ignore dependency files.
### Purpose:
Updated `.gitignore` to exclude build artifacts and dependency files specific to the `cloud-wrapper`, ensuring a cleaner repository and preventing unintentional commits of these files.
