### Changes:
- Introduced a new `examples/simplified-augmentations.js` file showcasing comprehensive usage examples:
- Example 1: Creating and using a simple memory augmentation.
- Example 2: Adding WebSocket support to a conduit augmentation.
- Example 3: Processing static data through a reusable pipeline.
- Example 4: Setting up and executing streaming data pipelines.
- Example 5: Dynamically loading augmentations from a module.
- Demonstrated augmentation factories, streamlined pipelines, dynamic loading, and reusable workflows through practical scenarios.
### Purpose:
Added detailed examples to assist developers in understanding and implementing augmentation factories and streamlined pipelines. These examples provide a hands-on reference for building static and streaming workflows, improving usability and accessibility for new and existing users.
### Changes:
- Updated `src/index.ts` exports to streamline pipeline and augmentation factory usage:
- Consolidated augmentation and streamlined pipeline exports under unified `Pipeline` module (e.g., `pipeline`, `createPipeline`, `executeStreamlined`).
- Added new augmentation factory exports:
- `createSenseAugmentation`, `addWebSocketSupport`, `executeAugmentation`, `loadAugmentationModule`.
- Enhanced backward compatibility with separate sequential pipeline exports.
- Introduced streamlined type exports, including `StreamlinedExecutionMode`, `StreamlinedPipelineOptions`, and `StreamlinedPipelineResult`.
### Purpose:
Standardized pipeline and augmentation factory exports for better modularity and backward compatibility. These changes improve developer experience by simplifying imports and aligning with the unified augmentation pipeline system.
### Changes:
- Added `Pipeline` class in `src/pipeline.ts` to unify primary and streamlined augmentation pipelines:
- Supports multiple execution modes: sequential, parallel, threaded, first success, and first result.
- Provides methods for registering, unregistering, and managing augmentations by type.
- Implements flexible execution options, including error handling, timeouts, and threading control.
- Introduced comprehensive augmentation registry for managing sense, dialog, memory, cognition, and WebSocket-based augmentations.
- Added global pipeline instance with legacy method support for backward compatibility.
- Enhanced augmentation execution capabilities:
- Support for static and streaming data processing.
- Pipeline creation for reusable augmentation workflows.
- Threaded execution for improved performance when supported.
- Added default pipeline integration with augmentation registry to resolve circular dependencies.
### Purpose:
Implemented a unified augmentation pipeline to streamline augmentation workflows, enhance flexibility, and simplify execution management. The changes improve performance, reduce complexity, and provide robust support for legacy methods.
### Changes:
- Updated the README to simplify descriptions and remove redundant details:
- Reduced verbosity in feature highlights, focusing on core functionality.
- Streamlined examples and usage sections for greater clarity.
- Simplified installation and quick start instructions.
- Removed overly casual or playful language to adopt a more professional tone.
- Consolidated streaming, environment detection, and build system explanations.
- Improved formatting consistency across sections:
- Aligned headings, examples, and lists for readability.
- Adjusted code indentation and structure in examples.
### Purpose:
Enhanced the README to provide a concise, focused, and professional presentation of Brainy. These changes improve readability, simplify onboarding for new users, and ensure consistency with recent terminology and documentation standards.
### Changes:
- Added a new `scalingStrategy.md` document explaining limitations and solutions for handling large-scale data with HNSW:
- Highlighted in-memory limitations of standard HNSW implementations.
- Detailed disk-based, distributed, and hybrid approaches for scaling to terabyte-scale datasets.
- Provided real-world examples from systems like Qdrant, Milvus, FAISS, and DiskANN.
- Emphasized trade-offs between memory usage, search accuracy, and performance when scaling.
### Purpose:
Documented approaches for managing large-scale data with HNSW to assist developers in implementing efficient, scalable solutions. The guide serves as a reference for overcoming memory limitations and adopting advanced techniques for handling terabyte-scale datasets.
### Changes:
- Introduced the `import-sparse` command in `src/cli.ts`:
- Allows importing sparse data from a JSON file into the database.
- Includes an option (`--clear`) to clear existing data before import.
- Displays counts of imported nouns and verbs upon successful completion.
- Updated CLI help and autocompletion configuration to support the new command.
### Purpose:
Enhanced the CLI by adding `import-sparse`, providing users with the ability to efficiently import sparse data into the database. This addition aligns with the application's functionality for managing sparse data operations.
### Changes:
- Introduced the `importSparseData` method in `src/brainyData.ts` to allow importing sparse data into the database.
- Supports embedding generation for nouns if vectors are missing.
- Includes options to clear existing data during import.
- Returns counts of imported nouns and verbs for tracking.
### Purpose:
Added a new method to manage sparse data imports, improving functionality and ensuring alignment with updated terminology across the application. This addition enhances flexibility in handling database operations.
### Changes:
- Introduced `initializeMCPService` in `cloud-wrapper/src/index.ts` to support initializing Model Control Protocol (MCP) services.
- Enabled configuration for WebSocket and REST APIs with customizable options like ports, authentication, rate limiting, and CORS.
- Exported MCP modules, types, and services (`BrainyMCPAdapter`, `BrainyMCPService`, etc.) in `src/index.ts`.
- Enhanced `unified.ts` to include global environment detection by populating `globalThis.__ENV__`.
- Added environment-specific logging for browser, Node.js, and serverless scenarios.
- Improved the structure of `environment` object and its global accessibility.
### Purpose:
Implemented MCP service integration to expand Brainy's external compatibility via WebSocket and REST interfaces. Enhanced environment detection provides better global access and debugging insights, ensuring robust operation across various runtime environments.
### Changes:
- Enhanced environment detection logic with a structured `environment` object (`isBrowser`, `isNode`, `isServerless`).
- Implemented fallback options for various environments:
- Added priority handling for storage types (e.g., MemoryStorage, OPFSStorage, FileSystemStorage).
- Improved dynamic imports for `s3CompatibleStorage.js` and `fileSystemStorage.js`.
- Unified and expanded cloud storage support by merging environment variables with provided options:
- Refined logic for R2, S3, and GCS storage configurations.
- Added detailed logging during storage type selection (`console.log` for selected storage).
- Introduced better error handling with warnings and defaults for unsupported environments.
- Removed redundant checks and reorganized storage selection logic for clarity.
### Purpose:
Streamlined environment detection and adaptive storage logic to ensure seamless operation across different platforms (browser, Node.js, serverless). These changes improve maintainability, enhance debugging, and simplify storage configuration while providing robust fallback mechanisms.
### Changes:
- Updated button label in `examples/demo.html` from "Restore Sparse Data" to "Import Sparse Data".
- Replaced references to `restoreSparseData` with `importSparseData` across code:
- Event listener for the sparse data button now uses `importSparseData`.
- Function name changed to `importSparseData` to reflect new terminology.
- Logging messages updated for consistency ("restored" -> "imported").
- Modified the database operation to use `importSparseData` for better alignment with the updated terminology.
### Purpose:
Improved clarity and consistency in terminology related to sparse data operations. The term "import" better aligns with its functionality and enhances user understanding across the application.
### Changes:
- **README Enhancements**:
- Added detection capabilities for browser, Node.js, and serverless environments with optimized configuration.
- Expanded details on adaptive storage strategies based on the detected environment.
- Included new information on the Model Control Protocol (MCP), including components like `BrainyMCPAdapter` and `MCPAugmentationToolset`.
- **Terminology Updates**:
- Replaced "restore sparse data" terminology with "import sparse data" in code samples and CLI commands for clarity (e.g., `importSparseData` method and `brainy import-sparse` command).
- Updated related examples for consistency with new term usage.
- **Cross-Environment Compatibility**:
- Highlighted dynamic imports and storage migration for seamless operation in different environments.
- Described augmented browser support and streamlined development practices.
- **MCP Documentation**:
- Detailed REST and WebSocket APIs for interacting with Brainy through MCP.
- Outlined configuration options like rate limiting and authentication for MCP services.
- **cloud-wrapper/README.md**:
- Added configuration details for MCP services, including specific usage of ports, API keys, and CORS settings.
- Expanded API scopes by introducing MCP-related endpoints for real-time and batch interactions.
### Purpose:
Enhanced documentation to reflect recent updates to environment detection, storage optimization, and MCP services. The changes improve clarity, usability, and developer understanding across environments and interaction protocols.
### Changes:
- Disabled the `@typescript-eslint/no-explicit-any` rule in `cloud-wrapper/package.json`.
- Reformatted ESLint rule configurations for `semi` and `@typescript-eslint/semi`.
- Reorganized dependencies in `package.json` for better readability:
- Moved `buffer`, `commander`, `omelette`, and `uuid` to the end of the list to match alphabetical order.
### Purpose:
Improved code clarity and maintainability by updating ESLint configurations and reordering dependencies in `package.json`. These adjustments ensure consistent formatting and easier navigation of dependencies.
### Changes:
- Introduced `global.d.ts` to define global type declarations for the `globalThis` interface.
- Added `__ENV__` object with the following properties:
- `isBrowser: boolean`
- `isNode: string | false`
- `isServerless: boolean`
- Included an export statement to ensure the file is treated as a module.
### Purpose:
Defined global environment properties to standardize and enhance environment detection across browser, Node.js, and serverless environments. This improves type safety and consistency in the codebase.
### Changes:
- Added `cloud-wrapper/dist` to ignore built output.
- Added `cloud-wrapper/node_modules` to ignore dependency files.
### Purpose:
Updated `.gitignore` to exclude build artifacts and dependency files specific to the `cloud-wrapper`, ensuring a cleaner repository and preventing unintentional commits of these files.
### Changes:
- **Core MCP Components**:
- Introduced `BrainyMCPAdapter`, `BrainyMCPService`, and `MCPAugmentationToolset` for handling data access, tool execution, system information, and authentication via MCP.
- Added asynchronous handlers to process requests for Brainy data, augmentations, and relationships.
- Built pipelines to expose Brainy augmentation capabilities as tools.
- **Server Implementations**:
- Added WebSocket and REST interfaces in `initializeMCPService` for external MCP requests.
- Included rate limiting, authentication, and CORS support for REST API.
- **Type Definitions**:
- Defined MCP-related types such as `MCPRequestType`, `MCPResponse`, and `MCPToolExecutionRequest` in `src/types/mcpTypes.ts`.
- Incorporated augmentation-type-specific methods from `augmentationPipeline`.
- **Exports**:
- Exposed MCP modules (`BrainyMCPAdapter`, `BrainyMCPService`, `MCPAugmentationToolset`) via `src/mcp/index.ts`.
### Purpose:
Introduced a Model Control Protocol (MCP) framework to enable seamless integration of Brainy data and augmentation tools with external models. The implementation provides structured, scalable access to data and tools through both WebSocket and REST APIs.
### Changes:
- Renamed the `publish` script to `deploy` in `package.json` for clarity and consistency with deployment terminology.
- Incremented version number from `0.8.11` to `0.9.0` in `src/utils/version.ts`.
### Purpose:
Aligned the naming of the deployment script with standard practices to improve readability and maintainability. Updated the version to `0.9.0` to reflect recent changes and enhancements in the project.
### Changes:
- Added `rollup.config.js` with custom build logic for unified and browser builds:
- Included TypeScript support with separate configurations (`tsconfig.unified.json` and `tsconfig.browser.json`).
- Consolidated environment detection logic with `globalThis.__ENV__` for browser, Node.js, and serverless environments.
- Provided global Buffer polyfill for browser environments.
- Enabled `inlineDynamicImports` for module compatibility.
- Introduced custom Rollup plugins:
- Shims for Node.js built-in modules to ensure compatibility in browsers.
- Fixes for `this` references in TensorFlow.js layers causing errors in browser environments.
- Enabled environment-specific replacements for `process.env.NODE_ENV` during builds.
- Configured outputs for both non-minified and minified builds (`.js` and `.min.js` files).
- Excluded specific dependencies from the bundle to optimize build size.
### Purpose:
Added a versatile Rollup configuration to streamline the build process for both browser and Node.js environments. This setup enhances cross-environment compatibility, reduces redundancy, and ensures better maintainability with unified build logic.
### Changes:
- Added explicit `.js` extensions to imports in `cloud-wrapper/src/index.ts` for Node.js compatibility.
- Refactored `setupRoutes` usage by introducing an `apiRouter` variable for clarity and consistency in middleware.
- Simplified environment detection in `src/unified.ts` by delegating it to build-time logic:
- Removed inline environment detection (`isBrowser`, `isNode`, `isServerless`).
- Updated `environment` export with build-generated properties.
### Purpose:
Enhanced compatibility with Node.js module resolution, improved code clarity in route handling, and streamlined environment detection by consolidating it into the build process. These changes reduce redundancy, improve maintainability, and align with modern JavaScript practices.
### Changes:
- Added new TensorFlow.js dependencies:
- `@tensorflow/tfjs-backend-cpu`
- `@tensorflow/tfjs-converter`
- `@tensorflow/tfjs-core`
- `@tensorflow/tfjs-layers`
- Updated `package-lock.json` to reflect dependency updates and changes in peer dependencies.
- Refactored build scripts for clarity and consistency:
- Simplified build commands using `BUILD_TYPE` environment variable for unified and browser builds.
- Updated `demo` and `publish` scripts to align with new build structure.
- Introduced specific deployment scripts for AWS, GCP, and Cloudflare in `cloud-wrapper`:
- `deploy☁️aws`
- `deploy☁️gcp`
- `deploy☁️cloudflare`
- Added a user-friendly `deploy:cloud` script for deployment guidance.
- Removed redundant script configurations and legacy-based Rollup files.
### Purpose:
Enhanced dependency management and standardized build/deployment workflows by improving script clarity, removing redundancy, and aligning configurations with TensorFlow.js changes. This improves maintainability and user experience in development and production environments.
### Changes:
- Introduced `asyncHandler` utility to simplify error handling in asynchronous route handlers.
- Updated all route definitions in `cloud-wrapper/src/routes.ts` to use the new `asyncHandler` wrapper:
- Standardized error management across CRUD operations for nouns, verbs, and search functionality.
- Replaced redundant `try-catch` blocks with the `asyncHandler` wrapper to streamline code.
- Added `NextFunction` and adjusted TypeScript typings for improved clarity and error handling.
### Purpose:
Refactored route handlers to enhance maintainability, readability, and consistency by centralizing asynchronous error handling with a reusable `asyncHandler` utility. This reduces redundancy and simplifies debugging for API routes.
### Changes:
- Expanded cloud deployment section in `README.md`:
- Added detailed npm scripts for deploying to AWS, Google Cloud, and Cloudflare.
- Described script usage for each cloud platform.
- Included instructions for configuring environment variables in `cloud-wrapper/.env`.
- Linked to the Cloud Wrapper README for additional setup and API documentation.
### Purpose:
Improved clarity and usability of cloud deployment instructions by providing specific npm script commands and configuration guidance. This helps streamline the deployment process for users across multiple cloud platforms.
### Changes:
- Removed two documentation files:
- `docs/build-time-augmentations.md`
- `docs/llm-augmentation.md`
- Deleted `rollup.unified.js` and `rollup_config.js` configurations to streamline build setup.
- Introduced custom plugins for Node.js module shims and "this" reference fixes directly into the remaining Rollup setup.
- Updated Rollup configuration entry and build logic for increased maintainability:
- Enabled `inlineDynamicImports` for unified builds.
- Enhanced browser compatibility with global Buffer polyfill and environment detection (`isBrowser`, `isNode`, `isServerless`).
- Adjusted error logging for module fetch failures in browser environments.
### Purpose:
Cleaned outdated and redundant documentation to align with the system's current capabilities. Consolidated and optimized Rollup build setup, ensuring a modern and maintainable configuration while improving cross-environment compatibility and error handling.
### Changes:
- Introduced `package-lock.json` file for the `cloud-wrapper` package.
- Includes configuration details for dependencies:
- Runtime dependencies: `@soulcraft/brainy`, `cors`, `dotenv`, `express`, `helmet`, `morgan`, `uuid`, `ws`.
- Development dependencies: TypeScript, nodemon, and type definitions for key libraries.
- Specifies supported Node.js version (`>=23.11.0`).
### Purpose:
Committed the `package-lock.json` to ensure consistency in dependency resolution during installation. This facilitates improved reliability across development and production environments.
### Changes:
- Introduced `ExtendedBrainyDataConfig` interface to include `rootDirectory` in the storage configuration.
- Enhanced S3 storage setup:
- Added support for `customS3Storage` when `S3_ENDPOINT` is provided.
- Retained standard `s3Storage` configuration for AWS S3 without custom endpoints.
- Updated fallback logic for default storage types (`filesystem`, `memory`) with consistent handling for `rootDirectory`.
- Refined `initializeBrainy` function to improve readability and configuration extensibility.
- Updated TypeScript typings for improved clarity and maintainability.
### Purpose:
Improved the `initializeBrainy` function by extending storage configuration capabilities, allowing seamless integration of custom S3 endpoints alongside standard AWS S3 settings. Enhanced overall code readability and adaptability for diverse storage configurations.
### Changes:
- Introduced environment-independent WebStreams API compatibility:
- Added `initializeStreamClasses` for dynamic detection of WebStreams API support in browsers and Node.js.
- Implemented fallback mechanism with detailed error messages for unsupported environments.
- Ensured asynchronous initialization of stream classes without blocking module execution.
- Updated `SequentialPipeline`:
- Added `ensureStreamClassesInitialized` method to streamline WebStreams setup across operations.
- Enhanced `initialize` to parallelize stream classes and `BrainyData` initialization.
- Refactored WebSocket handlers (`createWebSocketHandler` and `createWebSocketStreams`) to support async stream initialization.
- Improved TypeScript typings for streams (`TransformStreamDefaultController`, `PipelineResult<unknown>`).
- Adjusted `ServerSearchConduitAugmentation`:
- Introduced dependency on `BrainyDataInterface` instead of direct `BrainyData` usage.
- Ensured `localDb` is set prior to initialization with an appropriate error message.
- Updated methods (`setLocalDb`, `getLocalDb`, and vector operations) to rely on `BrainyDataInterface`.
- Enhanced error handling and logging throughout streaming and augmentation processes.
### Purpose:
Improved cross-environment compatibility of WebStreams API, ensuring seamless usage in both browsers and Node.js. Refactored `ServerSearchConduitAugmentation` to decouple `BrainyData` dependency,
### Changes:
- Renamed `nodes` to `nouns` and `edges` to `verbs` across `opfsStorage.ts`, improving code semantics:
- Directory constants (e.g., `NODES_DIR` → `NOUNS_DIR`).
- Class properties, method names, and variables updated accordingly.
- Introduced global type augmentation for `FileSystemDirectoryHandle` to support `entries()` method, improving compatibility.
- Updated directory management logic:
- Adjusted initialization, recreation, and metadata retrieval methods to reflect new nomenclature.
- Enhanced handling for directory entries in both `nouns` and `verbs` contexts.
- Refined comments, formatting, and TypeScript annotations for consistency.
### Purpose:
Improved the code's readability and semantic clarity by aligning directory and variable names with their conceptual purpose (`nouns` for entities, `verbs` for relationships). Enhanced maintainability and developer understanding of the storage implementation.
### Changes:
- Removed conditional configuration for `USE_SIMPLE_EMBEDDING` as Universal Sentence Encoder is now the sole embedding option.
- Updated comments to reflect the exclusive use of TensorFlow.js for embedding functionality.
- Cleaned up unnecessary blank lines and whitespace for improved code readability.
### Purpose:
Simplified the `initializeBrainy` function by consolidating embedding configuration and eliminating outdated options. This refinement aligns with the project's transition to TensorFlow-based embeddings and improves code clarity.
### Changes:
- Removed `SimpleEmbedding` class, including its character-based embedding functionality and associated utility methods.
- Refined the Universal Sentence Encoder (USE) implementation:
- Added support for handling edge cases such as empty strings and arrays by returning zero vectors.
- Filtered out invalid input, improving robustness.
- Addressed `EPSILON` flag initialization and TensorFlow.js environment setup to prevent runtime errors.
- Updated error handling for embedding failures to provide more detailed traceability.
- Simplified `createEmbeddingFunction` and `defaultEmbeddingFunction` to focus solely on TensorFlow-based embeddings.
### Purpose:
Streamlined the embedding API by removing the outdated `SimpleEmbedding` class and fully transitioning to TensorFlow's Universal Sentence Encoder. These changes improve input validation, error resilience, and compatibility with TensorFlow.js, ensuring a more consistent and reliable embedding experience.
### Changes:
- Enhanced `BrainyData` class to support `HNSWIndexOptimized` alongside the standard HNSW index.
- Added a new configuration parameter `hnswOptimized` for enabling optimized indexing.
- Created utility methods for handling optimized index-specific functionalities such as memory usage tracking and disk-based index management.
- Updated vector addition, retrieval, and management methods to accommodate both standard and optimized indices.
- Introduced support for using pre-defined IDs during vector and relationship creation.
- Standardized metadata handling for error resilience and embedding integrations.
- Implemented a `getAllNouns` method to retrieve all indexed data.
- Added validation for vector dimensions during relationship creation to ensure consistency.
- Refactored method signatures to improve readability and alignment.
### Purpose:
Improved the BrainyData API to support optimized indexing functionality powered by `HNSWIndexOptimized`, enhancing scalability for large datasets while maintaining backward compatibility. Elevated overall code structure and usability.
### Changes:
- Added `HNSWIndexOptimized` class in `src/hnsw/hnswIndexOptimized.ts`:
- Supports large datasets with product quantization for dimensionality reduction.
- Enables disk-based index storage via a configurable storage adapter.
- Introduces `HNSWOptimizedConfig` interface for advanced configuration:
- Includes memory thresholds, product quantization (subvectors, centroids), and disk-based indexing options.
- Implements methods for adding, searching, and managing indexed vectors with enhanced scalability.
- Introduced `ProductQuantizer` utility:
- Handles vector quantization with training and reconstruction capabilities.
- Reduces vector dimensions, improving memory and performance efficiency.
- Added accessor methods in `src/hnsw/hnswIndex.ts` for configuration and index metadata (e.g., dimension, max level, entry point).
### Purpose:
Enhanced the HNSW implementation to support optimized indexing and search for large datasets, addressing scalability challenges via product quantization and optional disk-based storage. This update elevates the library's performance and applicability for memory-constrained environments.
### Changes:
- Updated `VERSION` constant in `src/utils/version.ts` from `0.8.0` to `0.8.11`.
### Purpose:
Incremented the project version to reflect new changes, updates, or bug fixes, ensuring proper version tracking and release management.
### Changes:
- Added `backup` command to export database content into a JSON file.
- Includes noun and verb data with metadata.
- Allows specifying a custom output filename (`brainy-backup.json` by default).
- Added `restore` command to import data from a JSON file into the database.
- Supports clearing existing data with `--clear` option before restoration.
- Displays success information for imported nouns and verbs.
- Integrated help text examples for the new commands.
- Enhanced CLI interface with additional indentation for better readability.
- Adjusted `completion-setup` to include autocomplete for backup and restore operations.
### Purpose:
Introduced backup and restore functionality to improve database management and data portability, enabling users to save and load datasets seamlessly for better flexibility in usage and recovery.
### Changes:
- Created `examples/browser_compatible_exports.ts`:
- Introduced browser-friendly exports with polyfill for `Buffer` in browser environments.
- Implemented shims for Node.js modules (`fs`, `util`, and `path`) to avoid compatibility issues.
- Exported browser-compatible types and utilities: `BrainyData`, `NounType`, `VerbType`, graph types, core types, distance functions, embedding utilities, and storage adapters.
- Explicitly excluded Node.js/CLI-specific parts to focus on browser compatibility.
- Added `src/types/brainyDataInterface.ts`:
- Defined `BrainyDataInterface` to break the circular dependency between `brainyData.ts` and `serverSearchAugmentations.ts`.
- Provided methods for managing nodes, relationships, initialization, search, and adding vector data with metadata.
### Purpose:
- Enhanced the codebase to support browser environments by introducing browser-specific exports and avoiding Node.js dependencies.
- Established clearer separation of concerns and modularity by introducing the `BrainyDataInterface` for dependency management, improving maintainability and flexibility.
### Changes:
- Removed the `src/augmentations/llmAugmentations.ts` file:
- Contained cognition augmentations for creating, testing, exporting, and deploying LLM models.
- Included TensorFlow.js-based model configuration, training, testing, and deployment functionalities.
- Implemented various presets (`tiny`, `small`, `medium`, `large`) for simplified LLM model creation.
### Purpose:
This change removes legacy LLM-related augmentations and functionalities no longer relevant to the project's current focus, in alignment with the recent transition to TensorFlow-based embedding workflows. Cleaning up eliminates unused code and maintains a streamlined, up-to-date codebase.
### Changes:
- Updated README.md:
- Refined the project description to highlight its lightweight and powerful nature.
- Enhanced feature list, replacing outdated entries with modern updates (e.g., TensorFlow integration replacing LLM creation).
- Included detailed installation and usage instructions for TensorFlow.js support and embedding workflows.
- Added "Usage Options" section to define different import methods and supported environments.
- Expanded "Backup and Restore" details, showcasing CLI and scripting examples.
- Introduced optimized HNSW indexing for large datasets.
- Replaced examples with a unified and simplified demonstration approach.
- Embedded advanced build and environment-specific configurations for better clarity.
- Removed outdated content related to language model generation, including LLM-related commands and examples.
- Added README.demo.md with full instructions for running the interactive demo.
### Purpose:
This update modernizes the documentation and focuses on Brainy’s current capabilities, simplifying onboarding for new users while removing deprecated features. It also streamlines project understanding, enabling developers to better utilize its features in diverse environments.
### Changes:
- Added `rollup.unified.js` for building a unified bundle with environment detection (Node.js, Browser, Serverless).
- Added `rollup_config.js` for browser-specific bundle configurations.
- Created `tsconfig.browser.json` for browser builds and `tsconfig.unified.json` for unified builds.
- Implemented custom Rollup plugins:
- **`fixThisReferences`**: Resolves `this` reference issues in TensorFlow files.
- **`nodeModuleShims`**: Provides empty shims for Node.js built-in modules in browser environments.
- Updated dependencies in `package-lock.json` to include Rollup plugins and updated `buffer` dependency for polyfills.
- Created `src/unified.ts` as the unified library entry point with dynamic environment detection.
### Purpose:
- Introduced a unified and browser-compatible build pipeline to ensure `Brainy` can seamlessly operate across multiple environments (Node.js, Browser, Serverless).
- Resolved compatibility issues with Node.js modules and specific library builds.
- Enhanced flexibility and usability for developers working in diverse runtime environments.
### Changes:
- Deleted the following outdated example files from the `examples` directory:
- `buildTimeRegistration.js`
- `conduitAugmentationExample.js`
- `configurationTest.js`
- `dataInspectionExample.js`
- `import-graphTypes.js`
- `browser-server-search/index.html`
### Purpose:
These example files were removed as they were no longer relevant or up-to-date with the current state of the project. This cleanup ensures that users referencing examples have access to accurate and relevant documentation and code, reducing potential confusion.
### Changes:
- Adjusted the main project description in `README.md` for a more engaging tone by adding "A fun" to emphasize the platform's appeal.
### Purpose:
This update aims to make the project description more approachable and engaging for potential users while maintaining clarity about its capabilities.
