soulcraft/brainy
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions 1

**chore: remove outdated changelog and summary documents**

Browse source 
- Deleted `CHANGES.md`, `CHANGES_SUMMARY.md`, `CONCURRENCY_ANALYSIS.md`, `CONCURRENCY_IMPLEMENTATION_SUMMARY.md`, and related developer documentation files.
- Removed redundant or legacy content no longer aligned with the current codebase and workflows.
- Updated repository to reflect streamlined documentation approach, reducing clutter and improving maintainability.

**Purpose**: Simplify and declutter repository by removing obsolete documentation files, ensuring it remains focused and relevant.
This commit is contained in:
David Snelling 2025-07-30 11:51:39 -07:00
parent 0f2075ede4
commit 79df44351c
30 changed files with 343 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

250
docs/development/DEVELOPERS.md Normal file
View file

@ -0,0 +1,250 @@
# Brainy Developer Guide
<div align="center">
<img src="./brainy.png" alt="Brainy Logo" width="200"/>
</div>
This document contains detailed information for developers working with Brainy, including building, testing, and
publishing instructions.
## Table of Contents
- [Build System](#build-system)
- [Testing](#testing)
- [Testing All Environments](#testing-all-environments)
- [Testing the CLI Package Locally](#testing-the-cli-package-locally)
- [Publishing](#publishing)
- [Publishing the CLI Package](#publishing-the-cli-package)
- [Development Usage](#development-usage)
- [Node.js 24 Optimizations](#nodejs-24-optimizations)
- [Development Workflow](#development-workflow)
- [Reporting Issues](#reporting-issues)
- [Code Style Guidelines](#code-style-guidelines)
- [Badge Maintenance](#badge-maintenance)
## Build System
Brainy uses a modern build system that optimizes for both Node.js and browser environments:
1. **ES Modules**
- Built as ES modules for maximum compatibility
- Works in modern browsers and Node.js environments
- Separate optimized builds for browser and Node.js
2. **Environment-Specific Builds**
- **Node.js Build**: Optimized for server environments with full functionality
- **Browser Build**: Optimized for browser environments with reduced bundle size
- **CLI Build**: Separate build for command-line interface functionality
- Conditional exports in package.json for automatic environment detection
3. **Modular Architecture**
- Core functionality and CLI are built separately
- CLI (4MB) is only included when explicitly imported or used from command line
- Reduced bundle size for browser and Node.js applications
4. **Environment Detection**
- Automatically detects whether it's running in a browser or Node.js
- Loads appropriate dependencies and functionality based on the environment
- Provides consistent API across all environments
5. **TypeScript**
- Written in TypeScript for type safety and better developer experience
- Generates type definitions for TypeScript users
- Compiled to ES2020 for modern JavaScript environments
6. **Build Scripts**
- `npm run build`: Builds the core library without CLI
- `npm run build:browser`: Builds the browser-optimized version
- `npm run build:cli`: Builds the CLI version (only needed for CLI usage)
- `npm run prepare:cli`: Builds the CLI for command-line usage
- `npm run demo`: Builds both core library and browser versions and starts a demo server
- GitHub Actions workflow: Automatically deploys the demo directory to GitHub Pages when pushing to the main branch
## Testing
### Test Scripts
Brainy provides several test scripts for different testing scenarios:
```bash
# Run all tests
npm test
# Run tests with comprehensive reporting
npm run test:report
# Run tests in watch mode
npm test:watch
# Run tests with UI
npm test:ui
# Run specific test suites
npm run test:node
npm run test:browser
npm run test:core
# Run tests with coverage
npm run test:coverage
```
The `test:report` script provides a comprehensive test report showing detailed information about all tests that were run, including test names, execution time, and pass/fail status.
### Testing Best Practices
When developing and debugging Brainy, follow these testing guidelines:
1. **Use Proper Test Files**: All tests should be written as vitest test files in the `tests/` directory with `.test.ts` or `.spec.ts` extensions.
2. **Avoid Temporary Debug Files**: Do not create temporary debug files like `debug_test.js`, `reproduce_issue.js`, or similar files in the root directory. These files:
- Clutter the repository
- Are excluded by vitest configuration but remain in the codebase
- Often duplicate functionality already covered by proper tests
3. **Debugging Approach**: When debugging issues:
- Add temporary test cases to existing test files in the `tests/` directory
- Use `it.only()` or `describe.only()` to focus on specific tests during debugging
- Remove or convert temporary test cases to permanent tests before committing
- Use the existing test setup and utilities in `tests/setup.ts`
4. **Test Organization**:
- Core functionality tests go in `tests/core.test.ts`
- Environment-specific tests go in `tests/environment.*.test.ts`
- Utility function tests go in `tests/vector-operations.test.ts`
- New feature tests should follow the existing naming convention
5. **Cleanup**: Always clean up temporary files before committing. The vitest configuration already excludes `*.js` files in the root directory, but they should be deleted rather than left in the repository.
6. **Test Reporting**: Use the comprehensive test reporting feature when you need detailed information about test execution:
- Run `npm run test:report` to get a verbose report of all tests
- The report includes test names, execution time, and pass/fail status
- This is especially useful for CI/CD pipelines and debugging test failures
### Testing All Environments
Brainy provides a comprehensive test script that verifies the library works correctly in all supported environments (
browser, Node.js, and CLI):
```bash
# Test the library in all environments
npm run test:all
```
This script:
1. Builds all packages (main, browser, CLI)
2. Runs Node.js tests (worker tests and unified text encoding test)
3. Starts a local HTTP server and runs browser tests using Puppeteer (headless browser)
4. Runs CLI tests by installing the CLI package locally and testing basic commands
The test results are displayed with color-coded output for better readability.
### Testing the CLI Package Locally
Before publishing the CLI package to npm, you can test it locally to ensure it works as expected:
```bash
# Test the CLI package locally
npm run test:cli
```
This script:
1. Builds the main package
2. Creates a local tarball of the main package
3. Builds the CLI package
4. Updates the CLI package to use the local main package
5. Creates a local tarball of the CLI package
6. Installs the CLI package globally for testing
After running this script, you can use the CLI commands as if you had installed the package from npm:
```bash
# Test the CLI
brainy --version
brainy init
brainy add "Test data" '{"noun":"Thing"}'
brainy search "test"
```
When you're done testing, you can uninstall the CLI package:
```bash
npm uninstall -g @soulcraft/brainy-cli
```
## Publishing
### Publishing the CLI Package
If you need to publish the CLI package to npm, please refer to the [CLI Publishing Guide](docs/publishing-cli.md) for
detailed instructions.
## Development Usage
```bash
# Run the CLI directly from the source
npm run cli help
# Generate a random graph for testing
npm run cli generate-random-graph --noun-count 20 --verb-count 40
```
## Node.js 24 Optimizations
Brainy takes advantage of several optimizations available in Node.js 24:
1. **Improved Worker Threads Performance**: The multithreading system has been completely rewritten to leverage Node.js
24's enhanced Worker Threads API, resulting in better performance for compute-intensive operations like embedding
generation and vector similarity calculations.
2. **Worker Pool Management**: A sophisticated worker pool system reuses worker threads to minimize the overhead of
creating and destroying threads, leading to more efficient resource utilization.
3. **Dynamic Module Imports**: Uses the new `node:` protocol prefix for importing core modules, which provides better
performance and more reliable module resolution.
4. **ES Modules Optimizations**: Takes advantage of Node.js 24's improved ESM implementation for faster module loading
and execution.
5. **Enhanced Error Handling**: Implements more robust error handling patterns available in Node.js 24 for better
stability and debugging.
These optimizations are particularly beneficial for:
- Large-scale vector operations
- Batch processing of embeddings
- Real-time data processing pipelines
- High-throughput search operations
## Development Workflow
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Submit a pull request
## Reporting Issues
We use GitHub issues to track bugs and feature requests. When creating a new issue, please provide detailed information including steps to reproduce, expected behavior, and actual behavior for bugs, or clear use cases and benefits for feature requests.
## Code Style Guidelines
Brainy follows a specific code style to maintain consistency throughout the codebase:
1. **No Semicolons**: All code in the project should avoid using semicolons wherever possible
2. **Formatting**: The project uses Prettier for code formatting
3. **Linting**: ESLint is configured with specific rules for the project
4. **TypeScript Configuration**: Strict type checking enabled with ES2020 target
5. **Commit Messages**: Use the imperative mood and keep the first line concise
## Badge Maintenance
The README badges are automatically updated during the build process:
1. **npm Version Badge**: The npm version badge is automatically updated to match the version in package.json when:
- Running `npm run build` (via the prebuild script)
- Running `npm version` commands (patch, minor, major)
- Manually running `node scripts/generate-version.js`
This ensures that the badge always reflects the current version in package.json, even before publishing to npm.

153
docs/development/DOCUMENTATION_STANDARDS.md Normal file
View file

@ -0,0 +1,153 @@
# Documentation Standards for Brainy
<div align="center">
<img src="./brainy.png" alt="Brainy Logo" width="200"/>
</div>
This document outlines the documentation standards and conventions for the Brainy project, including markdown file naming conventions and troubleshooting information for common documentation issues.
## Table of Contents
- [Markdown File Naming Conventions](#markdown-file-naming-conventions)
- [Documentation Troubleshooting](#documentation-troubleshooting)
## Markdown File Naming Conventions
Based on the current project structure, we follow these conventions for markdown files:
### Uppercase Naming
Use uppercase filenames for project-level documentation:
- README.md - Project overview and main documentation
- CONTRIBUTING.md - Contribution guidelines
- LICENSE.md - License information
- CHANGES.md - Changelog
- CODE_OF_CONDUCT.md - Code of conduct
- Other project-level documentation files
Examples: `README.md`, `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`
### Lowercase Naming
Use lowercase filenames for technical documentation and implementation details:
- Technical guides
- Implementation details
- Architecture documentation
- Specific feature documentation
Examples: `scalingStrategy.md`, `statistics.md`
## Rationale
This convention makes it easy to distinguish between:
1. Project-level documentation that applies to the entire project and is relevant to all contributors and users (uppercase)
2. Technical documentation that focuses on specific implementation details and is primarily relevant to developers working on those features (lowercase)
## Recommendations
1. Continue using uppercase names for project-level documentation files
2. Continue using lowercase names for technical documentation files
3. Be consistent within each category
4. Always use `README.md` (uppercase) for directory-level documentation
## Examples
### Project-Level Documentation (Uppercase)
- README.md
- CONTRIBUTING.md
- LICENSE.md
- CHANGES.md
- CODE_OF_CONDUCT.md
- DEVELOPERS.md
- STORAGE_TESTING.md
- THREADING.md
### Technical Documentation (Lowercase)
- scalingStrategy.md
- statistics.md
- architecture.md
- implementation-details.md
By following these conventions, we maintain consistency and make it easier for contributors to find the right documentation.
## Documentation Troubleshooting
This section covers common documentation-related issues and their solutions.
### Fix for "process.memoryUsage is not a function" Error in Vitest
#### Issue
During test runs with Vitest, the following error was occurring:
```
TypeError: process.memoryUsage is not a function
❯ VitestTestRunner.onAfterRunSuite node_modules/vitest/dist/runners.js:150:95
```
This error was happening because Vitest was trying to use `process.memoryUsage()` to log heap usage statistics, but this function was not available in the current environment.
#### Solution
The issue was fixed by disabling the heap usage logging in the Vitest configuration:
In `vitest.config.ts`, changed:
```typescript
// Show test statistics
logHeapUsage: true,
```
To:
```typescript
// Show test statistics
logHeapUsage: false,
```
#### Explanation
The `logHeapUsage` option in Vitest attempts to use Node.js's `process.memoryUsage()` function to track and report memory usage during test runs. However, this function might not be available in all environments, particularly in certain browser-like environments or when using specific Node.js versions or configurations.
By setting `logHeapUsage: false`, we prevent Vitest from attempting to call this function, which resolves the error while still allowing tests to run successfully.
#### Verification
After making this change, the tests run without any unhandled errors, confirming that the issue has been resolved.
### Common Documentation Issues and Solutions
#### Issue: Inconsistent Markdown Formatting
**Symptoms**: Inconsistent heading levels, list formatting, or code block syntax across documentation files.
**Solution**:
- Use a markdown linter to enforce consistent formatting
- Follow the project's markdown style guide
- Use the same heading structure across similar documents
#### Issue: Broken Links in Documentation
**Symptoms**: Links to other documentation files or sections within files don't work.
**Solution**:
- Use relative links for references to other files in the repository
- Use anchor links for references to sections within the same file
- Regularly check for broken links, especially after moving or renaming files
#### Issue: Outdated Documentation
**Symptoms**: Documentation describes features or APIs that have changed or been removed.
**Solution**:
- Update documentation as part of the same PR that changes the code
- Add a "Last Updated" date to documentation files
- Regularly review and update documentation
#### Issue: Missing Documentation
**Symptoms**: Features or APIs lack documentation, making them difficult to use.
**Solution**:
- Require documentation for new features as part of the PR review process
- Create documentation templates for common types of documentation
- Identify and prioritize documentation gaps

35
docs/development/EXPECTED_TEST_MESSAGES.md Normal file
View file

@ -0,0 +1,35 @@
# Expected Messages During Vitest Execution
This document explains the various messages and errors that appear during test execution and why they are expected.
## Fixed Issues
### Duplicate Summary Output
- **Issue**: Previously, test summaries were appearing twice at the end of test runs
- **Fix**: Removed the verbose reporter from the configuration, keeping only the default and JSON reporters
- **Status**: Resolved
## Expected Error Messages
The following error messages appear during test runs and are expected as part of the test suite:
### S3 Storage Tests
- **Error**: `[MOCK S3] Error processing command: Error: NoSuchKey: The specified key does not exist.`
- **Source**: `tests/s3-storage.test.ts`
- **Explanation**: This error is expected and is part of the test for the S3 storage adapter. The test intentionally deletes a noun and then tries to retrieve it to verify it was properly deleted.
### Dimension Mismatch Errors
- **Error**: `Failed to add vector: Error: Vector dimension mismatch: expected 512, got X`
- **Source**: `tests/dimension-standardization.test.ts` and `tests/core.test.ts`
- **Explanation**: These tests specifically verify that the system correctly rejects vectors with incorrect dimensions. The error messages confirm that the validation is working as expected.
### API Integration Test Failure
- **Error**: `expected 500 to be 200 // Object.is equality`
- **Source**: `tests/api-integration.test.ts`
- **Explanation**: This appears to be an actual test failure that should be investigated separately. The test expects a 200 status code but is receiving a 500 error.
## Conclusion
Most of the error messages seen during test execution are expected and are part of testing error handling paths. These messages confirm that the system is correctly handling error conditions as designed.
The only unexpected issue is the API integration test failure, which should be investigated as a separate issue.

67
docs/development/MARKDOWN_CONVENTIONS.md Normal file
View file

@ -0,0 +1,67 @@
# Markdown File Naming Conventions
This document outlines the naming conventions for markdown (.md) files in the Brainy project.
## Naming Patterns
Based on the current project structure, we follow these conventions for markdown files:
### Uppercase Naming
Use uppercase filenames for project-level documentation:
- README.md - Project overview and main documentation
- CONTRIBUTING.md - Contribution guidelines
- LICENSE.md - License information
- CHANGES.md - Changelog
- CODE_OF_CONDUCT.md - Code of conduct
- Other project-level documentation files
Examples: `README.md`, `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`
### Lowercase Naming
Use lowercase filenames for technical documentation and implementation details:
- Technical guides
- Implementation details
- Architecture documentation
- Specific feature documentation
Examples: `scalingStrategy.md`, `statistics.md`
## Rationale
This convention makes it easy to distinguish between:
1. Project-level documentation that applies to the entire project and is relevant to all contributors and users (uppercase)
2. Technical documentation that focuses on specific implementation details and is primarily relevant to developers working on those features (lowercase)
## Recommendations
1. Continue using uppercase names for project-level documentation files
2. Continue using lowercase names for technical documentation files
3. Be consistent within each category
4. Always use `README.md` (uppercase) for directory-level documentation
## Examples
### Project-Level Documentation (Uppercase)
- README.md
- CONTRIBUTING.md
- LICENSE.md
- CHANGES.md
- CODE_OF_CONDUCT.md
- DEVELOPERS.md
- STORAGE_TESTING.md
- THREADING.md
### Technical Documentation (Lowercase)
- scalingStrategy.md
- statistics.md
- architecture.md
- implementation-details.md
By following these conventions, we maintain consistency and make it easier for contributors to find the right documentation.

73
docs/development/PRETTY_TEST_REPORTER.md Normal file
View file

@ -0,0 +1,73 @@
# Pretty Test Reporter for Brainy
This document describes the visually enhanced test reporter added to the Brainy project.
## Overview
The Pretty Test Reporter provides a visually appealing summary of test results with colors, symbols, and formatted output. It enhances the standard Vitest output with a clear, easy-to-read summary at the end of test runs.
## Features
- 🎨 **Colorful Output**: Uses colors to distinguish between passed, failed, and skipped tests
- 📊 **Tabular Format**: Displays test results in a clean, tabular format
- 📝 **Detailed Summary**: Shows overall test statistics and file-by-file breakdown
- ❌ **Error Reporting**: Clearly lists any failed tests with their error messages
- ⏱️ **Timing Information**: Displays test duration in a human-readable format
## Usage
To run tests with the pretty reporter, use the following npm script:
```bash
npm run test:report:pretty
```
You can also specify specific test files:
```bash
npm run test:report:pretty -- tests/core.test.ts
```
## Example Output
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📊 TEST SUMMARY REPORT
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Test Run Completed in: 7.9s
Date: 7/28/2025, 11:22:54 AM
Total Test Files: 1
Total Tests: 19
Results:
✓ Passed: 19
✗ Failed: 0
○ Skipped: 0
Test Files:
┌──────────────────────────────────────────────────┬──────────┬──────────┬──────────┐
│ File │ Passed │ Failed │ Skipped │
├──────────────────────────────────────────────────┼──────────┼──────────┼──────────┤
│ core.test.ts │ 19 │ 0 │ 0 │
└──────────────────────────────────────────────────┴──────────┴──────────┴──────────┘
PASSED All tests passed successfully!
```
## Implementation Details
The pretty reporter is implemented as a custom Vitest reporter in `src/testing/prettySummaryReporter.ts`. It:
1. Collects test information during the test run
2. Tracks passed, failed, and skipped tests
3. Organizes results by test file
4. Generates a formatted summary at the end of the test run
## Configuration
The reporter is configured in `vitest.config.ts` and works alongside the default Vitest reporter and JSON reporter. This provides both the standard output during test execution and the enhanced summary at the end.
## Customization
If you need to modify the reporter's appearance or behavior, you can edit the `prettySummaryReporter.ts` file. The main visual elements are in the `printSummary` method.