soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

**chore(docs): improve code formatting and readability in README.md**

Browse source 
- Reformatted code examples for consistency in spacing and alignment:
  - Adjusted indentation
This commit is contained in:
David Snelling 2025-08-01 08:56:41 -07:00
parent 8fb17aeb51
commit e71f5e0fae
1 changed files with 394 additions and 399 deletions
Download patch file Download diff file Expand all files Collapse all files

101
README.md
View file

@ -27,7 +27,8 @@ it gets - learning from your data to provide increasingly relevant results and c
- **Run Everywhere** - Works in browsers, Node.js, serverless functions, and containers
- **Vector Search** - Find semantically similar content using embeddings
- **Advanced JSON Document Search** - Search within specific fields of JSON documents with field prioritization and service-based field standardization
- **Advanced JSON Document Search** - Search within specific fields of JSON documents with field prioritization and
service-based field standardization
- **Graph Relationships** - Connect data with meaningful relationships
- **Streaming Pipeline** - Process data in real-time as it flows through the system
- **Extensible Augmentations** - Customize and extend functionality with pluggable components
@ -434,6 +435,7 @@ const verbTypeMap = getVerbTypeMap() // { RelatedTo: 'relatedTo', Contains: 'con
```
These utility functions make it easy to:
- Get a complete list of available noun and verb types
- Validate user input against valid types
- Create dynamic UI components that display or select from available types
@ -534,7 +536,9 @@ const restoreResult = await db.restore(backupData, {clearExisting: true})
### Database Statistics
Brainy provides a way to get statistics about the current state of the database. For detailed information about the statistics system, including implementation details, scalability improvements, and usage examples, see our [Statistics Guide](STATISTICS.md).
Brainy provides a way to get statistics about the current state of the database. For detailed information about the
statistics system, including implementation details, scalability improvements, and usage examples, see
our [Statistics Guide](STATISTICS.md).
```typescript
import { BrainyData, getStatistics } from '@soulcraft/brainy'
@ -608,7 +612,8 @@ const authorResults = await db.searchByStandardField("author", "johndoe", 10, {
### Field Standardization and Service Tracking
Brainy automatically tracks field names from JSON documents and associates them with the service that inserted the data. This enables powerful cross-service search capabilities:
Brainy automatically tracks field names from JSON documents and associates them with the service that inserted the data.
This enables powerful cross-service search capabilities:
```typescript
// Get all available field names organized by service
@ -698,8 +703,10 @@ db.setReadOnly(false)
db.setWriteOnly(false)
```
- **Read-Only Mode**: When enabled, prevents all write operations (add, update, delete). Useful for deployment scenarios where you want to prevent modifications to the database.
- **Write-Only Mode**: When enabled, prevents all search operations. Useful for initial data loading or when you want to optimize for write performance.
- **Read-Only Mode**: When enabled, prevents all write operations (add, update, delete). Useful for deployment scenarios
where you want to prevent modifications to the database.
- **Write-Only Mode**: When enabled, prevents all search operations. Useful for initial data loading or when you want to
optimize for write performance.
### Embedding
@ -1410,7 +1417,9 @@ terabyte-scale data that can't fit entirely in memory, we provide several approa
- **Distributed HNSW**: Sharding and partitioning across multiple machines
- **Hybrid Solutions**: Combining quantization techniques with multi-tier architectures
For detailed information on how to scale Brainy for large datasets, vector dimension standardization, threading implementation, storage testing, and other technical topics, see our comprehensive [Technical Guides](TECHNICAL_GUIDES.md).
For detailed information on how to scale Brainy for large datasets, vector dimension standardization, threading
implementation, storage testing, and other technical topics, see our
comprehensive [Technical Guides](TECHNICAL_GUIDES.md).
## Recent Changes and Performance Improvements
@ -1418,7 +1427,9 @@ For detailed information on how to scale Brainy for large datasets, vector dimen
Brainy has been significantly improved to handle larger datasets more efficiently:
- **Pagination Support**: All data retrieval methods now support pagination to avoid loading entire datasets into memory at once. The deprecated `getAllNouns()` and `getAllVerbs()` methods have been replaced with `getNouns()` and `getVerbs()` methods that support pagination, filtering, and cursor-based navigation.
- **Pagination Support**: All data retrieval methods now support pagination to avoid loading entire datasets into memory
at once. The deprecated `getAllNouns()` and `getAllVerbs()` methods have been replaced with `getNouns()` and
`getVerbs()` methods that support pagination, filtering, and cursor-based navigation.
- **Multi-level Caching**: A sophisticated three-level caching strategy has been implemented:
- **Level 1**: Hot cache (most accessed nodes) - RAM (automatically detecting and adjusting in each environment)
@ -1429,13 +1440,16 @@ Brainy has been significantly improved to handle larger datasets more efficientl
- In Node.js: Uses 10% of free memory (minimum 1000 entries)
- In browsers: Scales based on device memory (500 entries per GB, minimum 1000)
- **Intelligent Cache Eviction**: Implements a Least Recently Used (LRU) policy that evicts the oldest 20% of items when the cache reaches the configured threshold.
- **Intelligent Cache Eviction**: Implements a Least Recently Used (LRU) policy that evicts the oldest 20% of items when
the cache reaches the configured threshold.
- **Prefetching Strategy**: Implements batch prefetching to improve performance while avoiding overwhelming system resources.
- **Prefetching Strategy**: Implements batch prefetching to improve performance while avoiding overwhelming system
resources.
### S3-Compatible Storage Improvements
- **Enhanced Cloud Storage**: Improved support for S3-compatible storage services including AWS S3, Cloudflare R2, and others.
- **Enhanced Cloud Storage**: Improved support for S3-compatible storage services including AWS S3, Cloudflare R2, and
others.
- **Optimized Data Access**: Batch operations and error handling for efficient cloud storage access.
@ -1445,9 +1459,11 @@ Brainy has been significantly improved to handle larger datasets more efficientl
Yes, you can use existing data indexed from an old version. Brainy includes robust data migration capabilities:
- **Vector Regeneration**: If vectors are missing in imported data, they will be automatically created using the embedding function.
- **Vector Regeneration**: If vectors are missing in imported data, they will be automatically created using the
embedding function.
- **HNSW Index Reconstruction**: The system can reconstruct the HNSW index from backup data, ensuring compatibility with previous versions.
- **HNSW Index Reconstruction**: The system can reconstruct the HNSW index from backup data, ensuring compatibility with
previous versions.
- **Sparse Data Import**: Support for importing sparse data (without vectors) through the `importSparseData()` method.
@ -1487,7 +1503,8 @@ Read-only mode prevents all write operations (add, update, delete) and is optimi
#### Write-Only Mode
Write-only mode prevents all search operations and is optimized for initial data loading or when you want to optimize for write performance.
Write-only mode prevents all search operations and is optimized for initial data loading or when you want to optimize
for write performance.
- **Memory**:
- Minimum: 512MB RAM
@ -1503,7 +1520,9 @@ Write-only mode prevents all search operations and is optimized for initial data
### Performance Tuning Parameters
Brainy offers comprehensive configuration options for performance tuning, with enhanced support for large datasets in S3 or other remote storage. **All configuration is optional** - the system automatically detects the optimal settings based on your environment, dataset size, and usage patterns.
Brainy offers comprehensive configuration options for performance tuning, with enhanced support for large datasets in S3
or other remote storage. **All configuration is optional** - the system automatically detects the optimal settings based
on your environment, dataset size, and usage patterns.
#### Intelligent Defaults
@ -1577,11 +1596,13 @@ const brainy = new BrainyData({
});
```
These configuration options make Brainy more efficient, scalable, and adaptable to different environments and usage patterns, especially for large datasets in cloud storage.
These configuration options make Brainy more efficient, scalable, and adaptable to different environments and usage
patterns, especially for large datasets in cloud storage.
## Testing
Brainy uses Vitest for testing. For detailed information about testing in Brainy, including test configuration, scripts, reporting tools, and best practices, see our [Testing Guide](TESTING.md).
Brainy uses Vitest for testing. For detailed information about testing in Brainy, including test configuration, scripts,
reporting tools, and best practices, see our [Testing Guide](TESTING.md).
Here are some common test commands:
@ -1605,45 +1626,18 @@ see [DEVELOPERS.md](DEVELOPERS.md).
We have a [Code of Conduct](CODE_OF_CONDUCT.md) that all contributors are expected to follow.
## Release Workflow
Brainy uses a streamlined release workflow that automates version updates, changelog generation, GitHub releases, and NPM deployment.
### Automated Release Process
The release workflow combines several steps into a single command:
1. **Build the project** - Ensures the code compiles correctly
2. **Run tests** - Verifies that all tests pass
3. **Update version** - Bumps the version number (patch, minor, or major)
4. **Generate changelog** - Automatically updates CHANGELOG.md with commit messages since the last release
5. **Create GitHub release** - Creates a GitHub release with auto-generated notes
6. **Publish to NPM** - Deploys the package to NPM
### Release Commands
Use one of the following commands to release a new version:
```bash
# Release with patch version update (0.0.x)
npm run workflow:patch
# Release with minor version update (0.x.0)
npm run workflow:minor
# Release with major version update (x.0.0)
npm run workflow:major
# Default workflow (same as patch)
npm run workflow
# Dry run (build, test, and simulate version update without making changes)
npm run workflow:dry-run
```
### Commit Message Format
For best results with automatic changelog generation, follow the [Conventional Commits](https://www.conventionalcommits.org/) specification for your commit messages:
For best results with automatic changelog generation, follow
the [Conventional Commits](https://www.conventionalcommits.org/) specification for your commit messages:
```
AI Template for automated commit messages:
Use Conventional Commit format
Specify the changes in a structured format
Add information about the purpose of the commit
```
```
<type>(<scope>): <description>
@ -1654,6 +1648,7 @@ For best results with automatic changelog generation, follow the [Conventional C
```
Where `<type>` is one of:
- `feat`: A new feature (maps to **Added** section)
- `fix`: A bug fix (maps to **Fixed** section)
- `chore`: Regular maintenance tasks (maps to **Changed** section)