Compare commits
16 commits
feat/warm-
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| fc9f0d7222 | |||
| d9cc7b9024 | |||
| edf123a5e2 | |||
| 5b2cbf74e5 | |||
| 003e2a74ea | |||
| 22702b81c0 | |||
| 415e824a1d | |||
| 069a889489 | |||
| d918c060f4 | |||
| 4c8e384bc8 | |||
| 9a99a7b962 | |||
| 6ba94c8c43 | |||
| 3a1efc9460 | |||
| 3be4ba96c2 | |||
| 55b867c998 | |||
| 9a5a9cccbc |
23 changed files with 841 additions and 411 deletions
17
CHANGELOG.md
17
CHANGELOG.md
|
|
@ -2,6 +2,23 @@
|
||||||
|
|
||||||
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
|
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
|
||||||
|
|
||||||
|
### [8.10.1](https://source.soulcraft.com/soulcraft/brainy/compare/v8.10.0...v8.10.1) (2026-07-24)
|
||||||
|
|
||||||
|
- refactor: remove the orphaned transaction-result type left behind by the dead-path removal (edf123a5)
|
||||||
|
- fix: warm() metadata surface routes through the active provider (warm hook added to the metadata contract); add maintenanceDebt() observability surface (5b2cbf74)
|
||||||
|
- fix: transaction timeouts are a typed no-hot-retry contract; engine-side non-retry pinned; dead transaction path removed (003e2a74)
|
||||||
|
- chore: the forge is the address — retire the archived mirror from every live surface (22702b81)
|
||||||
|
|
||||||
|
|
||||||
|
### [8.10.0](https://github.com/soulcraftlabs/brainy/compare/v8.9.0...v8.10.0) (2026-07-23)
|
||||||
|
|
||||||
|
- docs: adoption storefront — contributing guide, security policy, README support + cor section (9a99a7b)
|
||||||
|
- fix(release): push the public mirror explicitly and verify the tag lands at the right commit before publishing (6ba94c8)
|
||||||
|
- docs: project guide version line points at npm instead of a hardcoded stale number (3a1efc9)
|
||||||
|
- feat: vector provider identity is a required name field (hnsw-js), rendered [vector-index:<name>] (3be4ba9)
|
||||||
|
- feat: warm contract (warm/warmOnOpen/provider warm hook), configurable transact budget floor, backend-neutral vector index op names (55b867c)
|
||||||
|
|
||||||
|
|
||||||
### [8.9.0](https://github.com/soulcraftlabs/brainy/compare/v8.8.2...v8.9.0) (2026-07-19)
|
### [8.9.0](https://github.com/soulcraftlabs/brainy/compare/v8.8.2...v8.9.0) (2026-07-19)
|
||||||
|
|
||||||
- docs: measured performance envelopes v1 (per-op p50/p95 at 1k and 10k, pure-JS floor) (5cabd78)
|
- docs: measured performance envelopes v1 (per-op p50/p95 at 1k and 10k, pure-JS floor) (5cabd78)
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ Handoff file: `/home/dpsifr/.strategy/PLATFORM-HANDOFF.md`
|
||||||
|
|
||||||
**Brainy's current open actions:** None. MIT open-source — no platform-specific actions.
|
**Brainy's current open actions:** None. MIT open-source — no platform-specific actions.
|
||||||
|
|
||||||
**Current version:** `@soulcraft/brainy@7.31.5` (latest published; 8.0.0 release candidate on `feat/8.0-u64-ids`)
|
**Current version:** run `npm view @soulcraft/brainy version` (never trust a hardcoded number here — this line went stale for months); consumer-facing changes tracked in `RELEASES.md`
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
|
||||||
318
CONTRIBUTING.md
318
CONTRIBUTING.md
|
|
@ -1,298 +1,66 @@
|
||||||
# Contributing to Brainy
|
# Contributing to Brainy
|
||||||
|
|
||||||
Thank you for your interest in contributing to Brainy! This document provides guidelines and instructions for contributing to the project.
|
Brainy is MIT-licensed and genuinely open to outside contributions. This page
|
||||||
|
is the honest, current path — please don't rely on older instructions you
|
||||||
|
may find elsewhere in the repo's history.
|
||||||
|
|
||||||
## Code of Conduct
|
## Where the project lives
|
||||||
|
|
||||||
By participating in this project, you agree to abide by our Code of Conduct:
|
The source of truth is a self-hosted forge: **source.soulcraft.com/soulcraft/brainy**.
|
||||||
- Be respectful and inclusive
|
It's anonymously readable and cloneable — no account needed to browse, clone,
|
||||||
- Welcome newcomers and help them get started
|
or build.
|
||||||
- Focus on constructive criticism
|
|
||||||
- Respect differing viewpoints and experiences
|
|
||||||
|
|
||||||
## How to Contribute
|
## How to contribute
|
||||||
|
|
||||||
### Reporting Issues
|
**Found a bug, or have an idea?** Email **brainy@soulcraft.com**. No account,
|
||||||
|
no ceremony — you'll get a receipt, and it goes to a human.
|
||||||
|
|
||||||
Before creating an issue, please check existing issues to avoid duplicates.
|
**Want to send a patch?** Two ways, both first-class:
|
||||||
|
|
||||||
When creating an issue, include:
|
- **Email a patch.** Run `git format-patch` against your change and email the
|
||||||
- Clear, descriptive title
|
output to **brainy@soulcraft.com**. This is a genuinely supported path, not
|
||||||
- Detailed description of the problem
|
a fallback — plenty of good contributions arrive this way.
|
||||||
- Steps to reproduce
|
- **Open a pull request on the forge.** Request an account at
|
||||||
- Expected vs actual behavior
|
**source.soulcraft.com** (registration is request-with-approval, so allow
|
||||||
- System information (OS, Node version, Brainy version)
|
a little lag), clone, push a branch, and open a PR there. Maintainers
|
||||||
- Code examples if applicable
|
review and land it.
|
||||||
|
|
||||||
### Suggesting Features
|
Either way, for anything beyond a small fix, opening an issue first (email is
|
||||||
|
fine) to talk through the approach saves everyone rework.
|
||||||
|
|
||||||
Feature requests are welcome! Please provide:
|
## Development setup
|
||||||
- Clear use case
|
|
||||||
- Proposed API/interface
|
|
||||||
- Examples of how it would work
|
|
||||||
- Any potential challenges or considerations
|
|
||||||
|
|
||||||
### Pull Requests
|
|
||||||
|
|
||||||
#### Before Starting
|
|
||||||
|
|
||||||
1. Check existing issues and PRs
|
|
||||||
2. Open an issue to discuss significant changes
|
|
||||||
3. Fork the repository
|
|
||||||
4. Create a feature branch from `main`
|
|
||||||
|
|
||||||
#### Development Setup
|
|
||||||
|
|
||||||
**Quick Setup (Recommended):**
|
|
||||||
```bash
|
```bash
|
||||||
# Clone your fork
|
git clone https://source.soulcraft.com/soulcraft/brainy.git
|
||||||
git clone https://github.com/your-username/brainy.git
|
|
||||||
cd brainy
|
cd brainy
|
||||||
|
|
||||||
# Run setup script (installs all dependencies including Rust)
|
|
||||||
./scripts/setup-dev.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
**Manual Setup:**
|
|
||||||
```bash
|
|
||||||
# Clone your fork
|
|
||||||
git clone https://github.com/your-username/brainy.git
|
|
||||||
cd brainy
|
|
||||||
|
|
||||||
# Install system dependencies (Ubuntu/Debian)
|
|
||||||
sudo apt-get install -y build-essential pkg-config libssl-dev
|
|
||||||
|
|
||||||
# Install Rust (for WASM embedding engine)
|
|
||||||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
|
||||||
source ~/.cargo/env
|
|
||||||
rustup target add wasm32-unknown-unknown
|
|
||||||
cargo install wasm-pack
|
|
||||||
|
|
||||||
# Install Node.js dependencies
|
|
||||||
npm install
|
npm install
|
||||||
|
|
||||||
# Build Candle WASM embedding engine
|
|
||||||
npm run build:candle
|
|
||||||
|
|
||||||
# Build TypeScript
|
|
||||||
npm run build
|
npm run build
|
||||||
|
|
||||||
# Run tests
|
|
||||||
npm test
|
npm test
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Making Changes
|
Tests run on [Vitest](https://vitest.dev/). `npm test` runs the unit suite;
|
||||||
|
see `package.json` for `test:integration`, `test:coverage`, and friends.
|
||||||
|
|
||||||
1. **Follow the code style**
|
## Standards
|
||||||
- TypeScript for all source code
|
|
||||||
- Clear variable and function names
|
|
||||||
- Comments for complex logic
|
|
||||||
- JSDoc for public APIs
|
|
||||||
|
|
||||||
2. **Write tests**
|
- **Strict TypeScript.** No `any` escape hatches to dodge the type checker.
|
||||||
- Add tests for new features
|
- **Tests exercise real behavior.** No mocking away the thing you're supposed
|
||||||
- Update tests for changes
|
to be testing.
|
||||||
- Ensure all tests pass
|
- **No stubs, no TODO-code.** If something can't be finished, say so and
|
||||||
|
leave it out — don't merge a placeholder.
|
||||||
|
- **JSDoc on every exported function, class, and type.**
|
||||||
|
- **[Conventional Commits](https://www.conventionalcommits.org/).** `feat:`,
|
||||||
|
`fix:`, `docs:`, `perf:`, `refactor:`, `test:`, `chore:`. Never
|
||||||
|
`BREAKING CHANGE` in a commit message — major version bumps are a separate,
|
||||||
|
deliberate decision.
|
||||||
|
- **Performance claims are measured or labeled projected.** If a PR or its
|
||||||
|
description states a number, cite the benchmark that produced it (see
|
||||||
|
[docs/performance-envelopes.md](docs/performance-envelopes.md) for the
|
||||||
|
pattern). Don't state an estimate as if it were measured.
|
||||||
|
|
||||||
3. **Update documentation**
|
## License
|
||||||
- Update README if needed
|
|
||||||
- Add/update API documentation
|
|
||||||
- Include examples
|
|
||||||
|
|
||||||
#### Commit Guidelines
|
Brainy is [MIT licensed](LICENSE). Contributions are accepted under the same
|
||||||
|
license — there's no CLA to sign.
|
||||||
|
|
||||||
Follow conventional commits format:
|
Thank you for considering a contribution.
|
||||||
|
|
||||||
```
|
|
||||||
type(scope): description
|
|
||||||
|
|
||||||
[optional body]
|
|
||||||
|
|
||||||
[optional footer]
|
|
||||||
```
|
|
||||||
|
|
||||||
Types:
|
|
||||||
- `feat`: New feature
|
|
||||||
- `fix`: Bug fix
|
|
||||||
- `docs`: Documentation changes
|
|
||||||
- `style`: Code style changes
|
|
||||||
- `refactor`: Code refactoring
|
|
||||||
- `perf`: Performance improvements
|
|
||||||
- `test`: Test changes
|
|
||||||
- `chore`: Build/tooling changes
|
|
||||||
|
|
||||||
Examples:
|
|
||||||
```bash
|
|
||||||
feat(triple): add graph traversal depth limit
|
|
||||||
fix(storage): handle concurrent write conflicts
|
|
||||||
docs(api): update search method documentation
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Submitting PR
|
|
||||||
|
|
||||||
1. Push to your fork
|
|
||||||
2. Create PR against `main` branch
|
|
||||||
3. Fill out PR template
|
|
||||||
4. Ensure CI checks pass
|
|
||||||
5. Wait for review
|
|
||||||
|
|
||||||
### Testing
|
|
||||||
|
|
||||||
#### Running Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Run all tests
|
|
||||||
npm test
|
|
||||||
|
|
||||||
# Run specific test file
|
|
||||||
npm test tests/core.test.ts
|
|
||||||
|
|
||||||
# Run with coverage
|
|
||||||
npm run test:coverage
|
|
||||||
|
|
||||||
# Watch mode
|
|
||||||
npm run test:watch
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Writing Tests
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import { describe, it, expect } from 'vitest'
|
|
||||||
import { Brainy } from '../src'
|
|
||||||
|
|
||||||
describe('Feature Name', () => {
|
|
||||||
it('should do something specific', async () => {
|
|
||||||
const brain = new Brainy()
|
|
||||||
await brain.init()
|
|
||||||
|
|
||||||
// Test implementation
|
|
||||||
const result = await brain.search("test")
|
|
||||||
|
|
||||||
expect(result).toBeDefined()
|
|
||||||
expect(result.length).toBeGreaterThan(0)
|
|
||||||
})
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
## Architecture Guidelines
|
|
||||||
|
|
||||||
### Adding New Features
|
|
||||||
|
|
||||||
1. **Check existing functionality**
|
|
||||||
- Review `ARCHITECTURE.md`
|
|
||||||
- Check if similar features exist
|
|
||||||
- Consider if it should be an augmentation
|
|
||||||
|
|
||||||
2. **Design considerations**
|
|
||||||
- Maintain backward compatibility
|
|
||||||
- Consider performance impact
|
|
||||||
- Think about all storage adapters
|
|
||||||
- Plan for extensibility
|
|
||||||
|
|
||||||
3. **Implementation checklist**
|
|
||||||
- [ ] Core functionality
|
|
||||||
- [ ] Tests (unit and integration)
|
|
||||||
- [ ] Documentation
|
|
||||||
- [ ] TypeScript types
|
|
||||||
- [ ] Examples
|
|
||||||
- [ ] Performance benchmarks (if applicable)
|
|
||||||
|
|
||||||
### Creating Augmentations
|
|
||||||
|
|
||||||
Augmentations extend Brainy's functionality:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import { BrainyAugmentation } from '../types'
|
|
||||||
|
|
||||||
export class MyAugmentation extends BrainyAugmentation {
|
|
||||||
name = 'MyAugmentation'
|
|
||||||
|
|
||||||
async onInit(brain: Brainy): Promise<void> {
|
|
||||||
// Initialize augmentation
|
|
||||||
}
|
|
||||||
|
|
||||||
async onAdd(item: any, brain: Brainy): Promise<any> {
|
|
||||||
// Process before adding
|
|
||||||
return item
|
|
||||||
}
|
|
||||||
|
|
||||||
async onSearch(query: any, results: any[], brain: Brainy): Promise<any[]> {
|
|
||||||
// Process search results
|
|
||||||
return results
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Performance Considerations
|
|
||||||
|
|
||||||
- Use batch operations where possible
|
|
||||||
- Implement caching strategically
|
|
||||||
- Consider memory usage
|
|
||||||
- Profile performance impacts
|
|
||||||
- Add benchmarks for critical paths
|
|
||||||
|
|
||||||
## Documentation
|
|
||||||
|
|
||||||
### API Documentation
|
|
||||||
|
|
||||||
Use JSDoc for all public APIs:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
/**
|
|
||||||
* Searches for similar items using vector similarity
|
|
||||||
* @param query - Search query (text or vector)
|
|
||||||
* @param options - Search options
|
|
||||||
* @returns Array of search results with scores
|
|
||||||
* @example
|
|
||||||
* ```typescript
|
|
||||||
* const results = await brain.search("machine learning", { limit: 10 })
|
|
||||||
* ```
|
|
||||||
*/
|
|
||||||
async search(query: string | Vector, options?: SearchOptions): Promise<SearchResult[]> {
|
|
||||||
// Implementation
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Examples
|
|
||||||
|
|
||||||
Add examples for new features:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// examples/feature-name.ts
|
|
||||||
import { Brainy } from 'brainy'
|
|
||||||
|
|
||||||
async function exampleUsage() {
|
|
||||||
const brain = new Brainy()
|
|
||||||
await brain.init()
|
|
||||||
|
|
||||||
// Show feature usage
|
|
||||||
// Include comments explaining what's happening
|
|
||||||
// Handle errors appropriately
|
|
||||||
}
|
|
||||||
|
|
||||||
exampleUsage().catch(console.error)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Release Process
|
|
||||||
|
|
||||||
1. **Version bump**: Follow semantic versioning
|
|
||||||
2. **Update CHANGELOG**: Document all changes
|
|
||||||
3. **Run tests**: Ensure all tests pass
|
|
||||||
4. **Build**: Generate distribution files
|
|
||||||
5. **Tag**: Create git tag for version
|
|
||||||
6. **Publish**: Release to npm
|
|
||||||
|
|
||||||
## Getting Help
|
|
||||||
|
|
||||||
- **Discord**: Join our community
|
|
||||||
- **Issues**: Ask questions on GitHub
|
|
||||||
- **Discussions**: Share ideas and get feedback
|
|
||||||
|
|
||||||
## Recognition
|
|
||||||
|
|
||||||
Contributors will be recognized in:
|
|
||||||
- CHANGELOG.md for their contributions
|
|
||||||
- README.md contributors section
|
|
||||||
- GitHub contributors page
|
|
||||||
|
|
||||||
Thank you for contributing to Brainy! 🧠
|
|
||||||
|
|
|
||||||
26
README.md
26
README.md
|
|
@ -13,7 +13,7 @@
|
||||||
<p align="center">
|
<p align="center">
|
||||||
<a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/v/@soulcraft/brainy.svg" alt="npm version"></a>
|
<a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/v/@soulcraft/brainy.svg" alt="npm version"></a>
|
||||||
<a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/dm/@soulcraft/brainy.svg" alt="npm downloads"></a>
|
<a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/dm/@soulcraft/brainy.svg" alt="npm downloads"></a>
|
||||||
<a href="https://github.com/soulcraftlabs/brainy/actions/workflows/ci.yml"><img src="https://github.com/soulcraftlabs/brainy/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
|
<a href="https://source.soulcraft.com/soulcraft/brainy/actions"><img src="https://source.soulcraft.com/soulcraft/brainy/actions/workflows/ci.yml/badge.svg?branch=main" alt="CI"></a>
|
||||||
<a href="https://soulcraft.com/docs"><img src="https://img.shields.io/badge/docs-soulcraft.com-blue.svg" alt="Documentation"></a>
|
<a href="https://soulcraft.com/docs"><img src="https://img.shields.io/badge/docs-soulcraft.com-blue.svg" alt="Documentation"></a>
|
||||||
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License"></a>
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License"></a>
|
||||||
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/%3C%2F%3E-TypeScript-%230074c1.svg" alt="TypeScript"></a>
|
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/%3C%2F%3E-TypeScript-%230074c1.svg" alt="TypeScript"></a>
|
||||||
|
|
@ -23,8 +23,9 @@
|
||||||
<a href="#quick-start">Quick start</a> ·
|
<a href="#quick-start">Quick start</a> ·
|
||||||
<a href="#one-query-three-engines">One query</a> ·
|
<a href="#one-query-three-engines">One query</a> ·
|
||||||
<a href="#feature-tour">Features</a> ·
|
<a href="#feature-tour">Features</a> ·
|
||||||
<a href="#from-laptop-to-hundreds-of-millions">Scale with Cor</a> ·
|
<a href="#when-you-outgrow-brainy">Scale with Cor</a> ·
|
||||||
<a href="#documentation">Docs</a>
|
<a href="#documentation">Docs</a> ·
|
||||||
|
<a href="#support--community">Support</a>
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
@ -172,9 +173,11 @@ await brain.vfs.search('React components with hooks') // semantic file
|
||||||
|
|
||||||
**[Multi-process model](docs/concepts/multi-process.md)** · **[Inspection guide](docs/guides/inspection.md)**
|
**[Multi-process model](docs/concepts/multi-process.md)** · **[Inspection guide](docs/guides/inspection.md)**
|
||||||
|
|
||||||
## From laptop to hundreds of millions
|
## When you outgrow Brainy
|
||||||
|
|
||||||
Brainy's TypeScript engines take you a long way. When you outgrow them, add the native engine — **the API doesn't change**:
|
Brainy's pure-TypeScript engines carry real workloads a long way on their own — see the measured, per-operation numbers (not marketing figures) in **[docs/performance-envelopes.md](docs/performance-envelopes.md)** for what to expect, unaccelerated, on plain filesystem storage.
|
||||||
|
|
||||||
|
When a deployment needs native-scale vector/graph performance — memory-mapped indexes that don't need your dataset in RAM, billion-scale ambitions — add the native engine. **The API doesn't change:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm install @soulcraft/cor
|
npm install @soulcraft/cor
|
||||||
|
|
@ -187,13 +190,14 @@ await brain.init() // @soulcraft/cor detected — same code, native engines un
|
||||||
|
|
||||||
Installing the package is the opt-in: if `@soulcraft/cor` is present, it loads and announces itself in the init log; if it's present but broken, `init()` **throws** — an installed accelerator never silently vanishes behind the JS engines. Opt out with `plugins: []`, or pin exactly what loads with `plugins: ['@soulcraft/cor']`. [`@soulcraft/cor`](https://www.npmjs.com/package/@soulcraft/cor) (Brainy 8.x ↔ Cor 3.x, version-matched) registers Rust implementations behind every provider seam: SIMD distance kernels, memory-mapped storage, a disk-native vector index that doesn't need your dataset in RAM, durable LSM field/graph indexes that serve cold opens instantly, and native aggregation. Recall@10 measured **0.99 / 0.96 / 0.96 at 1M / 10M / 100M vectors** in Cor's release gate.
|
Installing the package is the opt-in: if `@soulcraft/cor` is present, it loads and announces itself in the init log; if it's present but broken, `init()` **throws** — an installed accelerator never silently vanishes behind the JS engines. Opt out with `plugins: []`, or pin exactly what loads with `plugins: ['@soulcraft/cor']`. [`@soulcraft/cor`](https://www.npmjs.com/package/@soulcraft/cor) (Brainy 8.x ↔ Cor 3.x, version-matched) registers Rust implementations behind every provider seam: SIMD distance kernels, memory-mapped storage, a disk-native vector index that doesn't need your dataset in RAM, durable LSM field/graph indexes that serve cold opens instantly, and native aggregation. Recall@10 measured **0.99 / 0.96 / 0.96 at 1M / 10M / 100M vectors** in Cor's release gate.
|
||||||
|
|
||||||
Open core, commercial accelerator: Brainy is MIT and complete on its own; Cor is licensed and funds both.
|
Open core, commercial accelerator: Brainy is MIT and complete on its own — Cor is more headroom for when you need it, not capability held back to sell you later. Licensing and support: **cor@soulcraft.com**.
|
||||||
|
|
||||||
## Performance
|
## Performance
|
||||||
|
|
||||||
|
- Per-operation p50/p95 at 1k and 10k entities, pure-JS floor, measured and re-run every release that touches a measured path: **[docs/performance-envelopes.md](docs/performance-envelopes.md)**.
|
||||||
- JS distance kernels: **~6× faster cosine, ~1.4× euclidean** than 7.x (measured: [`tests/benchmarks/distance-microbench.mjs`](tests/benchmarks/distance-microbench.mjs), 384-dim, median of 41).
|
- JS distance kernels: **~6× faster cosine, ~1.4× euclidean** than 7.x (measured: [`tests/benchmarks/distance-microbench.mjs`](tests/benchmarks/distance-microbench.mjs), 384-dim, median of 41).
|
||||||
- Whole-graph reads are single **O(N + E)** cursor walks — a consumer-measured 19k-edge export dropped from ~27 s of per-node calls to one scan.
|
- Whole-graph reads are single **O(N + E)** cursor walks — a consumer-measured 19k-edge export dropped from ~27 s of per-node calls to one scan.
|
||||||
- Full numbers and capacity planning: **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)** · **[docs/SCALING.md](docs/SCALING.md)**
|
- Capacity planning and architecture: **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)** · **[docs/SCALING.md](docs/SCALING.md)**
|
||||||
|
|
||||||
## Use cases
|
## Use cases
|
||||||
|
|
||||||
|
|
@ -212,6 +216,10 @@ Open core, commercial accelerator: Brainy is MIT and complete on its own; Cor is
|
||||||
|
|
||||||
**Bun ≥ 1.1** (recommended) or **Node.js ≥ 22**. Brainy 8.x is server-only; the 7.x line remains on npm for browser use.
|
**Bun ≥ 1.1** (recommended) or **Node.js ≥ 22**. Brainy 8.x is server-only; the 7.x line remains on npm for browser use.
|
||||||
|
|
||||||
## Contributing & license
|
## Support & community
|
||||||
|
|
||||||
Contributions welcome — see **[CONTRIBUTING.md](CONTRIBUTING.md)**. MIT © Brainy Contributors.
|
- **Bugs and ideas** → **brainy@soulcraft.com** — no account needed, you'll get a receipt.
|
||||||
|
- **Security reports** → **security@soulcraft.com** — see **[SECURITY.md](SECURITY.md)**.
|
||||||
|
- **Contributing** → see **[CONTRIBUTING.md](CONTRIBUTING.md)**.
|
||||||
|
|
||||||
|
MIT © Brainy Contributors.
|
||||||
|
|
|
||||||
62
RELEASES.md
62
RELEASES.md
|
|
@ -31,6 +31,68 @@ is sometimes cited as a 7.x removal — those methods never existed on 7.x; the
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## v8.10.1 — 2026-07-24 (the no-hot-retry contract + warm()'s metadata surface under native providers)
|
||||||
|
|
||||||
|
From a production incident: a native-provider op ground 38-40s inside a transaction,
|
||||||
|
blew the ~32s apply-phase budget, was rolled back (zero loss, by design), and a
|
||||||
|
downstream pipeline hot-retried the identical operation into a 6-minute, 100%-CPU
|
||||||
|
storm. Investigation confirmed Brainy itself never auto-retries a timed-out
|
||||||
|
transaction — the storm was entirely the consumer's own retry loop, driven by a
|
||||||
|
"retryable" doc-prose claim with no machine-readable contract to branch on. This
|
||||||
|
release closes that contract gap and, separately, fixes a real `warm()` reporting gap
|
||||||
|
surfaced by the same investigation.
|
||||||
|
|
||||||
|
- **`TransactionTimeoutError` is now a machine-readable no-hot-retry contract.** Two
|
||||||
|
new typed, always-`true` fields replace prose-only guidance:
|
||||||
|
- `retryable: true` — the operation MAY succeed on a later attempt, once the
|
||||||
|
underlying slowness resolves or the budget is deliberately raised
|
||||||
|
(`transactionBudgetFloorMs`, or a batch's own `timeoutMs` override).
|
||||||
|
- `hotRetryUnsafe: true` — an immediate, identical retry re-pays the FULL cost of
|
||||||
|
the work that just timed out (it does not resume partway) and can cascade into
|
||||||
|
exactly the CPU storm above. **Never loop on this error.** The documented pattern
|
||||||
|
is a latch, not a retry loop:
|
||||||
|
```
|
||||||
|
on TransactionTimeoutError:
|
||||||
|
record { at: Date.now(), error }
|
||||||
|
rethrow loudly to your own caller
|
||||||
|
hold a cooldown window before any re-attempt
|
||||||
|
clear the latch only on a subsequent success
|
||||||
|
```
|
||||||
|
- `context` (unchanged, now fully documented) carries the backoff inputs:
|
||||||
|
`timeoutMs`, `operationIndex`, `elapsedMs`, `totalOperations`, `operationName`.
|
||||||
|
- Every "retryable" doc-prose site referencing this error (`transact()`'s
|
||||||
|
`timeoutMs` option, `transactionBudgetFloorMs`, `Transaction.execute()`) now
|
||||||
|
points at these fields instead of bare prose.
|
||||||
|
- Regression-pinned: the engine never internally re-drives a timed-out operation
|
||||||
|
(verified via an execution counter through both the single-op write path and
|
||||||
|
`add()`'s upsert-race retry loop), so this has always been true — it is now
|
||||||
|
provable and typed.
|
||||||
|
- **Dead code removed**: `TransactionManager.executeTransactionWithResult()` had zero
|
||||||
|
callers in this codebase and is deleted.
|
||||||
|
- **`brain.warm()`'s metadata surface now routes through the ACTIVE provider.** A
|
||||||
|
production deployment's warm report showed `metadata: 'unavailable'` under a native
|
||||||
|
metadata provider — the previous logic only duck-typed the built-in JS manager's
|
||||||
|
`hydrateAll()` method, which a native provider has no reason to implement. The
|
||||||
|
metadata provider contract (`MetadataIndexProvider`, `src/plugin.ts`) gains an
|
||||||
|
optional `warm?(): Promise<void>` hook, mirroring the existing vector and graph
|
||||||
|
provider hooks. `brain.warm()` now checks the active provider's own `warm()` FIRST,
|
||||||
|
falls back to the JS manager's `hydrateAll()` when absent, and only reports
|
||||||
|
`'unavailable'` when neither exists — never `init()` as a stand-in, since a native
|
||||||
|
provider's `init()` may be a cheap verify rather than a real warm. A native
|
||||||
|
provider lights this surface up the same way `@soulcraft/cor` already lights the
|
||||||
|
vector and graph surfaces: implement `warm()` on its metadata provider.
|
||||||
|
- **New: `brain.maintenanceDebt()`** — the observability seam so an operator sees a
|
||||||
|
provider's outstanding background maintenance work (pending bytes/items, last pass
|
||||||
|
outcome, whether it's converging) BEFORE it grinds into the kind of budget-busting
|
||||||
|
op this release's timeout contract exists for, instead of discovering it as a CPU
|
||||||
|
storm. It is a pure passthrough: brainy applies no thresholds, no polling, and no
|
||||||
|
estimation — it calls each active provider's own optional `maintenanceDebt?()` hook
|
||||||
|
(vector, metadata, graph — the same three contracts `warm?()` lives on) and reports
|
||||||
|
the payload verbatim, or `'unavailable'` when a surface's provider doesn't track
|
||||||
|
debt. Useful as a pre-warm/post-warm check or a boot gate. `@soulcraft/cor` does not
|
||||||
|
yet implement the hook as of this release — expect it on cor's next release; until
|
||||||
|
then all three surfaces honestly report `'unavailable'`.
|
||||||
|
|
||||||
## Unreleased (the warm contract: cold-restart writes stop paying demand-load latency)
|
## Unreleased (the warm contract: cold-restart writes stop paying demand-load latency)
|
||||||
|
|
||||||
From a production deployment's cold-restart incident: the FIRST writes after every
|
From a production deployment's cold-restart incident: the FIRST writes after every
|
||||||
|
|
|
||||||
36
SECURITY.md
Normal file
36
SECURITY.md
Normal file
|
|
@ -0,0 +1,36 @@
|
||||||
|
# Security Policy
|
||||||
|
|
||||||
|
## Reporting a vulnerability
|
||||||
|
|
||||||
|
Email **security@soulcraft.com**. That's the one door for security reports
|
||||||
|
across the company, and it works the same way for Brainy: every report is
|
||||||
|
read by a human, you'll get a private receipt, and we'll work with you on
|
||||||
|
coordinated disclosure — please don't open a public issue for anything
|
||||||
|
that isn't already public.
|
||||||
|
|
||||||
|
Include what you'd want if you were on the other end: affected version,
|
||||||
|
how to reproduce, and what you think the impact is. If you have a patch or
|
||||||
|
a suggested fix, send it along — it's welcome but not required.
|
||||||
|
|
||||||
|
There is no bounty program today. We're saying that plainly so you know
|
||||||
|
what to expect going in.
|
||||||
|
|
||||||
|
## Response time
|
||||||
|
|
||||||
|
We respond as fast as truth allows. That means: no fixed SLA, no promise of
|
||||||
|
a reply within a specific number of hours — but a real report from a real
|
||||||
|
person gets read promptly and taken seriously. If you haven't heard anything
|
||||||
|
in a reasonable stretch, a follow-up email is completely fine.
|
||||||
|
|
||||||
|
## Supported versions
|
||||||
|
|
||||||
|
The latest `8.x` minor release line receives security fixes. If you're
|
||||||
|
running an older major version, please upgrade before reporting — we can't
|
||||||
|
commit to backporting fixes to unsupported lines.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
This policy covers the `@soulcraft/brainy` package itself — the code in
|
||||||
|
this repository. If you're evaluating a deployment that also uses
|
||||||
|
`@soulcraft/cor`, report issues in that package the same way, to the same
|
||||||
|
address; we'll route internally.
|
||||||
4
package-lock.json
generated
4
package-lock.json
generated
|
|
@ -1,12 +1,12 @@
|
||||||
{
|
{
|
||||||
"name": "@soulcraft/brainy",
|
"name": "@soulcraft/brainy",
|
||||||
"version": "8.9.0",
|
"version": "8.10.1",
|
||||||
"lockfileVersion": 3,
|
"lockfileVersion": 3,
|
||||||
"requires": true,
|
"requires": true,
|
||||||
"packages": {
|
"packages": {
|
||||||
"": {
|
"": {
|
||||||
"name": "@soulcraft/brainy",
|
"name": "@soulcraft/brainy",
|
||||||
"version": "8.9.0",
|
"version": "8.10.1",
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@msgpack/msgpack": "^3.1.2",
|
"@msgpack/msgpack": "^3.1.2",
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
{
|
{
|
||||||
"name": "@soulcraft/brainy",
|
"name": "@soulcraft/brainy",
|
||||||
"version": "8.9.0",
|
"version": "8.10.1",
|
||||||
"description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. Stage 3 CANONICAL: 42 nouns × 127 verbs covering 96-97% of all human knowledge.",
|
"description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. Stage 3 CANONICAL: 42 nouns × 127 verbs covering 96-97% of all human knowledge.",
|
||||||
"main": "dist/index.js",
|
"main": "dist/index.js",
|
||||||
"module": "dist/index.js",
|
"module": "dist/index.js",
|
||||||
|
|
@ -128,13 +128,13 @@
|
||||||
"publishConfig": {
|
"publishConfig": {
|
||||||
"access": "public"
|
"access": "public"
|
||||||
},
|
},
|
||||||
"homepage": "https://github.com/soulcraftlabs/brainy",
|
"homepage": "https://source.soulcraft.com/soulcraft/brainy",
|
||||||
"bugs": {
|
"bugs": {
|
||||||
"url": "https://github.com/soulcraftlabs/brainy/issues"
|
"url": "https://source.soulcraft.com/soulcraft/brainy/issues"
|
||||||
},
|
},
|
||||||
"repository": {
|
"repository": {
|
||||||
"type": "git",
|
"type": "git",
|
||||||
"url": "git+https://github.com/soulcraftlabs/brainy.git"
|
"url": "git+https://source.soulcraft.com/soulcraft/brainy.git"
|
||||||
},
|
},
|
||||||
"files": [
|
"files": [
|
||||||
"dist/**/*.js",
|
"dist/**/*.js",
|
||||||
|
|
|
||||||
|
|
@ -142,7 +142,7 @@ else
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Create new changelog entry
|
# Create new changelog entry
|
||||||
CHANGELOG_ENTRY="### [${NEW_VERSION}](https://github.com/soulcraftlabs/brainy/compare/v${CURRENT_VERSION}...v${NEW_VERSION}) ($(date +%Y-%m-%d))
|
CHANGELOG_ENTRY="### [${NEW_VERSION}](https://source.soulcraft.com/soulcraft/brainy/compare/v${CURRENT_VERSION}...v${NEW_VERSION}) ($(date +%Y-%m-%d))
|
||||||
|
|
||||||
${COMMITS}
|
${COMMITS}
|
||||||
"
|
"
|
||||||
|
|
@ -175,26 +175,59 @@ echo -e "${BLUE}7️⃣ Creating git tag v${NEW_VERSION}...${NC}"
|
||||||
git tag -a "v${NEW_VERSION}" -m "Release v${NEW_VERSION}"
|
git tag -a "v${NEW_VERSION}" -m "Release v${NEW_VERSION}"
|
||||||
echo -e "${GREEN}✅ Tag created${NC}\n"
|
echo -e "${GREEN}✅ Tag created${NC}\n"
|
||||||
|
|
||||||
# Step 9: Push to GitHub
|
# Step 9: Push to origin — the forge is the one home (ruled 2026-07-23; the
|
||||||
echo -e "${BLUE}8️⃣ Pushing to GitHub...${NC}"
|
# old public GitHub repo is archived history, no longer part of any release).
|
||||||
|
echo -e "${BLUE}8️⃣ Pushing to origin...${NC}"
|
||||||
git push --follow-tags origin "$CURRENT_BRANCH"
|
git push --follow-tags origin "$CURRENT_BRANCH"
|
||||||
echo -e "${GREEN}✅ Pushed to GitHub${NC}\n"
|
echo -e "${GREEN}✅ Pushed to origin${NC}\n"
|
||||||
|
|
||||||
# Step 10: Publish to npm
|
# Step 10: Publish — forge FIRST (home), npmjs second (the world's storefront).
|
||||||
echo -e "${BLUE}9️⃣ Publishing to npm (dist-tag: ${NPM_TAG})...${NC}"
|
# The fleet-wide ~/.npmrc maps the @soulcraft scope to the forge registry, and
|
||||||
npm publish --tag "$NPM_TAG"
|
# a scope mapping BEATS `--registry` on the command line — so each publish
|
||||||
# Brainy is the only PUBLIC @soulcraft package — verify visibility after every publish.
|
# names its registry via the scope override explicitly. Nothing implicit.
|
||||||
npm access get status @soulcraft/brainy || true
|
FORGE_NPM_REG="https://source.soulcraft.com/api/packages/soulcraft/npm/"
|
||||||
echo -e "${GREEN}✅ Published to npm${NC}\n"
|
FORGE_NPM_TOKEN_FILE="$HOME/.config/soulcraft/npm-publish-brainy.token"
|
||||||
|
echo -e "${BLUE}9️⃣ Publishing to the forge registry (home)...${NC}"
|
||||||
# Step 11: Create GitHub release
|
if [ -f "$FORGE_NPM_TOKEN_FILE" ]; then
|
||||||
echo -e "${BLUE}🔟 Creating GitHub release...${NC}"
|
TMPRC="$(mktemp)"
|
||||||
if [ "$PRERELEASE" = true ]; then
|
chmod 600 "$TMPRC"
|
||||||
gh release create "v${NEW_VERSION}" --generate-notes --prerelease
|
{
|
||||||
|
echo "@soulcraft:registry=${FORGE_NPM_REG}"
|
||||||
|
echo "//source.soulcraft.com/api/packages/soulcraft/npm/:_authToken=$(cat "$FORGE_NPM_TOKEN_FILE")"
|
||||||
|
} > "$TMPRC"
|
||||||
|
if npm publish --tag "$NPM_TAG" --userconfig "$TMPRC"; then
|
||||||
|
echo -e "${GREEN}✅ Published to the forge${NC}\n"
|
||||||
|
else
|
||||||
|
rm -f "$TMPRC"
|
||||||
|
echo -e "${RED}❌ Forge publish FAILED — aborting before npmjs so the pair never diverges. Fix and re-run.${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
rm -f "$TMPRC"
|
||||||
else
|
else
|
||||||
gh release create "v${NEW_VERSION}" --generate-notes
|
echo -e "${RED}❌ Forge publish token missing (${FORGE_NPM_TOKEN_FILE}) — aborting. The forge is home; publish it first or restage the token.${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo -e "${BLUE}9️⃣½ Publishing to npmjs (storefront, dist-tag: ${NPM_TAG})...${NC}"
|
||||||
|
npm publish --tag "$NPM_TAG" "--@soulcraft:registry=https://registry.npmjs.org/"
|
||||||
|
# Brainy is the only PUBLIC @soulcraft package — verify visibility after every publish.
|
||||||
|
npm access get status @soulcraft/brainy "--@soulcraft:registry=https://registry.npmjs.org/" || true
|
||||||
|
echo -e "${GREEN}✅ Published to npmjs${NC}\n"
|
||||||
|
|
||||||
|
# Step 11: Release object on the forge (presentational — the tag, CHANGELOG,
|
||||||
|
# and RELEASES.md are the record; this just gives the forge UI a release page).
|
||||||
|
echo -e "${BLUE}🔟 Creating forge release...${NC}"
|
||||||
|
if [ -n "${FORGEJO_RELEASE_TOKEN:-}" ]; then
|
||||||
|
if curl -sf -X POST "https://source.soulcraft.com/api/v1/repos/soulcraft/brainy/releases" \
|
||||||
|
-H "Authorization: token ${FORGEJO_RELEASE_TOKEN}" -H "Content-Type: application/json" \
|
||||||
|
-d "{\"tag_name\":\"v${NEW_VERSION}\",\"name\":\"v${NEW_VERSION}\",\"prerelease\":${PRERELEASE}}" >/dev/null; then
|
||||||
|
echo -e "${GREEN}✅ Forge release created${NC}\n"
|
||||||
|
else
|
||||||
|
echo -e "${RED}⚠️ Forge release API call failed — tag + CHANGELOG remain the record; create the release page via the forge UI if wanted${NC}\n"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo -e "${RED}⚠️ FORGEJO_RELEASE_TOKEN unset — no release page created; tag + CHANGELOG remain the record${NC}\n"
|
||||||
fi
|
fi
|
||||||
echo -e "${GREEN}✅ GitHub release created${NC}\n"
|
|
||||||
|
|
||||||
# Step 12: Push public docs to the soulcraft.com docs ingest door
|
# Step 12: Push public docs to the soulcraft.com docs ingest door
|
||||||
# (VENUE-DOCS-RELEASE-PUSH). Skips with a loud warning when
|
# (VENUE-DOCS-RELEASE-PUSH). Skips with a loud warning when
|
||||||
|
|
@ -213,4 +246,4 @@ echo -e "${GREEN}🎉 Release ${NEW_VERSION} complete!${NC}"
|
||||||
echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
|
echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
|
||||||
echo ""
|
echo ""
|
||||||
echo -e "📦 npm: ${BLUE}https://www.npmjs.com/package/@soulcraft/brainy/v/${NEW_VERSION}${NC}"
|
echo -e "📦 npm: ${BLUE}https://www.npmjs.com/package/@soulcraft/brainy/v/${NEW_VERSION}${NC}"
|
||||||
echo -e "🐙 GitHub: ${BLUE}https://github.com/soulcraftlabs/brainy/releases/tag/v${NEW_VERSION}${NC}"
|
echo -e "🏠 Forge: ${BLUE}https://source.soulcraft.com/soulcraft/brainy/releases/tag/v${NEW_VERSION}${NC}"
|
||||||
|
|
|
||||||
112
src/brainy.ts
112
src/brainy.ts
|
|
@ -65,7 +65,8 @@ import type {
|
||||||
OpaqueIdSet,
|
OpaqueIdSet,
|
||||||
AtGenerationVectors,
|
AtGenerationVectors,
|
||||||
VectorIndexProvider,
|
VectorIndexProvider,
|
||||||
GraphIndexProvider
|
GraphIndexProvider,
|
||||||
|
ProviderMaintenanceDebt
|
||||||
} from './plugin.js'
|
} from './plugin.js'
|
||||||
import type {
|
import type {
|
||||||
BrainyPlugin,
|
BrainyPlugin,
|
||||||
|
|
@ -424,6 +425,30 @@ export interface WarmReport {
|
||||||
totalDurationMs: number
|
totalDurationMs: number
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Result of {@link Brainy.maintenanceDebt}: one outcome per
|
||||||
|
* index surface, mirroring {@link WarmReport}'s shape.
|
||||||
|
* - `'reported'` — the active provider for this surface implements
|
||||||
|
* `maintenanceDebt?()` and its {@link ProviderMaintenanceDebt} payload is
|
||||||
|
* attached verbatim under `debt`.
|
||||||
|
* - `'unavailable'` — the active provider does not implement the hook, so
|
||||||
|
* nothing is known; brainy never estimates or infers a payload on its
|
||||||
|
* behalf.
|
||||||
|
*/
|
||||||
|
export type MaintenanceDebtOutcome = 'reported' | 'unavailable'
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Per-surface result of {@link Brainy.maintenanceDebt}. Brainy
|
||||||
|
* performs no thresholding, polling, or estimation over this data — it is a
|
||||||
|
* pure passthrough of each active provider's own self-report (the provider
|
||||||
|
* owns the numbers; the operator owns the policy).
|
||||||
|
*/
|
||||||
|
export interface MaintenanceDebtReport {
|
||||||
|
vector: { outcome: MaintenanceDebtOutcome; debt?: ProviderMaintenanceDebt }
|
||||||
|
metadata: { outcome: MaintenanceDebtOutcome; debt?: ProviderMaintenanceDebt }
|
||||||
|
graph: { outcome: MaintenanceDebtOutcome; debt?: ProviderMaintenanceDebt }
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* How long a failed aggregation-backfill walk suppresses fresh walk attempts.
|
* How long a failed aggregation-backfill walk suppresses fresh walk attempts.
|
||||||
* Within the window, queries rethrow the recorded failure instantly (loud,
|
* Within the window, queries rethrow the recorded failure instantly (loud,
|
||||||
|
|
@ -14352,9 +14377,16 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
* *some* backing storage as a side effect but is reported honestly as
|
* *some* backing storage as a side effect but is reported honestly as
|
||||||
* `'probed'`, never `'warmed'`. An empty index or unknown dimension has
|
* `'probed'`, never `'warmed'`. An empty index or unknown dimension has
|
||||||
* nothing to probe (`'unavailable'`).
|
* nothing to probe (`'unavailable'`).
|
||||||
* - **Metadata**: full hydration — every persisted field's sparse index is
|
* - **Metadata**: calls the provider's own `warm?()` when the active
|
||||||
* loaded from storage (`MetadataIndexManager.hydrateAll()`), not just the
|
* `'metadataIndex'` provider implements it (`'warmed'`) — the seam a
|
||||||
* heuristic common-fields subset `init()` warms.
|
* native metadata provider lights up so it is not duck-typed against the
|
||||||
|
* JS manager's method. Otherwise falls back to full hydration on the
|
||||||
|
* built-in JS manager — every persisted field's sparse index is loaded
|
||||||
|
* from storage (`MetadataIndexManager.hydrateAll()`), not just the
|
||||||
|
* heuristic common-fields subset `init()` warms — and reports `'warmed'`.
|
||||||
|
* Neither seam present → `'unavailable'` (honest: `init()` is never used
|
||||||
|
* as a substitute here, since a native provider's `init()` may be a
|
||||||
|
* cheap verify rather than a real warm).
|
||||||
* - **Graph**: calls the provider's own `warm?()` when the active graph
|
* - **Graph**: calls the provider's own `warm?()` when the active graph
|
||||||
* provider implements it; otherwise re-runs its existing eager cold-load
|
* provider implements it; otherwise re-runs its existing eager cold-load
|
||||||
* `init()` seam (idempotent — the JS adjacency index's `init()` already
|
* `init()` seam (idempotent — the JS adjacency index's `init()` already
|
||||||
|
|
@ -14410,12 +14442,23 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
// --- Metadata --------------------------------------------------------
|
// --- Metadata --------------------------------------------------------
|
||||||
const metadataStart = Date.now()
|
const metadataStart = Date.now()
|
||||||
let metadataOutcome: WarmOutcome
|
let metadataOutcome: WarmOutcome
|
||||||
|
const metadataProvider = this.metadataIndex as unknown as MetadataIndexProvider
|
||||||
const metadataWithHydrate = this.metadataIndex as unknown as { hydrateAll?: () => Promise<void> }
|
const metadataWithHydrate = this.metadataIndex as unknown as { hydrateAll?: () => Promise<void> }
|
||||||
if (typeof metadataWithHydrate.hydrateAll === 'function') {
|
if (typeof metadataProvider.warm === 'function') {
|
||||||
|
// Active provider (e.g. a native metadata index) declares its own warm
|
||||||
|
// seam — route through it FIRST so a native provider's warmth is
|
||||||
|
// reported honestly instead of being duck-typed against the JS
|
||||||
|
// manager's hydrateAll(), which a native provider does not implement.
|
||||||
|
await metadataProvider.warm()
|
||||||
|
metadataOutcome = 'warmed'
|
||||||
|
} else if (typeof metadataWithHydrate.hydrateAll === 'function') {
|
||||||
|
// Built-in JS manager path — full sparse-index hydration.
|
||||||
await metadataWithHydrate.hydrateAll()
|
await metadataWithHydrate.hydrateAll()
|
||||||
metadataOutcome = 'warmed'
|
metadataOutcome = 'warmed'
|
||||||
} else {
|
} else {
|
||||||
// No hydration seam on this metadata provider — nothing to run.
|
// No hydration seam on this metadata provider — nothing to run. (No
|
||||||
|
// init() fallback here: init() on a native provider may be a cheap
|
||||||
|
// verify, and reporting that as warmth would lie.)
|
||||||
metadataOutcome = 'unavailable'
|
metadataOutcome = 'unavailable'
|
||||||
}
|
}
|
||||||
const metadataDurationMs = Date.now() - metadataStart
|
const metadataDurationMs = Date.now() - metadataStart
|
||||||
|
|
@ -14449,6 +14492,63 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Read each index surface's self-reported outstanding maintenance work —
|
||||||
|
* the observability seam so an operator sees a grind coming (rising
|
||||||
|
* pending bytes/items, a stalled background pass) instead of discovering
|
||||||
|
* it as a CPU storm or a transaction blowing its budget mid-flight (see
|
||||||
|
* {@link TransactionTimeoutError}).
|
||||||
|
*
|
||||||
|
* PURE PASSTHROUGH: for each of vector/metadata/graph, this calls ONLY the
|
||||||
|
* ACTIVE provider's own `maintenanceDebt?()` hook (the same per-surface
|
||||||
|
* provider resolution {@link Brainy.warm} uses) and reports its
|
||||||
|
* {@link ProviderMaintenanceDebt} payload verbatim. There is no JS-side
|
||||||
|
* fallback computation, no threshold evaluation, and no polling — brainy
|
||||||
|
* surfaces the truth the provider measured; the provider owns the numbers
|
||||||
|
* and the operator owns the policy (what threshold matters, what action to
|
||||||
|
* take). A surface whose active provider does not implement the hook
|
||||||
|
* reports `'unavailable'` — never a guessed or zeroed payload.
|
||||||
|
*
|
||||||
|
* @returns A {@link MaintenanceDebtReport}: per-surface outcome + payload.
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* const debt = await brain.maintenanceDebt()
|
||||||
|
* if (debt.metadata.outcome === 'reported' && debt.metadata.debt?.pendingBytes) {
|
||||||
|
* console.log('metadata pending bytes:', debt.metadata.debt.pendingBytes)
|
||||||
|
* }
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
async maintenanceDebt(): Promise<MaintenanceDebtReport> {
|
||||||
|
await this.ensureInitialized({ needs: ['vector', 'metadata', 'graph'] })
|
||||||
|
|
||||||
|
// --- Vector ---------------------------------------------------------
|
||||||
|
const vectorProvider = this.index as VectorIndexProvider & {
|
||||||
|
maintenanceDebt?: () => Promise<ProviderMaintenanceDebt>
|
||||||
|
}
|
||||||
|
const vector =
|
||||||
|
typeof vectorProvider.maintenanceDebt === 'function'
|
||||||
|
? { outcome: 'reported' as const, debt: await vectorProvider.maintenanceDebt() }
|
||||||
|
: { outcome: 'unavailable' as const }
|
||||||
|
|
||||||
|
// --- Metadata --------------------------------------------------------
|
||||||
|
const metadataProvider = this.metadataIndex as unknown as MetadataIndexProvider
|
||||||
|
const metadata =
|
||||||
|
typeof metadataProvider.maintenanceDebt === 'function'
|
||||||
|
? { outcome: 'reported' as const, debt: await metadataProvider.maintenanceDebt() }
|
||||||
|
: { outcome: 'unavailable' as const }
|
||||||
|
|
||||||
|
// --- Graph -------------------------------------------------------------
|
||||||
|
const graphProvider = this.graphIndex as GraphIndexProvider & {
|
||||||
|
maintenanceDebt?: () => Promise<ProviderMaintenanceDebt>
|
||||||
|
}
|
||||||
|
const graph =
|
||||||
|
typeof graphProvider.maintenanceDebt === 'function'
|
||||||
|
? { outcome: 'reported' as const, debt: await graphProvider.maintenanceDebt() }
|
||||||
|
: { outcome: 'unavailable' as const }
|
||||||
|
|
||||||
|
return { vector, metadata, graph }
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Explicitly warm up the embedding engine
|
* Explicitly warm up the embedding engine
|
||||||
*
|
*
|
||||||
|
|
|
||||||
|
|
@ -121,9 +121,13 @@ export interface TransactOptions {
|
||||||
* with the batch: `max(30 000, opCount × 2 000)` — production imports on
|
* with the batch: `max(30 000, opCount × 2 000)` — production imports on
|
||||||
* network-attached disks measure ~2 s per operation, so a flat 30 s budget
|
* network-attached disks measure ~2 s per operation, so a flat 30 s budget
|
||||||
* silently capped honest bulk work at ~15 operations. A tripped budget
|
* silently capped honest bulk work at ~15 operations. A tripped budget
|
||||||
* rolls the whole batch back and throws a retryable
|
* rolls the whole batch back and throws a `TransactionTimeoutError` naming
|
||||||
* `TransactionTimeoutError` naming the operation it stopped at, the batch
|
* the operation it stopped at, the batch size, and the elapsed/budget
|
||||||
* size, and the elapsed/budget times.
|
* times. That error is retryable-with-latch, never hot-retry: its
|
||||||
|
* `retryable` field says a later attempt may succeed, its
|
||||||
|
* `hotRetryUnsafe` field says an immediate identical retry re-pays the
|
||||||
|
* full cost that just timed out — callers must latch and back off, never
|
||||||
|
* loop.
|
||||||
*/
|
*/
|
||||||
timeoutMs?: number
|
timeoutMs?: number
|
||||||
}
|
}
|
||||||
|
|
|
||||||
10
src/index.ts
10
src/index.ts
|
|
@ -31,6 +31,11 @@ export type { DiagnosticsResult } from './brainy.js'
|
||||||
// brain.warm() — eager index/storage readiness report (per-surface honest
|
// brain.warm() — eager index/storage readiness report (per-surface honest
|
||||||
// outcome + timing). See the WarmReport JSDoc in brainy.ts.
|
// outcome + timing). See the WarmReport JSDoc in brainy.ts.
|
||||||
export type { WarmReport, WarmOutcome } from './brainy.js'
|
export type { WarmReport, WarmOutcome } from './brainy.js'
|
||||||
|
// brain.maintenanceDebt() — per-surface passthrough of each active
|
||||||
|
// provider's self-reported background maintenance debt. See the
|
||||||
|
// MaintenanceDebtReport JSDoc in brainy.ts and ProviderMaintenanceDebt in
|
||||||
|
// plugin.ts for the measure-only-what-you-track contract.
|
||||||
|
export type { MaintenanceDebtReport, MaintenanceDebtOutcome } from './brainy.js'
|
||||||
export type {
|
export type {
|
||||||
GraphAuditReport,
|
GraphAuditReport,
|
||||||
GraphAuditDiscrepancy
|
GraphAuditDiscrepancy
|
||||||
|
|
@ -228,6 +233,11 @@ export type { FamilyStamp, StampMembers, StampVerdict } from './db/familyStamp.j
|
||||||
export { isVersionedIndexProvider } from './plugin.js'
|
export { isVersionedIndexProvider } from './plugin.js'
|
||||||
export type { VersionedIndexProvider } from './plugin.js'
|
export type { VersionedIndexProvider } from './plugin.js'
|
||||||
export type { ProviderInvariantReport, InvariantResult, InvariantHeal } from './plugin.js'
|
export type { ProviderInvariantReport, InvariantResult, InvariantHeal } from './plugin.js'
|
||||||
|
// Optional provider self-report of outstanding background maintenance work
|
||||||
|
// (compaction, deferred writes, etc.) — the payload type for
|
||||||
|
// brain.maintenanceDebt(). See the measure-only-what-you-track contract on
|
||||||
|
// ProviderMaintenanceDebt in plugin.ts.
|
||||||
|
export type { ProviderMaintenanceDebt } from './plugin.js'
|
||||||
// Optional native graph-acceleration engine (cor 3.0) — the published provider
|
// Optional native graph-acceleration engine (cor 3.0) — the published provider
|
||||||
// contract + its columnar wire types. Brainy feature-detects an implementation
|
// contract + its columnar wire types. Brainy feature-detects an implementation
|
||||||
// and falls back to its pure-TS adjacency when absent.
|
// and falls back to its pure-TS adjacency when absent.
|
||||||
|
|
|
||||||
|
|
@ -171,6 +171,37 @@ export interface ProviderInvariantReport {
|
||||||
durationMs: number
|
durationMs: number
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description A provider's self-report of its own outstanding background
|
||||||
|
* maintenance work (compaction, deferred writes, a build-new→verify→swap in
|
||||||
|
* flight, etc.) — the observability seam so an operator sees a grind coming
|
||||||
|
* (rising pending bytes/items, a stalled pass) instead of discovering it as a
|
||||||
|
* CPU storm or a timeout under transaction budget pressure. Every field is
|
||||||
|
* OPTIONAL and every field is a MEASUREMENT: a provider reports ONLY what it
|
||||||
|
* actually tracks, never an estimate dressed up as a fact. Absence of the
|
||||||
|
* {@link VectorIndexProvider.maintenanceDebt} /
|
||||||
|
* {@link GraphIndexProvider.maintenanceDebt} /
|
||||||
|
* {@link MetadataIndexProvider.maintenanceDebt} hook itself means the
|
||||||
|
* provider does not track debt at all — brainy reports that surface
|
||||||
|
* `'unavailable'` rather than inventing zeros. Brainy performs NO threshold
|
||||||
|
* checks, NO polling, and NO JS-side estimation over this payload — it is a
|
||||||
|
* pure passthrough via {@link Brainy.maintenanceDebt}; the provider owns the
|
||||||
|
* numbers and the operator owns the policy (what threshold matters, what to
|
||||||
|
* do about it).
|
||||||
|
*/
|
||||||
|
export interface ProviderMaintenanceDebt {
|
||||||
|
/** Bytes of outstanding/unmerged work, if the provider measures it (e.g. unflushed writes, unmerged segments). */
|
||||||
|
pendingBytes?: number
|
||||||
|
/** Count of outstanding items (records, segments, nodes) awaiting the provider's background pass. */
|
||||||
|
pendingItems?: number
|
||||||
|
/** Epoch millis when the provider's last maintenance pass finished, if it tracks one. */
|
||||||
|
lastPassCompletedAt?: number
|
||||||
|
/** How the last pass ended, if the provider tracks pass outcomes. */
|
||||||
|
lastPassOutcome?: 'completed' | 'partial' | 'failed'
|
||||||
|
/** `true` if the provider's own measurements show debt trending down (making progress); `false` if flat or growing; omitted if the provider can't tell. */
|
||||||
|
converging?: boolean
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The `'metadataIndex'` provider — a drop-in for `MetadataIndexManager`.
|
* The `'metadataIndex'` provider — a drop-in for `MetadataIndexManager`.
|
||||||
* Brainy calls this surface via `this.metadataIndex.*` (see `brainy.ts`) and
|
* Brainy calls this surface via `this.metadataIndex.*` (see `brainy.ts`) and
|
||||||
|
|
@ -181,6 +212,34 @@ export interface MetadataIndexProvider {
|
||||||
flush(): Promise<void>
|
flush(): Promise<void>
|
||||||
rebuild(): Promise<void>
|
rebuild(): Promise<void>
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description OPTIONAL. Eagerly load/fault-in backing storage (e.g. mmap
|
||||||
|
* pretouch, full sparse-index hydration) so first queries run at
|
||||||
|
* steady-state cost. Optional; absence means the provider demand-loads.
|
||||||
|
* Mirrors {@link GraphIndexProvider.warm} / the vector provider's `warm?()`
|
||||||
|
* (`src/plugin.ts` VectorIndexProvider). Distinct from `init()`: `init` is
|
||||||
|
* required and runs once automatically during brain startup; `warm` is a
|
||||||
|
* separate, explicit step a caller opts into via `brain.warm()` (or
|
||||||
|
* `warmOnOpen`) to pre-pay demand-load cost `init` left lazy. Idempotent —
|
||||||
|
* calling it more than once must be safe and cheap on a brain that is
|
||||||
|
* already warm. A provider that already loads everything eagerly in
|
||||||
|
* `init()` may implement `warm` as a no-op or omit it — `brain.warm()`
|
||||||
|
* falls back to the built-in JS manager's `hydrateAll()` duck-type when
|
||||||
|
* absent, and to an honest `'unavailable'` when neither exists.
|
||||||
|
*/
|
||||||
|
warm?(): Promise<void>
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description OPTIONAL self-reported {@link ProviderMaintenanceDebt} —
|
||||||
|
* the observability seam so an operator sees outstanding background
|
||||||
|
* maintenance work (e.g. unmerged postings) BEFORE it grinds a transaction
|
||||||
|
* into a budget-busting op. Absence means this provider does not track
|
||||||
|
* debt; `brain.maintenanceDebt()` reports this surface `'unavailable'`
|
||||||
|
* rather than guessing. See {@link ProviderMaintenanceDebt} for the
|
||||||
|
* measure-only-what-you-track contract.
|
||||||
|
*/
|
||||||
|
maintenanceDebt?(): Promise<ProviderMaintenanceDebt>
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @description OPTIONAL honest durability signal (readiness contract,
|
* @description OPTIONAL honest durability signal (readiness contract,
|
||||||
* mirrors `isReady?()` on the graph and vector providers). `true` ⇔ the
|
* mirrors `isReady?()` on the graph and vector providers). `true` ⇔ the
|
||||||
|
|
@ -395,6 +454,18 @@ export interface GraphIndexProvider {
|
||||||
*/
|
*/
|
||||||
warm?(): Promise<void>
|
warm?(): Promise<void>
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description OPTIONAL self-reported {@link ProviderMaintenanceDebt} —
|
||||||
|
* the observability seam so an operator sees outstanding background
|
||||||
|
* maintenance work (e.g. a build-new→verify→swap in flight, unmerged
|
||||||
|
* adjacency segments) BEFORE it grinds a transaction into a
|
||||||
|
* budget-busting op. Absence means this provider does not track debt;
|
||||||
|
* `brain.maintenanceDebt()` reports this surface `'unavailable'` rather
|
||||||
|
* than guessing. See {@link ProviderMaintenanceDebt} for the
|
||||||
|
* measure-only-what-you-track contract.
|
||||||
|
*/
|
||||||
|
maintenanceDebt?(): Promise<ProviderMaintenanceDebt>
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @description OPTIONAL. A native provider returns true from the moment its
|
* @description OPTIONAL. A native provider returns true from the moment its
|
||||||
* `init()` detects a large epoch-drift until its background
|
* `init()` detects a large epoch-drift until its background
|
||||||
|
|
@ -1057,6 +1128,18 @@ export interface VectorIndexProvider {
|
||||||
*/
|
*/
|
||||||
warm?(): Promise<void>
|
warm?(): Promise<void>
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description OPTIONAL self-reported {@link ProviderMaintenanceDebt} —
|
||||||
|
* the observability seam so an operator sees outstanding background
|
||||||
|
* maintenance work (e.g. unflushed writes, a pending rebuild) BEFORE it
|
||||||
|
* grinds a transaction into a budget-busting op. Absence means this
|
||||||
|
* provider does not track debt; `brain.maintenanceDebt()` reports this
|
||||||
|
* surface `'unavailable'` rather than guessing. See
|
||||||
|
* {@link ProviderMaintenanceDebt} for the measure-only-what-you-track
|
||||||
|
* contract.
|
||||||
|
*/
|
||||||
|
maintenanceDebt?(): Promise<ProviderMaintenanceDebt>
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @description OPTIONAL honest durability signal (readiness contract,
|
* @description OPTIONAL honest durability signal (readiness contract,
|
||||||
* mirrors {@link GraphIndexProvider.isReady}). `true` ⇔ the persisted
|
* mirrors {@link GraphIndexProvider.isReady}). `true` ⇔ the persisted
|
||||||
|
|
|
||||||
|
|
@ -62,9 +62,10 @@ const DEFAULT_BUDGET_FLOOR_MS = 30_000
|
||||||
* NEXT operation may start (see {@link Transaction.execute}), never whether
|
* NEXT operation may start (see {@link Transaction.execute}), never whether
|
||||||
* already-completed work is rolled back after the fact. A trip mid-batch
|
* already-completed work is rolled back after the fact. A trip mid-batch
|
||||||
* still rolls back every operation applied so far, atomically, and throws a
|
* still rolls back every operation applied so far, atomically, and throws a
|
||||||
* retryable, fully-labeled TransactionTimeoutError — that zero-loss guarantee
|
* fully-labeled `TransactionTimeoutError` — retryable-with-latch, never
|
||||||
* doesn't change; only the point at which the clock stops mattering does (at
|
* hot-retry (see its `retryable` and `hotRetryUnsafe` fields) — that
|
||||||
* the last operation, not one check later).
|
* zero-loss guarantee doesn't change; only the point at which the clock
|
||||||
|
* stops mattering does (at the last operation, not one check later).
|
||||||
*
|
*
|
||||||
* @param opCount - Number of operations in the batch.
|
* @param opCount - Number of operations in the batch.
|
||||||
* @param override - A full override for this call; wins over everything else.
|
* @param override - A full override for this call; wins over everything else.
|
||||||
|
|
|
||||||
|
|
@ -19,7 +19,6 @@
|
||||||
import { Transaction } from './Transaction.js'
|
import { Transaction } from './Transaction.js'
|
||||||
import {
|
import {
|
||||||
TransactionFunction,
|
TransactionFunction,
|
||||||
TransactionResult,
|
|
||||||
TransactionOptions
|
TransactionOptions
|
||||||
} from './types.js'
|
} from './types.js'
|
||||||
import { TransactionError } from './errors.js'
|
import { TransactionError } from './errors.js'
|
||||||
|
|
@ -105,34 +104,6 @@ export class TransactionManager {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Execute a transaction and return detailed result
|
|
||||||
*/
|
|
||||||
async executeTransactionWithResult<T>(
|
|
||||||
fn: TransactionFunction<T>,
|
|
||||||
options?: TransactionOptions
|
|
||||||
): Promise<TransactionResult<T>> {
|
|
||||||
const startTime = Date.now()
|
|
||||||
const transaction = new Transaction(options)
|
|
||||||
|
|
||||||
try {
|
|
||||||
const value = await fn(transaction)
|
|
||||||
await transaction.execute()
|
|
||||||
|
|
||||||
const executionTimeMs = Date.now() - startTime
|
|
||||||
|
|
||||||
return {
|
|
||||||
value,
|
|
||||||
operationCount: transaction.getOperationCount(),
|
|
||||||
executionTimeMs
|
|
||||||
}
|
|
||||||
|
|
||||||
} catch (error) {
|
|
||||||
// Transaction failed
|
|
||||||
throw error
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Get transaction statistics
|
* Get transaction statistics
|
||||||
*/
|
*/
|
||||||
|
|
|
||||||
|
|
@ -73,14 +73,47 @@ export class InvalidTransactionStateError extends TransactionError {
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Error for transaction timeout
|
* Error for transaction timeout
|
||||||
|
*
|
||||||
|
* Machine-readable no-hot-retry contract: {@link retryable} and
|
||||||
|
* {@link hotRetryUnsafe} are both always `true` on this class — they exist
|
||||||
|
* so a caller can branch on the *shape* of the error instead of parsing
|
||||||
|
* message text. Read them together: the operation may eventually succeed,
|
||||||
|
* but never by looping on it immediately.
|
||||||
|
*
|
||||||
|
* `context` (inherited from {@link TransactionError}) carries the caller's
|
||||||
|
* backoff inputs — see the field docs below.
|
||||||
*/
|
*/
|
||||||
export class TransactionTimeoutError extends TransactionError {
|
export class TransactionTimeoutError extends TransactionError {
|
||||||
|
/**
|
||||||
|
* The failed operation MAY succeed on a later attempt — once the
|
||||||
|
* underlying slowness resolves (e.g. a cold page cache warms up) or the
|
||||||
|
* budget is deliberately raised (`transactionBudgetFloorMs`, or a larger
|
||||||
|
* `timeoutMs` override on the batch). This is a statement about eventual
|
||||||
|
* retryability, not a license to retry now — see {@link hotRetryUnsafe}.
|
||||||
|
*/
|
||||||
|
public readonly retryable = true
|
||||||
|
|
||||||
|
/**
|
||||||
|
* An immediate, identical retry re-pays the FULL cost of the work that
|
||||||
|
* just timed out — it does not resume partway. Looping on this error
|
||||||
|
* (hot-retrying) repeats that full cost every attempt and can cascade
|
||||||
|
* into a CPU/resource storm on the caller's side. Callers MUST latch: on
|
||||||
|
* this error, record `{ at: Date.now(), error }`, surface one loud
|
||||||
|
* failure to their own caller, and hold a cooldown window before any
|
||||||
|
* re-attempt (clearing the latch only on success). Never retry this error
|
||||||
|
* in a tight loop.
|
||||||
|
*/
|
||||||
|
public readonly hotRetryUnsafe = true
|
||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
timeoutMs: number,
|
timeoutMs: number,
|
||||||
operationIndex: number,
|
operationIndex: number,
|
||||||
telemetry?: {
|
telemetry?: {
|
||||||
|
/** Milliseconds elapsed in the transaction when the budget tripped. */
|
||||||
elapsedMs?: number
|
elapsedMs?: number
|
||||||
|
/** Total number of operations in the batch that timed out. */
|
||||||
totalOperations?: number
|
totalOperations?: number
|
||||||
|
/** Name of the operation the batch was about to start when it tripped, if named. */
|
||||||
operationName?: string
|
operationName?: string
|
||||||
}
|
}
|
||||||
) {
|
) {
|
||||||
|
|
@ -93,8 +126,16 @@ export class TransactionTimeoutError extends TransactionError {
|
||||||
telemetry?.elapsedMs !== undefined ? `${telemetry.elapsedMs}ms elapsed, ` : ''
|
telemetry?.elapsedMs !== undefined ? `${telemetry.elapsedMs}ms elapsed, ` : ''
|
||||||
super(
|
super(
|
||||||
`Transaction timed out at operation ${progress}${name} — ${elapsed}budget ${timeoutMs}ms. ` +
|
`Transaction timed out at operation ${progress}${name} — ${elapsed}budget ${timeoutMs}ms. ` +
|
||||||
`The batch rolled back atomically; retry with a higher timeoutMs or a smaller batch.`,
|
`The batch rolled back atomically; retryable after the underlying slowness resolves or ` +
|
||||||
{ timeoutMs, operationIndex, ...telemetry }
|
`the budget is raised, but hot-retry-unsafe — latch and back off, never loop.`,
|
||||||
|
{
|
||||||
|
// Caller backoff inputs — all present on every instance:
|
||||||
|
/** Configured budget (ms) that was exceeded. */
|
||||||
|
timeoutMs,
|
||||||
|
/** Index of the operation the batch was about to start when it tripped. */
|
||||||
|
operationIndex,
|
||||||
|
...telemetry
|
||||||
|
}
|
||||||
)
|
)
|
||||||
this.name = 'TransactionTimeoutError'
|
this.name = 'TransactionTimeoutError'
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -66,26 +66,6 @@ export interface TransactionContext {
|
||||||
*/
|
*/
|
||||||
export type TransactionFunction<T> = (ctx: TransactionContext) => Promise<T>
|
export type TransactionFunction<T> = (ctx: TransactionContext) => Promise<T>
|
||||||
|
|
||||||
/**
|
|
||||||
* Transaction execution result
|
|
||||||
*/
|
|
||||||
export interface TransactionResult<T> {
|
|
||||||
/**
|
|
||||||
* Result value from user function
|
|
||||||
*/
|
|
||||||
value: T
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Number of operations executed
|
|
||||||
*/
|
|
||||||
operationCount: number
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Execution time in milliseconds
|
|
||||||
*/
|
|
||||||
executionTimeMs: number
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Transaction execution options
|
* Transaction execution options
|
||||||
*/
|
*/
|
||||||
|
|
|
||||||
|
|
@ -1658,8 +1658,10 @@ export interface BrainyConfig {
|
||||||
* **start** — never whether already-completed work gets rolled back after
|
* **start** — never whether already-completed work gets rolled back after
|
||||||
* the fact (a single-op write can never time out post-hoc: it either runs
|
* the fact (a single-op write can never time out post-hoc: it either runs
|
||||||
* or it commits). A trip mid-batch still rolls back every applied operation
|
* or it commits). A trip mid-batch still rolls back every applied operation
|
||||||
* atomically and throws a retryable `TransactionTimeoutError`; only the
|
* atomically and throws a `TransactionTimeoutError` that is
|
||||||
* floor of the formula is configurable here.
|
* retryable-with-latch, never hot-retry (see its `retryable` and
|
||||||
|
* `hotRetryUnsafe` fields); only the floor of the formula is configurable
|
||||||
|
* here.
|
||||||
*
|
*
|
||||||
* Raise this when a cold store's first writes after a restart legitimately
|
* Raise this when a cold store's first writes after a restart legitimately
|
||||||
* take longer than 30s per operation (e.g. page-cache-cold canonical writes
|
* take longer than 30s per operation (e.g. page-cache-cold canonical writes
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,6 @@
|
||||||
* - High-level transaction API
|
* - High-level transaction API
|
||||||
* - Statistics tracking
|
* - Statistics tracking
|
||||||
* - Error handling
|
* - Error handling
|
||||||
* - Result wrapping
|
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { describe, it, expect, beforeEach } from 'vitest'
|
import { describe, it, expect, beforeEach } from 'vitest'
|
||||||
|
|
@ -84,42 +83,6 @@ describe('TransactionManager', () => {
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
describe('executeTransactionWithResult', () => {
|
|
||||||
it('should return detailed result', async () => {
|
|
||||||
const result = await manager.executeTransactionWithResult(async (tx) => {
|
|
||||||
tx.addOperation({
|
|
||||||
execute: async () => {
|
|
||||||
await new Promise(resolve => setTimeout(resolve, 1))
|
|
||||||
return async () => {}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
tx.addOperation({ execute: async () => undefined })
|
|
||||||
return 'success'
|
|
||||||
})
|
|
||||||
|
|
||||||
expect(result.value).toBe('success')
|
|
||||||
expect(result.operationCount).toBe(2)
|
|
||||||
expect(result.executionTimeMs).toBeGreaterThanOrEqual(0)
|
|
||||||
})
|
|
||||||
|
|
||||||
it('should measure execution time', async () => {
|
|
||||||
const result = await manager.executeTransactionWithResult(async (tx) => {
|
|
||||||
tx.addOperation({
|
|
||||||
execute: async () => {
|
|
||||||
await new Promise(resolve => setTimeout(resolve, 25))
|
|
||||||
return async () => {}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
return 'done'
|
|
||||||
})
|
|
||||||
|
|
||||||
// Timer coalescing can fire a setTimeout up to a few ms EARLY under
|
|
||||||
// load, so assert well below the sleep — this tests that time is
|
|
||||||
// MEASURED, not the OS timer's precision.
|
|
||||||
expect(result.executionTimeMs).toBeGreaterThanOrEqual(20)
|
|
||||||
})
|
|
||||||
})
|
|
||||||
|
|
||||||
describe('Statistics Tracking', () => {
|
describe('Statistics Tracking', () => {
|
||||||
it('should track total transactions', async () => {
|
it('should track total transactions', async () => {
|
||||||
await manager.executeTransaction(async (tx) => {
|
await manager.executeTransaction(async (tx) => {
|
||||||
|
|
|
||||||
122
tests/unit/brainy/maintenance-debt.test.ts
Normal file
122
tests/unit/brainy/maintenance-debt.test.ts
Normal file
|
|
@ -0,0 +1,122 @@
|
||||||
|
/**
|
||||||
|
* @module tests/unit/brainy/maintenance-debt
|
||||||
|
* @description Coverage for `brain.maintenanceDebt()` (8.10.1) — the
|
||||||
|
* observability seam so an operator sees a provider's outstanding background
|
||||||
|
* maintenance work (compaction, deferred writes, a build-new→verify→swap in
|
||||||
|
* flight, ...) BEFORE it grinds a transaction into a budget-busting op, the
|
||||||
|
* same failure class documented on `TransactionTimeoutError`
|
||||||
|
* (src/transaction/errors.ts). Sibling to tests/unit/brainy/warm.test.ts,
|
||||||
|
* which establishes this file's technique: shape the probe points brain.ts
|
||||||
|
* reads (`typeof provider.maintenanceDebt === 'function'`) directly on the
|
||||||
|
* REAL, live provider instances rather than hand-rolling full fakes for the
|
||||||
|
* larger `MetadataIndexProvider` / `GraphIndexProvider` interfaces.
|
||||||
|
*
|
||||||
|
* `brain.maintenanceDebt()` is a PURE PASSTHROUGH: no thresholds, no
|
||||||
|
* polling, no JS-side estimation — these tests pin exactly that by asserting
|
||||||
|
* the returned payload is the provider's object, verbatim.
|
||||||
|
*/
|
||||||
|
import { describe, it, expect } from 'vitest'
|
||||||
|
import { Brainy } from '../../../src/brainy.js'
|
||||||
|
import { NounType } from '../../../src/types/graphTypes.js'
|
||||||
|
import type { ProviderMaintenanceDebt } from '../../../src/plugin.js'
|
||||||
|
|
||||||
|
// Brainy's ValidationConfig fixes vectors at exactly 384 dimensions
|
||||||
|
// (src/utils/paramValidation.ts) — match it so add() doesn't reject test data.
|
||||||
|
const DIM = 384
|
||||||
|
const V = (seed = 1): number[] => Array.from({ length: DIM }, (_, i) => Math.sin(seed + i))
|
||||||
|
|
||||||
|
async function freshBrain(): Promise<Brainy<any>> {
|
||||||
|
const brain = new Brainy({
|
||||||
|
requireSubtype: false,
|
||||||
|
storage: { type: 'memory' },
|
||||||
|
silent: true
|
||||||
|
})
|
||||||
|
await brain.init()
|
||||||
|
await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
|
||||||
|
return brain
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('brain.maintenanceDebt()', () => {
|
||||||
|
it('reports "unavailable" for every surface when no active provider implements maintenanceDebt() (the built-in JS stack today)', async () => {
|
||||||
|
const brain = await freshBrain()
|
||||||
|
|
||||||
|
const report = await brain.maintenanceDebt()
|
||||||
|
|
||||||
|
expect(report.vector.outcome).toBe('unavailable')
|
||||||
|
expect(report.vector.debt).toBeUndefined()
|
||||||
|
expect(report.metadata.outcome).toBe('unavailable')
|
||||||
|
expect(report.metadata.debt).toBeUndefined()
|
||||||
|
expect(report.graph.outcome).toBe('unavailable')
|
||||||
|
expect(report.graph.debt).toBeUndefined()
|
||||||
|
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports "reported" + the exact payload when the active provider implements maintenanceDebt() (verbatim passthrough, no thresholding)', async () => {
|
||||||
|
const brain = await freshBrain()
|
||||||
|
|
||||||
|
const vectorDebt: ProviderMaintenanceDebt = {
|
||||||
|
pendingBytes: 4_096,
|
||||||
|
pendingItems: 12,
|
||||||
|
lastPassCompletedAt: 1_700_000_000_000,
|
||||||
|
lastPassOutcome: 'completed',
|
||||||
|
converging: true
|
||||||
|
}
|
||||||
|
;(brain as any).index.maintenanceDebt = async () => vectorDebt
|
||||||
|
|
||||||
|
const report = await brain.maintenanceDebt()
|
||||||
|
|
||||||
|
expect(report.vector.outcome).toBe('reported')
|
||||||
|
// Verbatim passthrough — the exact object, not a re-derived copy.
|
||||||
|
expect(report.vector.debt).toBe(vectorDebt)
|
||||||
|
// Untouched surfaces stay honestly 'unavailable'.
|
||||||
|
expect(report.metadata.outcome).toBe('unavailable')
|
||||||
|
expect(report.graph.outcome).toBe('unavailable')
|
||||||
|
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('mixed surfaces: each surface\'s outcome depends ONLY on its OWN active provider — one surface reporting never leaks into another', async () => {
|
||||||
|
const brain = await freshBrain()
|
||||||
|
|
||||||
|
const metadataDebt: ProviderMaintenanceDebt = {
|
||||||
|
pendingItems: 3,
|
||||||
|
lastPassOutcome: 'partial',
|
||||||
|
converging: false
|
||||||
|
}
|
||||||
|
const graphDebt: ProviderMaintenanceDebt = {
|
||||||
|
pendingBytes: 0,
|
||||||
|
converging: true
|
||||||
|
}
|
||||||
|
;(brain as any).metadataIndex.maintenanceDebt = async () => metadataDebt
|
||||||
|
;(brain as any).graphIndex.maintenanceDebt = async () => graphDebt
|
||||||
|
// Vector is deliberately left unpatched.
|
||||||
|
|
||||||
|
const report = await brain.maintenanceDebt()
|
||||||
|
|
||||||
|
expect(report.vector.outcome).toBe('unavailable')
|
||||||
|
expect(report.vector.debt).toBeUndefined()
|
||||||
|
|
||||||
|
expect(report.metadata.outcome).toBe('reported')
|
||||||
|
expect(report.metadata.debt).toBe(metadataDebt)
|
||||||
|
|
||||||
|
expect(report.graph.outcome).toBe('reported')
|
||||||
|
expect(report.graph.debt).toBe(graphDebt)
|
||||||
|
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('an empty ProviderMaintenanceDebt object (every field omitted) is still honestly "reported" — presence of the hook, not the payload\'s richness, drives the outcome', async () => {
|
||||||
|
const brain = await freshBrain()
|
||||||
|
|
||||||
|
const emptyDebt: ProviderMaintenanceDebt = {}
|
||||||
|
;(brain as any).graphIndex.maintenanceDebt = async () => emptyDebt
|
||||||
|
|
||||||
|
const report = await brain.maintenanceDebt()
|
||||||
|
|
||||||
|
expect(report.graph.outcome).toBe('reported')
|
||||||
|
expect(report.graph.debt).toEqual({})
|
||||||
|
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
@ -307,4 +307,92 @@ describe('brain.warm()', () => {
|
||||||
expect(report.totalDurationMs).toBeGreaterThanOrEqual(0)
|
expect(report.totalDurationMs).toBeGreaterThanOrEqual(0)
|
||||||
await brain.close()
|
await brain.close()
|
||||||
})
|
})
|
||||||
|
|
||||||
|
// --- Metadata leg routes through the ACTIVE provider (8.10.1) -----------
|
||||||
|
//
|
||||||
|
// `MetadataIndexProvider` is a ~50-method interface (src/plugin.ts) — far
|
||||||
|
// too large to hand-write a compliant fake class the way `FakeVectorProvider`
|
||||||
|
// fakes the ~8-method `VectorIndexProvider` above. Test (c) already
|
||||||
|
// establishes this file's pattern for the metadata leg: exercise the REAL
|
||||||
|
// `MetadataIndexManager` instance and shape just the probe points brain.ts
|
||||||
|
// reads (`typeof provider.warm === 'function'` /
|
||||||
|
// `typeof provider.hydrateAll === 'function'`) directly on that instance.
|
||||||
|
// Shadowing an own property on the live object stands in for "a different
|
||||||
|
// provider implementation" without needing a hand-rolled full fake — the
|
||||||
|
// rest of the real manager (used by add()/init() above) is untouched.
|
||||||
|
describe('metadata leg — warm() routes through the active provider', () => {
|
||||||
|
it('(f) calls the ACTIVE metadata provider\'s warm() when present and reports "warmed", never falling back to hydrateAll', async () => {
|
||||||
|
const brain = new Brainy({
|
||||||
|
requireSubtype: false,
|
||||||
|
storage: { type: 'memory' },
|
||||||
|
silent: true
|
||||||
|
})
|
||||||
|
await brain.init()
|
||||||
|
await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
|
||||||
|
|
||||||
|
const metadataIndex = (brain as any).metadataIndex
|
||||||
|
let warmCalls = 0
|
||||||
|
let hydrateAllCalls = 0
|
||||||
|
const origHydrateAll = metadataIndex.hydrateAll.bind(metadataIndex)
|
||||||
|
metadataIndex.hydrateAll = async (...args: unknown[]) => {
|
||||||
|
hydrateAllCalls++
|
||||||
|
return origHydrateAll(...args)
|
||||||
|
}
|
||||||
|
// Simulates a native metadata provider declaring the optional `warm()`
|
||||||
|
// hook added to `MetadataIndexProvider` (src/plugin.ts) in 8.10.1.
|
||||||
|
metadataIndex.warm = async () => {
|
||||||
|
warmCalls++
|
||||||
|
}
|
||||||
|
|
||||||
|
const report = await brain.warm()
|
||||||
|
|
||||||
|
expect(warmCalls).toBe(1)
|
||||||
|
expect(hydrateAllCalls).toBe(0) // warm() ran — no hydrateAll fallback
|
||||||
|
expect(report.metadata.outcome).toBe('warmed')
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports "unavailable" when the active metadata provider implements neither warm() nor hydrateAll() (the honest branch a native provider without either hook must hit)', async () => {
|
||||||
|
const brain = new Brainy({
|
||||||
|
requireSubtype: false,
|
||||||
|
storage: { type: 'memory' },
|
||||||
|
silent: true
|
||||||
|
})
|
||||||
|
await brain.init()
|
||||||
|
await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
|
||||||
|
|
||||||
|
const metadataIndex = (brain as any).metadataIndex
|
||||||
|
// Shadow away BOTH optional hooks — models a genuinely native provider
|
||||||
|
// that (unlike the built-in JS manager) offers neither seam. This must
|
||||||
|
// never fall back to calling init() as a stand-in for warmth.
|
||||||
|
metadataIndex.warm = undefined
|
||||||
|
metadataIndex.hydrateAll = undefined
|
||||||
|
|
||||||
|
const report = await brain.warm()
|
||||||
|
|
||||||
|
expect(report.metadata.outcome).toBe('unavailable')
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('the built-in JS manager (no warm()) still reports "warmed" via its existing hydrateAll() duck-type — unchanged by the new provider hook', async () => {
|
||||||
|
const brain = new Brainy({
|
||||||
|
requireSubtype: false,
|
||||||
|
storage: { type: 'memory' },
|
||||||
|
silent: true
|
||||||
|
})
|
||||||
|
await brain.init()
|
||||||
|
await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
|
||||||
|
|
||||||
|
// No patching at all — the default built-in MetadataIndexManager has
|
||||||
|
// hydrateAll() but no warm(), exactly as it did before this change.
|
||||||
|
const metadataIndex = (brain as any).metadataIndex
|
||||||
|
expect(typeof metadataIndex.warm).not.toBe('function')
|
||||||
|
expect(typeof metadataIndex.hydrateAll).toBe('function')
|
||||||
|
|
||||||
|
const report = await brain.warm()
|
||||||
|
|
||||||
|
expect(report.metadata.outcome).toBe('warmed')
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
})
|
||||||
})
|
})
|
||||||
|
|
|
||||||
141
tests/unit/transaction/timeout-never-internally-retried.test.ts
Normal file
141
tests/unit/transaction/timeout-never-internally-retried.test.ts
Normal file
|
|
@ -0,0 +1,141 @@
|
||||||
|
/**
|
||||||
|
* @module tests/unit/transaction/timeout-never-internally-retried
|
||||||
|
* @description Regression pin for the no-hot-retry contract (8.10.1).
|
||||||
|
*
|
||||||
|
* A production incident: a native-provider op ground 38-40s inside a
|
||||||
|
* transaction, blew the ~32s budget, was rolled back, and a CONSUMER pipeline
|
||||||
|
* hot-retried the identical operation into a 6-minute 100%-CPU storm.
|
||||||
|
* Investigation established brainy itself never auto-retries a
|
||||||
|
* `TransactionTimeoutError` — the storm was entirely the consumer's hot-retry
|
||||||
|
* loop, driven by a "retryable" doc-prose claim with no machine-readable
|
||||||
|
* contract. This file pins the brainy-side half of that story so it can never
|
||||||
|
* regress silently:
|
||||||
|
*
|
||||||
|
* (i) the underlying engine (`TransactionManager.executeTransaction()` →
|
||||||
|
* `Transaction.execute()`) — the exact machinery every single-record
|
||||||
|
* write (`add`/`update`/`remove`/...) drives via
|
||||||
|
* `Brainy.persistSingleOp()` — never internally re-executes a timed-out
|
||||||
|
* operation, and the error it surfaces carries `retryable === true` and
|
||||||
|
* `hotRetryUnsafe === true` (see `src/transaction/errors.ts`).
|
||||||
|
*
|
||||||
|
* Constructed directly (mirrors the existing
|
||||||
|
* `tests/unit/transaction/timeout-rollback.test.ts` pattern) rather than
|
||||||
|
* through a real `brain.add()` call: `transactTimeoutBudget()` floors
|
||||||
|
* every single-op write's budget at `opCount * 2000`ms with NO override
|
||||||
|
* seam (`transactionBudgetFloorMs` only RAISES that floor — it cannot
|
||||||
|
* lower it below the per-op-count term), so getting a real `add()` to
|
||||||
|
* time out requires a multi-second sleep. The engine-level
|
||||||
|
* `options.timeout` override used here is the exact same
|
||||||
|
* `TransactionManager`/`Transaction` code `persistSingleOp` calls —
|
||||||
|
* pinning it here pins add()'s guarantee without paying that wall-clock
|
||||||
|
* cost.
|
||||||
|
*
|
||||||
|
* (ii) `Brainy.add()`'s upsert-race retry loop (src/brainy.ts,
|
||||||
|
* `MAX_UPSERT_ATTEMPTS = 10`) — proving the loop's `catch` treats a
|
||||||
|
* `TransactionTimeoutError` as terminal (immediate rethrow) rather than
|
||||||
|
* the `InsertPreconditionExistsSignal` it retries on, so a mid-flight
|
||||||
|
* timeout can never be silently swallowed and re-attempted up to 10
|
||||||
|
* times.
|
||||||
|
*/
|
||||||
|
import { describe, it, expect } from 'vitest'
|
||||||
|
import { TransactionManager } from '../../../src/transaction/TransactionManager.js'
|
||||||
|
import type { Operation, RollbackAction } from '../../../src/transaction/types.js'
|
||||||
|
import { TransactionTimeoutError } from '../../../src/transaction/errors.js'
|
||||||
|
import { Brainy } from '../../../src/brainy.js'
|
||||||
|
import { NounType } from '../../../src/types/graphTypes.js'
|
||||||
|
|
||||||
|
const sleep = (ms: number) => new Promise<void>((r) => setTimeout(r, ms))
|
||||||
|
|
||||||
|
// Brainy's ValidationConfig fixes vectors at exactly 384 dimensions
|
||||||
|
// (src/utils/paramValidation.ts) — match it so add() doesn't reject test data.
|
||||||
|
const DIM = 384
|
||||||
|
const V = (): number[] => Array(DIM).fill(0.1)
|
||||||
|
|
||||||
|
/** An operation that counts every `execute()` invocation — the re-drive detector. */
|
||||||
|
function countingOp(opts: { delayMs?: number; name: string }): Operation & { calls: number } {
|
||||||
|
const op = {
|
||||||
|
name: opts.name,
|
||||||
|
calls: 0,
|
||||||
|
async execute(): Promise<RollbackAction | undefined> {
|
||||||
|
op.calls++
|
||||||
|
if (opts.delayMs) await sleep(opts.delayMs)
|
||||||
|
return async () => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return op
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('transaction timeouts are never internally re-driven (8.10.1 no-hot-retry contract)', () => {
|
||||||
|
it('(i) the single-op engine (TransactionManager.executeTransaction / Transaction.execute — what add() drives via persistSingleOp) runs the overrun operation EXACTLY once and surfaces ONE retryable+hotRetryUnsafe error', async () => {
|
||||||
|
const manager = new TransactionManager()
|
||||||
|
const op0 = countingOp({ name: 'op0-overruns-budget', delayMs: 30 })
|
||||||
|
const op1 = countingOp({ name: 'op1-must-never-start' })
|
||||||
|
|
||||||
|
let caught: unknown
|
||||||
|
try {
|
||||||
|
await manager.executeTransaction(
|
||||||
|
async (tx) => {
|
||||||
|
tx.addOperation(op0)
|
||||||
|
tx.addOperation(op1)
|
||||||
|
},
|
||||||
|
// Tiny explicit override — the same override seam `transact()`
|
||||||
|
// exposes as `options.timeoutMs`; wins outright over the
|
||||||
|
// opCount*2000 floor that gates every real single-op write
|
||||||
|
// (transactTimeoutBudget()'s override semantics).
|
||||||
|
{ timeout: 5 }
|
||||||
|
)
|
||||||
|
} catch (err) {
|
||||||
|
caught = err
|
||||||
|
}
|
||||||
|
|
||||||
|
expect(caught).toBeInstanceOf(TransactionTimeoutError)
|
||||||
|
const err = caught as TransactionTimeoutError
|
||||||
|
// The machine-readable contract callers branch on instead of parsing
|
||||||
|
// message text (src/transaction/errors.ts).
|
||||||
|
expect(err.retryable).toBe(true)
|
||||||
|
expect(err.hotRetryUnsafe).toBe(true)
|
||||||
|
|
||||||
|
// The re-drive assertion: op0 (the one that overran) executed EXACTLY
|
||||||
|
// once — nothing inside TransactionManager/Transaction looped back and
|
||||||
|
// re-ran it — and op1 never started at all (the budget gate stopped it
|
||||||
|
// before it began, per Transaction.execute()'s per-operation loop).
|
||||||
|
expect(op0.calls).toBe(1)
|
||||||
|
expect(op1.calls).toBe(0)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('(ii) add()\'s upsert-race retry loop (MAX_UPSERT_ATTEMPTS=10) exits on the FIRST TransactionTimeoutError — attempt counter stays at 1, never mistaken for the lost-insert-race signal it retries on', async () => {
|
||||||
|
const brain = new Brainy({
|
||||||
|
requireSubtype: false,
|
||||||
|
storage: { type: 'memory' },
|
||||||
|
silent: true
|
||||||
|
})
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
let persistSingleOpCalls = 0
|
||||||
|
const timeoutError = new TransactionTimeoutError(5, 1, {
|
||||||
|
elapsedMs: 6,
|
||||||
|
totalOperations: 2,
|
||||||
|
operationName: 'SaveNounMetadata'
|
||||||
|
})
|
||||||
|
// Stub the private commit seam add() drives (persistSingleOp) to throw
|
||||||
|
// the exact error type the real engine surfaces on a mid-flight timeout.
|
||||||
|
// This test pins the upsert loop's EXCEPTION-HANDLING contract (does it
|
||||||
|
// retry a TransactionTimeoutError like it retries
|
||||||
|
// InsertPreconditionExistsSignal?), not the timing mechanics of a real
|
||||||
|
// timeout — those are pinned by test (i) and by
|
||||||
|
// tests/unit/transaction/timeout-rollback.test.ts.
|
||||||
|
;(brain as any).persistSingleOp = async (): Promise<never> => {
|
||||||
|
persistSingleOpCalls++
|
||||||
|
throw timeoutError
|
||||||
|
}
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
brain.add({ data: 'a', type: NounType.Thing, vector: V() })
|
||||||
|
).rejects.toBe(timeoutError)
|
||||||
|
|
||||||
|
// The loop's attempt counter: exactly one call, never retried up to
|
||||||
|
// MAX_UPSERT_ATTEMPTS.
|
||||||
|
expect(persistSingleOpCalls).toBe(1)
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
})
|
||||||
Loading…
Add table
Add a link
Reference in a new issue