diff --git a/.forgejo/workflows/ci.yml b/.github/workflows/ci.yml similarity index 100% rename from .forgejo/workflows/ci.yml rename to .github/workflows/ci.yml diff --git a/CHANGELOG.md b/CHANGELOG.md index a5f344cc..fd0de54e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,23 +2,6 @@ All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines. -### [8.10.1](https://source.soulcraft.com/soulcraft/brainy/compare/v8.10.0...v8.10.1) (2026-07-24) - -- refactor: remove the orphaned transaction-result type left behind by the dead-path removal (edf123a5) -- fix: warm() metadata surface routes through the active provider (warm hook added to the metadata contract); add maintenanceDebt() observability surface (5b2cbf74) -- fix: transaction timeouts are a typed no-hot-retry contract; engine-side non-retry pinned; dead transaction path removed (003e2a74) -- chore: the forge is the address — retire the archived mirror from every live surface (22702b81) - - -### [8.10.0](https://github.com/soulcraftlabs/brainy/compare/v8.9.0...v8.10.0) (2026-07-23) - -- docs: adoption storefront — contributing guide, security policy, README support + cor section (9a99a7b) -- fix(release): push the public mirror explicitly and verify the tag lands at the right commit before publishing (6ba94c8) -- docs: project guide version line points at npm instead of a hardcoded stale number (3a1efc9) -- feat: vector provider identity is a required name field (hnsw-js), rendered [vector-index:] (3be4ba9) -- feat: warm contract (warm/warmOnOpen/provider warm hook), configurable transact budget floor, backend-neutral vector index op names (55b867c) - - ### [8.9.0](https://github.com/soulcraftlabs/brainy/compare/v8.8.2...v8.9.0) (2026-07-19) - docs: measured performance envelopes v1 (per-op p50/p95 at 1k and 10k, pure-JS floor) (5cabd78) diff --git a/CLAUDE.md b/CLAUDE.md index c7336a18..568c10db 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,7 +12,7 @@ Handoff file: `/home/dpsifr/.strategy/PLATFORM-HANDOFF.md` **Brainy's current open actions:** None. MIT open-source — no platform-specific actions. -**Current version:** run `npm view @soulcraft/brainy version` (never trust a hardcoded number here — this line went stale for months); consumer-facing changes tracked in `RELEASES.md` +**Current version:** `@soulcraft/brainy@7.31.5` (latest published; 8.0.0 release candidate on `feat/8.0-u64-ids`) --- diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d277091d..ab8a0246 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,66 +1,298 @@ # Contributing to Brainy -Brainy is MIT-licensed and genuinely open to outside contributions. This page -is the honest, current path — please don't rely on older instructions you -may find elsewhere in the repo's history. +Thank you for your interest in contributing to Brainy! This document provides guidelines and instructions for contributing to the project. -## Where the project lives +## Code of Conduct -The source of truth is a self-hosted forge: **source.soulcraft.com/soulcraft/brainy**. -It's anonymously readable and cloneable — no account needed to browse, clone, -or build. +By participating in this project, you agree to abide by our Code of Conduct: +- Be respectful and inclusive +- Welcome newcomers and help them get started +- Focus on constructive criticism +- Respect differing viewpoints and experiences -## How to contribute +## How to Contribute -**Found a bug, or have an idea?** Email **brainy@soulcraft.com**. No account, -no ceremony — you'll get a receipt, and it goes to a human. +### Reporting Issues -**Want to send a patch?** Two ways, both first-class: +Before creating an issue, please check existing issues to avoid duplicates. -- **Email a patch.** Run `git format-patch` against your change and email the - output to **brainy@soulcraft.com**. This is a genuinely supported path, not - a fallback — plenty of good contributions arrive this way. -- **Open a pull request on the forge.** Request an account at - **source.soulcraft.com** (registration is request-with-approval, so allow - a little lag), clone, push a branch, and open a PR there. Maintainers - review and land it. +When creating an issue, include: +- Clear, descriptive title +- Detailed description of the problem +- Steps to reproduce +- Expected vs actual behavior +- System information (OS, Node version, Brainy version) +- Code examples if applicable -Either way, for anything beyond a small fix, opening an issue first (email is -fine) to talk through the approach saves everyone rework. +### Suggesting Features -## Development setup +Feature requests are welcome! Please provide: +- Clear use case +- Proposed API/interface +- Examples of how it would work +- Any potential challenges or considerations +### Pull Requests + +#### Before Starting + +1. Check existing issues and PRs +2. Open an issue to discuss significant changes +3. Fork the repository +4. Create a feature branch from `main` + +#### Development Setup + +**Quick Setup (Recommended):** ```bash -git clone https://source.soulcraft.com/soulcraft/brainy.git +# Clone your fork +git clone https://github.com/your-username/brainy.git cd brainy + +# Run setup script (installs all dependencies including Rust) +./scripts/setup-dev.sh +``` + +**Manual Setup:** +```bash +# Clone your fork +git clone https://github.com/your-username/brainy.git +cd brainy + +# Install system dependencies (Ubuntu/Debian) +sudo apt-get install -y build-essential pkg-config libssl-dev + +# Install Rust (for WASM embedding engine) +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh +source ~/.cargo/env +rustup target add wasm32-unknown-unknown +cargo install wasm-pack + +# Install Node.js dependencies npm install + +# Build Candle WASM embedding engine +npm run build:candle + +# Build TypeScript npm run build + +# Run tests npm test ``` -Tests run on [Vitest](https://vitest.dev/). `npm test` runs the unit suite; -see `package.json` for `test:integration`, `test:coverage`, and friends. +#### Making Changes -## Standards +1. **Follow the code style** + - TypeScript for all source code + - Clear variable and function names + - Comments for complex logic + - JSDoc for public APIs -- **Strict TypeScript.** No `any` escape hatches to dodge the type checker. -- **Tests exercise real behavior.** No mocking away the thing you're supposed - to be testing. -- **No stubs, no TODO-code.** If something can't be finished, say so and - leave it out — don't merge a placeholder. -- **JSDoc on every exported function, class, and type.** -- **[Conventional Commits](https://www.conventionalcommits.org/).** `feat:`, - `fix:`, `docs:`, `perf:`, `refactor:`, `test:`, `chore:`. Never - `BREAKING CHANGE` in a commit message — major version bumps are a separate, - deliberate decision. -- **Performance claims are measured or labeled projected.** If a PR or its - description states a number, cite the benchmark that produced it (see - [docs/performance-envelopes.md](docs/performance-envelopes.md) for the - pattern). Don't state an estimate as if it were measured. +2. **Write tests** + - Add tests for new features + - Update tests for changes + - Ensure all tests pass -## License +3. **Update documentation** + - Update README if needed + - Add/update API documentation + - Include examples -Brainy is [MIT licensed](LICENSE). Contributions are accepted under the same -license — there's no CLA to sign. +#### Commit Guidelines -Thank you for considering a contribution. +Follow conventional commits format: + +``` +type(scope): description + +[optional body] + +[optional footer] +``` + +Types: +- `feat`: New feature +- `fix`: Bug fix +- `docs`: Documentation changes +- `style`: Code style changes +- `refactor`: Code refactoring +- `perf`: Performance improvements +- `test`: Test changes +- `chore`: Build/tooling changes + +Examples: +```bash +feat(triple): add graph traversal depth limit +fix(storage): handle concurrent write conflicts +docs(api): update search method documentation +``` + +#### Submitting PR + +1. Push to your fork +2. Create PR against `main` branch +3. Fill out PR template +4. Ensure CI checks pass +5. Wait for review + +### Testing + +#### Running Tests + +```bash +# Run all tests +npm test + +# Run specific test file +npm test tests/core.test.ts + +# Run with coverage +npm run test:coverage + +# Watch mode +npm run test:watch +``` + +#### Writing Tests + +```typescript +import { describe, it, expect } from 'vitest' +import { Brainy } from '../src' + +describe('Feature Name', () => { + it('should do something specific', async () => { + const brain = new Brainy() + await brain.init() + + // Test implementation + const result = await brain.search("test") + + expect(result).toBeDefined() + expect(result.length).toBeGreaterThan(0) + }) +}) +``` + +## Architecture Guidelines + +### Adding New Features + +1. **Check existing functionality** + - Review `ARCHITECTURE.md` + - Check if similar features exist + - Consider if it should be an augmentation + +2. **Design considerations** + - Maintain backward compatibility + - Consider performance impact + - Think about all storage adapters + - Plan for extensibility + +3. **Implementation checklist** + - [ ] Core functionality + - [ ] Tests (unit and integration) + - [ ] Documentation + - [ ] TypeScript types + - [ ] Examples + - [ ] Performance benchmarks (if applicable) + +### Creating Augmentations + +Augmentations extend Brainy's functionality: + +```typescript +import { BrainyAugmentation } from '../types' + +export class MyAugmentation extends BrainyAugmentation { + name = 'MyAugmentation' + + async onInit(brain: Brainy): Promise { + // Initialize augmentation + } + + async onAdd(item: any, brain: Brainy): Promise { + // Process before adding + return item + } + + async onSearch(query: any, results: any[], brain: Brainy): Promise { + // Process search results + return results + } +} +``` + +### Performance Considerations + +- Use batch operations where possible +- Implement caching strategically +- Consider memory usage +- Profile performance impacts +- Add benchmarks for critical paths + +## Documentation + +### API Documentation + +Use JSDoc for all public APIs: + +```typescript +/** + * Searches for similar items using vector similarity + * @param query - Search query (text or vector) + * @param options - Search options + * @returns Array of search results with scores + * @example + * ```typescript + * const results = await brain.search("machine learning", { limit: 10 }) + * ``` + */ +async search(query: string | Vector, options?: SearchOptions): Promise { + // Implementation +} +``` + +### Examples + +Add examples for new features: + +```typescript +// examples/feature-name.ts +import { Brainy } from 'brainy' + +async function exampleUsage() { + const brain = new Brainy() + await brain.init() + + // Show feature usage + // Include comments explaining what's happening + // Handle errors appropriately +} + +exampleUsage().catch(console.error) +``` + +## Release Process + +1. **Version bump**: Follow semantic versioning +2. **Update CHANGELOG**: Document all changes +3. **Run tests**: Ensure all tests pass +4. **Build**: Generate distribution files +5. **Tag**: Create git tag for version +6. **Publish**: Release to npm + +## Getting Help + +- **Discord**: Join our community +- **Issues**: Ask questions on GitHub +- **Discussions**: Share ideas and get feedback + +## Recognition + +Contributors will be recognized in: +- CHANGELOG.md for their contributions +- README.md contributors section +- GitHub contributors page + +Thank you for contributing to Brainy! 🧠 \ No newline at end of file diff --git a/README.md b/README.md index 2caf6493..d1342f52 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@

npm version npm downloads - CI + CI Documentation MIT License TypeScript @@ -23,9 +23,8 @@ Quick start · One query · Features · - Scale with Cor · - Docs · - Support + Scale with Cor · + Docs

--- @@ -173,11 +172,9 @@ await brain.vfs.search('React components with hooks') // semantic file **[Multi-process model](docs/concepts/multi-process.md)** · **[Inspection guide](docs/guides/inspection.md)** -## When you outgrow Brainy +## From laptop to hundreds of millions -Brainy's pure-TypeScript engines carry real workloads a long way on their own — see the measured, per-operation numbers (not marketing figures) in **[docs/performance-envelopes.md](docs/performance-envelopes.md)** for what to expect, unaccelerated, on plain filesystem storage. - -When a deployment needs native-scale vector/graph performance — memory-mapped indexes that don't need your dataset in RAM, billion-scale ambitions — add the native engine. **The API doesn't change:** +Brainy's TypeScript engines take you a long way. When you outgrow them, add the native engine — **the API doesn't change**: ```bash npm install @soulcraft/cor @@ -190,14 +187,13 @@ await brain.init() // @soulcraft/cor detected — same code, native engines un Installing the package is the opt-in: if `@soulcraft/cor` is present, it loads and announces itself in the init log; if it's present but broken, `init()` **throws** — an installed accelerator never silently vanishes behind the JS engines. Opt out with `plugins: []`, or pin exactly what loads with `plugins: ['@soulcraft/cor']`. [`@soulcraft/cor`](https://www.npmjs.com/package/@soulcraft/cor) (Brainy 8.x ↔ Cor 3.x, version-matched) registers Rust implementations behind every provider seam: SIMD distance kernels, memory-mapped storage, a disk-native vector index that doesn't need your dataset in RAM, durable LSM field/graph indexes that serve cold opens instantly, and native aggregation. Recall@10 measured **0.99 / 0.96 / 0.96 at 1M / 10M / 100M vectors** in Cor's release gate. -Open core, commercial accelerator: Brainy is MIT and complete on its own — Cor is more headroom for when you need it, not capability held back to sell you later. Licensing and support: **cor@soulcraft.com**. +Open core, commercial accelerator: Brainy is MIT and complete on its own; Cor is licensed and funds both. ## Performance -- Per-operation p50/p95 at 1k and 10k entities, pure-JS floor, measured and re-run every release that touches a measured path: **[docs/performance-envelopes.md](docs/performance-envelopes.md)**. - JS distance kernels: **~6× faster cosine, ~1.4× euclidean** than 7.x (measured: [`tests/benchmarks/distance-microbench.mjs`](tests/benchmarks/distance-microbench.mjs), 384-dim, median of 41). - Whole-graph reads are single **O(N + E)** cursor walks — a consumer-measured 19k-edge export dropped from ~27 s of per-node calls to one scan. -- Capacity planning and architecture: **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)** · **[docs/SCALING.md](docs/SCALING.md)** +- Full numbers and capacity planning: **[docs/PERFORMANCE.md](docs/PERFORMANCE.md)** · **[docs/SCALING.md](docs/SCALING.md)** ## Use cases @@ -216,10 +212,6 @@ Open core, commercial accelerator: Brainy is MIT and complete on its own — Cor **Bun ≥ 1.1** (recommended) or **Node.js ≥ 22**. Brainy 8.x is server-only; the 7.x line remains on npm for browser use. -## Support & community +## Contributing & license -- **Bugs and ideas** → **brainy@soulcraft.com** — no account needed, you'll get a receipt. -- **Security reports** → **security@soulcraft.com** — see **[SECURITY.md](SECURITY.md)**. -- **Contributing** → see **[CONTRIBUTING.md](CONTRIBUTING.md)**. - -MIT © Brainy Contributors. +Contributions welcome — see **[CONTRIBUTING.md](CONTRIBUTING.md)**. MIT © Brainy Contributors. diff --git a/RELEASES.md b/RELEASES.md index 03d7283a..89bf38e7 100644 --- a/RELEASES.md +++ b/RELEASES.md @@ -31,68 +31,6 @@ is sometimes cited as a 7.x removal — those methods never existed on 7.x; the --- -## v8.10.1 — 2026-07-24 (the no-hot-retry contract + warm()'s metadata surface under native providers) - -From a production incident: a native-provider op ground 38-40s inside a transaction, -blew the ~32s apply-phase budget, was rolled back (zero loss, by design), and a -downstream pipeline hot-retried the identical operation into a 6-minute, 100%-CPU -storm. Investigation confirmed Brainy itself never auto-retries a timed-out -transaction — the storm was entirely the consumer's own retry loop, driven by a -"retryable" doc-prose claim with no machine-readable contract to branch on. This -release closes that contract gap and, separately, fixes a real `warm()` reporting gap -surfaced by the same investigation. - -- **`TransactionTimeoutError` is now a machine-readable no-hot-retry contract.** Two - new typed, always-`true` fields replace prose-only guidance: - - `retryable: true` — the operation MAY succeed on a later attempt, once the - underlying slowness resolves or the budget is deliberately raised - (`transactionBudgetFloorMs`, or a batch's own `timeoutMs` override). - - `hotRetryUnsafe: true` — an immediate, identical retry re-pays the FULL cost of - the work that just timed out (it does not resume partway) and can cascade into - exactly the CPU storm above. **Never loop on this error.** The documented pattern - is a latch, not a retry loop: - ``` - on TransactionTimeoutError: - record { at: Date.now(), error } - rethrow loudly to your own caller - hold a cooldown window before any re-attempt - clear the latch only on a subsequent success - ``` - - `context` (unchanged, now fully documented) carries the backoff inputs: - `timeoutMs`, `operationIndex`, `elapsedMs`, `totalOperations`, `operationName`. - - Every "retryable" doc-prose site referencing this error (`transact()`'s - `timeoutMs` option, `transactionBudgetFloorMs`, `Transaction.execute()`) now - points at these fields instead of bare prose. - - Regression-pinned: the engine never internally re-drives a timed-out operation - (verified via an execution counter through both the single-op write path and - `add()`'s upsert-race retry loop), so this has always been true — it is now - provable and typed. -- **Dead code removed**: `TransactionManager.executeTransactionWithResult()` had zero - callers in this codebase and is deleted. -- **`brain.warm()`'s metadata surface now routes through the ACTIVE provider.** A - production deployment's warm report showed `metadata: 'unavailable'` under a native - metadata provider — the previous logic only duck-typed the built-in JS manager's - `hydrateAll()` method, which a native provider has no reason to implement. The - metadata provider contract (`MetadataIndexProvider`, `src/plugin.ts`) gains an - optional `warm?(): Promise` hook, mirroring the existing vector and graph - provider hooks. `brain.warm()` now checks the active provider's own `warm()` FIRST, - falls back to the JS manager's `hydrateAll()` when absent, and only reports - `'unavailable'` when neither exists — never `init()` as a stand-in, since a native - provider's `init()` may be a cheap verify rather than a real warm. A native - provider lights this surface up the same way `@soulcraft/cor` already lights the - vector and graph surfaces: implement `warm()` on its metadata provider. -- **New: `brain.maintenanceDebt()`** — the observability seam so an operator sees a - provider's outstanding background maintenance work (pending bytes/items, last pass - outcome, whether it's converging) BEFORE it grinds into the kind of budget-busting - op this release's timeout contract exists for, instead of discovering it as a CPU - storm. It is a pure passthrough: brainy applies no thresholds, no polling, and no - estimation — it calls each active provider's own optional `maintenanceDebt?()` hook - (vector, metadata, graph — the same three contracts `warm?()` lives on) and reports - the payload verbatim, or `'unavailable'` when a surface's provider doesn't track - debt. Useful as a pre-warm/post-warm check or a boot gate. `@soulcraft/cor` does not - yet implement the hook as of this release — expect it on cor's next release; until - then all three surfaces honestly report `'unavailable'`. - ## Unreleased (the warm contract: cold-restart writes stop paying demand-load latency) From a production deployment's cold-restart incident: the FIRST writes after every diff --git a/SECURITY.md b/SECURITY.md deleted file mode 100644 index 1f3c4732..00000000 --- a/SECURITY.md +++ /dev/null @@ -1,36 +0,0 @@ -# Security Policy - -## Reporting a vulnerability - -Email **security@soulcraft.com**. That's the one door for security reports -across the company, and it works the same way for Brainy: every report is -read by a human, you'll get a private receipt, and we'll work with you on -coordinated disclosure — please don't open a public issue for anything -that isn't already public. - -Include what you'd want if you were on the other end: affected version, -how to reproduce, and what you think the impact is. If you have a patch or -a suggested fix, send it along — it's welcome but not required. - -There is no bounty program today. We're saying that plainly so you know -what to expect going in. - -## Response time - -We respond as fast as truth allows. That means: no fixed SLA, no promise of -a reply within a specific number of hours — but a real report from a real -person gets read promptly and taken seriously. If you haven't heard anything -in a reasonable stretch, a follow-up email is completely fine. - -## Supported versions - -The latest `8.x` minor release line receives security fixes. If you're -running an older major version, please upgrade before reporting — we can't -commit to backporting fixes to unsupported lines. - -## Scope - -This policy covers the `@soulcraft/brainy` package itself — the code in -this repository. If you're evaluating a deployment that also uses -`@soulcraft/cor`, report issues in that package the same way, to the same -address; we'll route internally. diff --git a/package-lock.json b/package-lock.json index d0c7b9d9..fb9262e9 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@soulcraft/brainy", - "version": "8.10.1", + "version": "8.9.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@soulcraft/brainy", - "version": "8.10.1", + "version": "8.9.0", "license": "MIT", "dependencies": { "@msgpack/msgpack": "^3.1.2", diff --git a/package.json b/package.json index ce670369..7366ce98 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@soulcraft/brainy", - "version": "8.10.1", + "version": "8.9.0", "description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. Stage 3 CANONICAL: 42 nouns × 127 verbs covering 96-97% of all human knowledge.", "main": "dist/index.js", "module": "dist/index.js", @@ -128,13 +128,13 @@ "publishConfig": { "access": "public" }, - "homepage": "https://source.soulcraft.com/soulcraft/brainy", + "homepage": "https://github.com/soulcraftlabs/brainy", "bugs": { - "url": "https://source.soulcraft.com/soulcraft/brainy/issues" + "url": "https://github.com/soulcraftlabs/brainy/issues" }, "repository": { "type": "git", - "url": "git+https://source.soulcraft.com/soulcraft/brainy.git" + "url": "git+https://github.com/soulcraftlabs/brainy.git" }, "files": [ "dist/**/*.js", diff --git a/scripts/release.sh b/scripts/release.sh index 43fa50bd..7d860564 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -142,7 +142,7 @@ else fi # Create new changelog entry -CHANGELOG_ENTRY="### [${NEW_VERSION}](https://source.soulcraft.com/soulcraft/brainy/compare/v${CURRENT_VERSION}...v${NEW_VERSION}) ($(date +%Y-%m-%d)) +CHANGELOG_ENTRY="### [${NEW_VERSION}](https://github.com/soulcraftlabs/brainy/compare/v${CURRENT_VERSION}...v${NEW_VERSION}) ($(date +%Y-%m-%d)) ${COMMITS} " @@ -175,59 +175,26 @@ echo -e "${BLUE}7️⃣ Creating git tag v${NEW_VERSION}...${NC}" git tag -a "v${NEW_VERSION}" -m "Release v${NEW_VERSION}" echo -e "${GREEN}✅ Tag created${NC}\n" -# Step 9: Push to origin — the forge is the one home (ruled 2026-07-23; the -# old public GitHub repo is archived history, no longer part of any release). -echo -e "${BLUE}8️⃣ Pushing to origin...${NC}" +# Step 9: Push to GitHub +echo -e "${BLUE}8️⃣ Pushing to GitHub...${NC}" git push --follow-tags origin "$CURRENT_BRANCH" -echo -e "${GREEN}✅ Pushed to origin${NC}\n" +echo -e "${GREEN}✅ Pushed to GitHub${NC}\n" -# Step 10: Publish — forge FIRST (home), npmjs second (the world's storefront). -# The fleet-wide ~/.npmrc maps the @soulcraft scope to the forge registry, and -# a scope mapping BEATS `--registry` on the command line — so each publish -# names its registry via the scope override explicitly. Nothing implicit. -FORGE_NPM_REG="https://source.soulcraft.com/api/packages/soulcraft/npm/" -FORGE_NPM_TOKEN_FILE="$HOME/.config/soulcraft/npm-publish-brainy.token" -echo -e "${BLUE}9️⃣ Publishing to the forge registry (home)...${NC}" -if [ -f "$FORGE_NPM_TOKEN_FILE" ]; then - TMPRC="$(mktemp)" - chmod 600 "$TMPRC" - { - echo "@soulcraft:registry=${FORGE_NPM_REG}" - echo "//source.soulcraft.com/api/packages/soulcraft/npm/:_authToken=$(cat "$FORGE_NPM_TOKEN_FILE")" - } > "$TMPRC" - if npm publish --tag "$NPM_TAG" --userconfig "$TMPRC"; then - echo -e "${GREEN}✅ Published to the forge${NC}\n" - else - rm -f "$TMPRC" - echo -e "${RED}❌ Forge publish FAILED — aborting before npmjs so the pair never diverges. Fix and re-run.${NC}" - exit 1 - fi - rm -f "$TMPRC" -else - echo -e "${RED}❌ Forge publish token missing (${FORGE_NPM_TOKEN_FILE}) — aborting. The forge is home; publish it first or restage the token.${NC}" - exit 1 -fi - -echo -e "${BLUE}9️⃣½ Publishing to npmjs (storefront, dist-tag: ${NPM_TAG})...${NC}" -npm publish --tag "$NPM_TAG" "--@soulcraft:registry=https://registry.npmjs.org/" +# Step 10: Publish to npm +echo -e "${BLUE}9️⃣ Publishing to npm (dist-tag: ${NPM_TAG})...${NC}" +npm publish --tag "$NPM_TAG" # Brainy is the only PUBLIC @soulcraft package — verify visibility after every publish. -npm access get status @soulcraft/brainy "--@soulcraft:registry=https://registry.npmjs.org/" || true -echo -e "${GREEN}✅ Published to npmjs${NC}\n" +npm access get status @soulcraft/brainy || true +echo -e "${GREEN}✅ Published to npm${NC}\n" -# Step 11: Release object on the forge (presentational — the tag, CHANGELOG, -# and RELEASES.md are the record; this just gives the forge UI a release page). -echo -e "${BLUE}🔟 Creating forge release...${NC}" -if [ -n "${FORGEJO_RELEASE_TOKEN:-}" ]; then - if curl -sf -X POST "https://source.soulcraft.com/api/v1/repos/soulcraft/brainy/releases" \ - -H "Authorization: token ${FORGEJO_RELEASE_TOKEN}" -H "Content-Type: application/json" \ - -d "{\"tag_name\":\"v${NEW_VERSION}\",\"name\":\"v${NEW_VERSION}\",\"prerelease\":${PRERELEASE}}" >/dev/null; then - echo -e "${GREEN}✅ Forge release created${NC}\n" - else - echo -e "${RED}⚠️ Forge release API call failed — tag + CHANGELOG remain the record; create the release page via the forge UI if wanted${NC}\n" - fi +# Step 11: Create GitHub release +echo -e "${BLUE}🔟 Creating GitHub release...${NC}" +if [ "$PRERELEASE" = true ]; then + gh release create "v${NEW_VERSION}" --generate-notes --prerelease else - echo -e "${RED}⚠️ FORGEJO_RELEASE_TOKEN unset — no release page created; tag + CHANGELOG remain the record${NC}\n" + gh release create "v${NEW_VERSION}" --generate-notes fi +echo -e "${GREEN}✅ GitHub release created${NC}\n" # Step 12: Push public docs to the soulcraft.com docs ingest door # (VENUE-DOCS-RELEASE-PUSH). Skips with a loud warning when @@ -246,4 +213,4 @@ echo -e "${GREEN}🎉 Release ${NEW_VERSION} complete!${NC}" echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" echo "" echo -e "📦 npm: ${BLUE}https://www.npmjs.com/package/@soulcraft/brainy/v/${NEW_VERSION}${NC}" -echo -e "🏠 Forge: ${BLUE}https://source.soulcraft.com/soulcraft/brainy/releases/tag/v${NEW_VERSION}${NC}" +echo -e "🐙 GitHub: ${BLUE}https://github.com/soulcraftlabs/brainy/releases/tag/v${NEW_VERSION}${NC}" diff --git a/src/brainy.ts b/src/brainy.ts index f6b09e25..fd9e1ffa 100644 --- a/src/brainy.ts +++ b/src/brainy.ts @@ -65,8 +65,7 @@ import type { OpaqueIdSet, AtGenerationVectors, VectorIndexProvider, - GraphIndexProvider, - ProviderMaintenanceDebt + GraphIndexProvider } from './plugin.js' import type { BrainyPlugin, @@ -425,30 +424,6 @@ export interface WarmReport { totalDurationMs: number } -/** - * @description Result of {@link Brainy.maintenanceDebt}: one outcome per - * index surface, mirroring {@link WarmReport}'s shape. - * - `'reported'` — the active provider for this surface implements - * `maintenanceDebt?()` and its {@link ProviderMaintenanceDebt} payload is - * attached verbatim under `debt`. - * - `'unavailable'` — the active provider does not implement the hook, so - * nothing is known; brainy never estimates or infers a payload on its - * behalf. - */ -export type MaintenanceDebtOutcome = 'reported' | 'unavailable' - -/** - * @description Per-surface result of {@link Brainy.maintenanceDebt}. Brainy - * performs no thresholding, polling, or estimation over this data — it is a - * pure passthrough of each active provider's own self-report (the provider - * owns the numbers; the operator owns the policy). - */ -export interface MaintenanceDebtReport { - vector: { outcome: MaintenanceDebtOutcome; debt?: ProviderMaintenanceDebt } - metadata: { outcome: MaintenanceDebtOutcome; debt?: ProviderMaintenanceDebt } - graph: { outcome: MaintenanceDebtOutcome; debt?: ProviderMaintenanceDebt } -} - /** * How long a failed aggregation-backfill walk suppresses fresh walk attempts. * Within the window, queries rethrow the recorded failure instantly (loud, @@ -14377,16 +14352,9 @@ export class Brainy implements BrainyInterface { * *some* backing storage as a side effect but is reported honestly as * `'probed'`, never `'warmed'`. An empty index or unknown dimension has * nothing to probe (`'unavailable'`). - * - **Metadata**: calls the provider's own `warm?()` when the active - * `'metadataIndex'` provider implements it (`'warmed'`) — the seam a - * native metadata provider lights up so it is not duck-typed against the - * JS manager's method. Otherwise falls back to full hydration on the - * built-in JS manager — every persisted field's sparse index is loaded - * from storage (`MetadataIndexManager.hydrateAll()`), not just the - * heuristic common-fields subset `init()` warms — and reports `'warmed'`. - * Neither seam present → `'unavailable'` (honest: `init()` is never used - * as a substitute here, since a native provider's `init()` may be a - * cheap verify rather than a real warm). + * - **Metadata**: full hydration — every persisted field's sparse index is + * loaded from storage (`MetadataIndexManager.hydrateAll()`), not just the + * heuristic common-fields subset `init()` warms. * - **Graph**: calls the provider's own `warm?()` when the active graph * provider implements it; otherwise re-runs its existing eager cold-load * `init()` seam (idempotent — the JS adjacency index's `init()` already @@ -14442,23 +14410,12 @@ export class Brainy implements BrainyInterface { // --- Metadata -------------------------------------------------------- const metadataStart = Date.now() let metadataOutcome: WarmOutcome - const metadataProvider = this.metadataIndex as unknown as MetadataIndexProvider const metadataWithHydrate = this.metadataIndex as unknown as { hydrateAll?: () => Promise } - if (typeof metadataProvider.warm === 'function') { - // Active provider (e.g. a native metadata index) declares its own warm - // seam — route through it FIRST so a native provider's warmth is - // reported honestly instead of being duck-typed against the JS - // manager's hydrateAll(), which a native provider does not implement. - await metadataProvider.warm() - metadataOutcome = 'warmed' - } else if (typeof metadataWithHydrate.hydrateAll === 'function') { - // Built-in JS manager path — full sparse-index hydration. + if (typeof metadataWithHydrate.hydrateAll === 'function') { await metadataWithHydrate.hydrateAll() metadataOutcome = 'warmed' } else { - // No hydration seam on this metadata provider — nothing to run. (No - // init() fallback here: init() on a native provider may be a cheap - // verify, and reporting that as warmth would lie.) + // No hydration seam on this metadata provider — nothing to run. metadataOutcome = 'unavailable' } const metadataDurationMs = Date.now() - metadataStart @@ -14492,63 +14449,6 @@ export class Brainy implements BrainyInterface { } } - /** - * Read each index surface's self-reported outstanding maintenance work — - * the observability seam so an operator sees a grind coming (rising - * pending bytes/items, a stalled background pass) instead of discovering - * it as a CPU storm or a transaction blowing its budget mid-flight (see - * {@link TransactionTimeoutError}). - * - * PURE PASSTHROUGH: for each of vector/metadata/graph, this calls ONLY the - * ACTIVE provider's own `maintenanceDebt?()` hook (the same per-surface - * provider resolution {@link Brainy.warm} uses) and reports its - * {@link ProviderMaintenanceDebt} payload verbatim. There is no JS-side - * fallback computation, no threshold evaluation, and no polling — brainy - * surfaces the truth the provider measured; the provider owns the numbers - * and the operator owns the policy (what threshold matters, what action to - * take). A surface whose active provider does not implement the hook - * reports `'unavailable'` — never a guessed or zeroed payload. - * - * @returns A {@link MaintenanceDebtReport}: per-surface outcome + payload. - * @example - * ```typescript - * const debt = await brain.maintenanceDebt() - * if (debt.metadata.outcome === 'reported' && debt.metadata.debt?.pendingBytes) { - * console.log('metadata pending bytes:', debt.metadata.debt.pendingBytes) - * } - * ``` - */ - async maintenanceDebt(): Promise { - await this.ensureInitialized({ needs: ['vector', 'metadata', 'graph'] }) - - // --- Vector --------------------------------------------------------- - const vectorProvider = this.index as VectorIndexProvider & { - maintenanceDebt?: () => Promise - } - const vector = - typeof vectorProvider.maintenanceDebt === 'function' - ? { outcome: 'reported' as const, debt: await vectorProvider.maintenanceDebt() } - : { outcome: 'unavailable' as const } - - // --- Metadata -------------------------------------------------------- - const metadataProvider = this.metadataIndex as unknown as MetadataIndexProvider - const metadata = - typeof metadataProvider.maintenanceDebt === 'function' - ? { outcome: 'reported' as const, debt: await metadataProvider.maintenanceDebt() } - : { outcome: 'unavailable' as const } - - // --- Graph ------------------------------------------------------------- - const graphProvider = this.graphIndex as GraphIndexProvider & { - maintenanceDebt?: () => Promise - } - const graph = - typeof graphProvider.maintenanceDebt === 'function' - ? { outcome: 'reported' as const, debt: await graphProvider.maintenanceDebt() } - : { outcome: 'unavailable' as const } - - return { vector, metadata, graph } - } - /** * Explicitly warm up the embedding engine * diff --git a/src/db/types.ts b/src/db/types.ts index a7f86a89..4c8a4957 100644 --- a/src/db/types.ts +++ b/src/db/types.ts @@ -121,13 +121,9 @@ export interface TransactOptions { * with the batch: `max(30 000, opCount × 2 000)` — production imports on * network-attached disks measure ~2 s per operation, so a flat 30 s budget * silently capped honest bulk work at ~15 operations. A tripped budget - * rolls the whole batch back and throws a `TransactionTimeoutError` naming - * the operation it stopped at, the batch size, and the elapsed/budget - * times. That error is retryable-with-latch, never hot-retry: its - * `retryable` field says a later attempt may succeed, its - * `hotRetryUnsafe` field says an immediate identical retry re-pays the - * full cost that just timed out — callers must latch and back off, never - * loop. + * rolls the whole batch back and throws a retryable + * `TransactionTimeoutError` naming the operation it stopped at, the batch + * size, and the elapsed/budget times. */ timeoutMs?: number } diff --git a/src/index.ts b/src/index.ts index 70d43edc..bfab39da 100644 --- a/src/index.ts +++ b/src/index.ts @@ -31,11 +31,6 @@ export type { DiagnosticsResult } from './brainy.js' // brain.warm() — eager index/storage readiness report (per-surface honest // outcome + timing). See the WarmReport JSDoc in brainy.ts. export type { WarmReport, WarmOutcome } from './brainy.js' -// brain.maintenanceDebt() — per-surface passthrough of each active -// provider's self-reported background maintenance debt. See the -// MaintenanceDebtReport JSDoc in brainy.ts and ProviderMaintenanceDebt in -// plugin.ts for the measure-only-what-you-track contract. -export type { MaintenanceDebtReport, MaintenanceDebtOutcome } from './brainy.js' export type { GraphAuditReport, GraphAuditDiscrepancy @@ -233,11 +228,6 @@ export type { FamilyStamp, StampMembers, StampVerdict } from './db/familyStamp.j export { isVersionedIndexProvider } from './plugin.js' export type { VersionedIndexProvider } from './plugin.js' export type { ProviderInvariantReport, InvariantResult, InvariantHeal } from './plugin.js' -// Optional provider self-report of outstanding background maintenance work -// (compaction, deferred writes, etc.) — the payload type for -// brain.maintenanceDebt(). See the measure-only-what-you-track contract on -// ProviderMaintenanceDebt in plugin.ts. -export type { ProviderMaintenanceDebt } from './plugin.js' // Optional native graph-acceleration engine (cor 3.0) — the published provider // contract + its columnar wire types. Brainy feature-detects an implementation // and falls back to its pure-TS adjacency when absent. diff --git a/src/plugin.ts b/src/plugin.ts index ce973386..02639955 100644 --- a/src/plugin.ts +++ b/src/plugin.ts @@ -171,37 +171,6 @@ export interface ProviderInvariantReport { durationMs: number } -/** - * @description A provider's self-report of its own outstanding background - * maintenance work (compaction, deferred writes, a build-new→verify→swap in - * flight, etc.) — the observability seam so an operator sees a grind coming - * (rising pending bytes/items, a stalled pass) instead of discovering it as a - * CPU storm or a timeout under transaction budget pressure. Every field is - * OPTIONAL and every field is a MEASUREMENT: a provider reports ONLY what it - * actually tracks, never an estimate dressed up as a fact. Absence of the - * {@link VectorIndexProvider.maintenanceDebt} / - * {@link GraphIndexProvider.maintenanceDebt} / - * {@link MetadataIndexProvider.maintenanceDebt} hook itself means the - * provider does not track debt at all — brainy reports that surface - * `'unavailable'` rather than inventing zeros. Brainy performs NO threshold - * checks, NO polling, and NO JS-side estimation over this payload — it is a - * pure passthrough via {@link Brainy.maintenanceDebt}; the provider owns the - * numbers and the operator owns the policy (what threshold matters, what to - * do about it). - */ -export interface ProviderMaintenanceDebt { - /** Bytes of outstanding/unmerged work, if the provider measures it (e.g. unflushed writes, unmerged segments). */ - pendingBytes?: number - /** Count of outstanding items (records, segments, nodes) awaiting the provider's background pass. */ - pendingItems?: number - /** Epoch millis when the provider's last maintenance pass finished, if it tracks one. */ - lastPassCompletedAt?: number - /** How the last pass ended, if the provider tracks pass outcomes. */ - lastPassOutcome?: 'completed' | 'partial' | 'failed' - /** `true` if the provider's own measurements show debt trending down (making progress); `false` if flat or growing; omitted if the provider can't tell. */ - converging?: boolean -} - /** * The `'metadataIndex'` provider — a drop-in for `MetadataIndexManager`. * Brainy calls this surface via `this.metadataIndex.*` (see `brainy.ts`) and @@ -212,34 +181,6 @@ export interface MetadataIndexProvider { flush(): Promise rebuild(): Promise - /** - * @description OPTIONAL. Eagerly load/fault-in backing storage (e.g. mmap - * pretouch, full sparse-index hydration) so first queries run at - * steady-state cost. Optional; absence means the provider demand-loads. - * Mirrors {@link GraphIndexProvider.warm} / the vector provider's `warm?()` - * (`src/plugin.ts` VectorIndexProvider). Distinct from `init()`: `init` is - * required and runs once automatically during brain startup; `warm` is a - * separate, explicit step a caller opts into via `brain.warm()` (or - * `warmOnOpen`) to pre-pay demand-load cost `init` left lazy. Idempotent — - * calling it more than once must be safe and cheap on a brain that is - * already warm. A provider that already loads everything eagerly in - * `init()` may implement `warm` as a no-op or omit it — `brain.warm()` - * falls back to the built-in JS manager's `hydrateAll()` duck-type when - * absent, and to an honest `'unavailable'` when neither exists. - */ - warm?(): Promise - - /** - * @description OPTIONAL self-reported {@link ProviderMaintenanceDebt} — - * the observability seam so an operator sees outstanding background - * maintenance work (e.g. unmerged postings) BEFORE it grinds a transaction - * into a budget-busting op. Absence means this provider does not track - * debt; `brain.maintenanceDebt()` reports this surface `'unavailable'` - * rather than guessing. See {@link ProviderMaintenanceDebt} for the - * measure-only-what-you-track contract. - */ - maintenanceDebt?(): Promise - /** * @description OPTIONAL honest durability signal (readiness contract, * mirrors `isReady?()` on the graph and vector providers). `true` ⇔ the @@ -454,18 +395,6 @@ export interface GraphIndexProvider { */ warm?(): Promise - /** - * @description OPTIONAL self-reported {@link ProviderMaintenanceDebt} — - * the observability seam so an operator sees outstanding background - * maintenance work (e.g. a build-new→verify→swap in flight, unmerged - * adjacency segments) BEFORE it grinds a transaction into a - * budget-busting op. Absence means this provider does not track debt; - * `brain.maintenanceDebt()` reports this surface `'unavailable'` rather - * than guessing. See {@link ProviderMaintenanceDebt} for the - * measure-only-what-you-track contract. - */ - maintenanceDebt?(): Promise - /** * @description OPTIONAL. A native provider returns true from the moment its * `init()` detects a large epoch-drift until its background @@ -1128,18 +1057,6 @@ export interface VectorIndexProvider { */ warm?(): Promise - /** - * @description OPTIONAL self-reported {@link ProviderMaintenanceDebt} — - * the observability seam so an operator sees outstanding background - * maintenance work (e.g. unflushed writes, a pending rebuild) BEFORE it - * grinds a transaction into a budget-busting op. Absence means this - * provider does not track debt; `brain.maintenanceDebt()` reports this - * surface `'unavailable'` rather than guessing. See - * {@link ProviderMaintenanceDebt} for the measure-only-what-you-track - * contract. - */ - maintenanceDebt?(): Promise - /** * @description OPTIONAL honest durability signal (readiness contract, * mirrors {@link GraphIndexProvider.isReady}). `true` ⇔ the persisted diff --git a/src/transaction/Transaction.ts b/src/transaction/Transaction.ts index ede65e83..79b53006 100644 --- a/src/transaction/Transaction.ts +++ b/src/transaction/Transaction.ts @@ -62,10 +62,9 @@ const DEFAULT_BUDGET_FLOOR_MS = 30_000 * NEXT operation may start (see {@link Transaction.execute}), never whether * already-completed work is rolled back after the fact. A trip mid-batch * still rolls back every operation applied so far, atomically, and throws a - * fully-labeled `TransactionTimeoutError` — retryable-with-latch, never - * hot-retry (see its `retryable` and `hotRetryUnsafe` fields) — that - * zero-loss guarantee doesn't change; only the point at which the clock - * stops mattering does (at the last operation, not one check later). + * retryable, fully-labeled TransactionTimeoutError — that zero-loss guarantee + * doesn't change; only the point at which the clock stops mattering does (at + * the last operation, not one check later). * * @param opCount - Number of operations in the batch. * @param override - A full override for this call; wins over everything else. diff --git a/src/transaction/TransactionManager.ts b/src/transaction/TransactionManager.ts index 5abf48ba..0f13b6a3 100644 --- a/src/transaction/TransactionManager.ts +++ b/src/transaction/TransactionManager.ts @@ -19,6 +19,7 @@ import { Transaction } from './Transaction.js' import { TransactionFunction, + TransactionResult, TransactionOptions } from './types.js' import { TransactionError } from './errors.js' @@ -104,6 +105,34 @@ export class TransactionManager { } } + /** + * Execute a transaction and return detailed result + */ + async executeTransactionWithResult( + fn: TransactionFunction, + options?: TransactionOptions + ): Promise> { + const startTime = Date.now() + const transaction = new Transaction(options) + + try { + const value = await fn(transaction) + await transaction.execute() + + const executionTimeMs = Date.now() - startTime + + return { + value, + operationCount: transaction.getOperationCount(), + executionTimeMs + } + + } catch (error) { + // Transaction failed + throw error + } + } + /** * Get transaction statistics */ diff --git a/src/transaction/errors.ts b/src/transaction/errors.ts index d8382a4b..c270d0ed 100644 --- a/src/transaction/errors.ts +++ b/src/transaction/errors.ts @@ -73,47 +73,14 @@ export class InvalidTransactionStateError extends TransactionError { /** * Error for transaction timeout - * - * Machine-readable no-hot-retry contract: {@link retryable} and - * {@link hotRetryUnsafe} are both always `true` on this class — they exist - * so a caller can branch on the *shape* of the error instead of parsing - * message text. Read them together: the operation may eventually succeed, - * but never by looping on it immediately. - * - * `context` (inherited from {@link TransactionError}) carries the caller's - * backoff inputs — see the field docs below. */ export class TransactionTimeoutError extends TransactionError { - /** - * The failed operation MAY succeed on a later attempt — once the - * underlying slowness resolves (e.g. a cold page cache warms up) or the - * budget is deliberately raised (`transactionBudgetFloorMs`, or a larger - * `timeoutMs` override on the batch). This is a statement about eventual - * retryability, not a license to retry now — see {@link hotRetryUnsafe}. - */ - public readonly retryable = true - - /** - * An immediate, identical retry re-pays the FULL cost of the work that - * just timed out — it does not resume partway. Looping on this error - * (hot-retrying) repeats that full cost every attempt and can cascade - * into a CPU/resource storm on the caller's side. Callers MUST latch: on - * this error, record `{ at: Date.now(), error }`, surface one loud - * failure to their own caller, and hold a cooldown window before any - * re-attempt (clearing the latch only on success). Never retry this error - * in a tight loop. - */ - public readonly hotRetryUnsafe = true - constructor( timeoutMs: number, operationIndex: number, telemetry?: { - /** Milliseconds elapsed in the transaction when the budget tripped. */ elapsedMs?: number - /** Total number of operations in the batch that timed out. */ totalOperations?: number - /** Name of the operation the batch was about to start when it tripped, if named. */ operationName?: string } ) { @@ -126,16 +93,8 @@ export class TransactionTimeoutError extends TransactionError { telemetry?.elapsedMs !== undefined ? `${telemetry.elapsedMs}ms elapsed, ` : '' super( `Transaction timed out at operation ${progress}${name} — ${elapsed}budget ${timeoutMs}ms. ` + - `The batch rolled back atomically; retryable after the underlying slowness resolves or ` + - `the budget is raised, but hot-retry-unsafe — latch and back off, never loop.`, - { - // Caller backoff inputs — all present on every instance: - /** Configured budget (ms) that was exceeded. */ - timeoutMs, - /** Index of the operation the batch was about to start when it tripped. */ - operationIndex, - ...telemetry - } + `The batch rolled back atomically; retry with a higher timeoutMs or a smaller batch.`, + { timeoutMs, operationIndex, ...telemetry } ) this.name = 'TransactionTimeoutError' } diff --git a/src/transaction/types.ts b/src/transaction/types.ts index 6cbc56ca..9a3a2eaa 100644 --- a/src/transaction/types.ts +++ b/src/transaction/types.ts @@ -66,6 +66,26 @@ export interface TransactionContext { */ export type TransactionFunction = (ctx: TransactionContext) => Promise +/** + * Transaction execution result + */ +export interface TransactionResult { + /** + * Result value from user function + */ + value: T + + /** + * Number of operations executed + */ + operationCount: number + + /** + * Execution time in milliseconds + */ + executionTimeMs: number +} + /** * Transaction execution options */ diff --git a/src/types/brainy.types.ts b/src/types/brainy.types.ts index b5f286ed..8bede8d3 100644 --- a/src/types/brainy.types.ts +++ b/src/types/brainy.types.ts @@ -1658,10 +1658,8 @@ export interface BrainyConfig { * **start** — never whether already-completed work gets rolled back after * the fact (a single-op write can never time out post-hoc: it either runs * or it commits). A trip mid-batch still rolls back every applied operation - * atomically and throws a `TransactionTimeoutError` that is - * retryable-with-latch, never hot-retry (see its `retryable` and - * `hotRetryUnsafe` fields); only the floor of the formula is configurable - * here. + * atomically and throws a retryable `TransactionTimeoutError`; only the + * floor of the formula is configurable here. * * Raise this when a cold store's first writes after a restart legitimately * take longer than 30s per operation (e.g. page-cache-cold canonical writes diff --git a/tests/transaction/TransactionManager.unit.test.ts b/tests/transaction/TransactionManager.unit.test.ts index 29e7f7ae..86b1692c 100644 --- a/tests/transaction/TransactionManager.unit.test.ts +++ b/tests/transaction/TransactionManager.unit.test.ts @@ -5,6 +5,7 @@ * - High-level transaction API * - Statistics tracking * - Error handling + * - Result wrapping */ import { describe, it, expect, beforeEach } from 'vitest' @@ -83,6 +84,42 @@ describe('TransactionManager', () => { }) }) + describe('executeTransactionWithResult', () => { + it('should return detailed result', async () => { + const result = await manager.executeTransactionWithResult(async (tx) => { + tx.addOperation({ + execute: async () => { + await new Promise(resolve => setTimeout(resolve, 1)) + return async () => {} + } + }) + tx.addOperation({ execute: async () => undefined }) + return 'success' + }) + + expect(result.value).toBe('success') + expect(result.operationCount).toBe(2) + expect(result.executionTimeMs).toBeGreaterThanOrEqual(0) + }) + + it('should measure execution time', async () => { + const result = await manager.executeTransactionWithResult(async (tx) => { + tx.addOperation({ + execute: async () => { + await new Promise(resolve => setTimeout(resolve, 25)) + return async () => {} + } + }) + return 'done' + }) + + // Timer coalescing can fire a setTimeout up to a few ms EARLY under + // load, so assert well below the sleep — this tests that time is + // MEASURED, not the OS timer's precision. + expect(result.executionTimeMs).toBeGreaterThanOrEqual(20) + }) + }) + describe('Statistics Tracking', () => { it('should track total transactions', async () => { await manager.executeTransaction(async (tx) => { diff --git a/tests/unit/brainy/maintenance-debt.test.ts b/tests/unit/brainy/maintenance-debt.test.ts deleted file mode 100644 index 4026d655..00000000 --- a/tests/unit/brainy/maintenance-debt.test.ts +++ /dev/null @@ -1,122 +0,0 @@ -/** - * @module tests/unit/brainy/maintenance-debt - * @description Coverage for `brain.maintenanceDebt()` (8.10.1) — the - * observability seam so an operator sees a provider's outstanding background - * maintenance work (compaction, deferred writes, a build-new→verify→swap in - * flight, ...) BEFORE it grinds a transaction into a budget-busting op, the - * same failure class documented on `TransactionTimeoutError` - * (src/transaction/errors.ts). Sibling to tests/unit/brainy/warm.test.ts, - * which establishes this file's technique: shape the probe points brain.ts - * reads (`typeof provider.maintenanceDebt === 'function'`) directly on the - * REAL, live provider instances rather than hand-rolling full fakes for the - * larger `MetadataIndexProvider` / `GraphIndexProvider` interfaces. - * - * `brain.maintenanceDebt()` is a PURE PASSTHROUGH: no thresholds, no - * polling, no JS-side estimation — these tests pin exactly that by asserting - * the returned payload is the provider's object, verbatim. - */ -import { describe, it, expect } from 'vitest' -import { Brainy } from '../../../src/brainy.js' -import { NounType } from '../../../src/types/graphTypes.js' -import type { ProviderMaintenanceDebt } from '../../../src/plugin.js' - -// Brainy's ValidationConfig fixes vectors at exactly 384 dimensions -// (src/utils/paramValidation.ts) — match it so add() doesn't reject test data. -const DIM = 384 -const V = (seed = 1): number[] => Array.from({ length: DIM }, (_, i) => Math.sin(seed + i)) - -async function freshBrain(): Promise> { - const brain = new Brainy({ - requireSubtype: false, - storage: { type: 'memory' }, - silent: true - }) - await brain.init() - await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) }) - return brain -} - -describe('brain.maintenanceDebt()', () => { - it('reports "unavailable" for every surface when no active provider implements maintenanceDebt() (the built-in JS stack today)', async () => { - const brain = await freshBrain() - - const report = await brain.maintenanceDebt() - - expect(report.vector.outcome).toBe('unavailable') - expect(report.vector.debt).toBeUndefined() - expect(report.metadata.outcome).toBe('unavailable') - expect(report.metadata.debt).toBeUndefined() - expect(report.graph.outcome).toBe('unavailable') - expect(report.graph.debt).toBeUndefined() - - await brain.close() - }) - - it('reports "reported" + the exact payload when the active provider implements maintenanceDebt() (verbatim passthrough, no thresholding)', async () => { - const brain = await freshBrain() - - const vectorDebt: ProviderMaintenanceDebt = { - pendingBytes: 4_096, - pendingItems: 12, - lastPassCompletedAt: 1_700_000_000_000, - lastPassOutcome: 'completed', - converging: true - } - ;(brain as any).index.maintenanceDebt = async () => vectorDebt - - const report = await brain.maintenanceDebt() - - expect(report.vector.outcome).toBe('reported') - // Verbatim passthrough — the exact object, not a re-derived copy. - expect(report.vector.debt).toBe(vectorDebt) - // Untouched surfaces stay honestly 'unavailable'. - expect(report.metadata.outcome).toBe('unavailable') - expect(report.graph.outcome).toBe('unavailable') - - await brain.close() - }) - - it('mixed surfaces: each surface\'s outcome depends ONLY on its OWN active provider — one surface reporting never leaks into another', async () => { - const brain = await freshBrain() - - const metadataDebt: ProviderMaintenanceDebt = { - pendingItems: 3, - lastPassOutcome: 'partial', - converging: false - } - const graphDebt: ProviderMaintenanceDebt = { - pendingBytes: 0, - converging: true - } - ;(brain as any).metadataIndex.maintenanceDebt = async () => metadataDebt - ;(brain as any).graphIndex.maintenanceDebt = async () => graphDebt - // Vector is deliberately left unpatched. - - const report = await brain.maintenanceDebt() - - expect(report.vector.outcome).toBe('unavailable') - expect(report.vector.debt).toBeUndefined() - - expect(report.metadata.outcome).toBe('reported') - expect(report.metadata.debt).toBe(metadataDebt) - - expect(report.graph.outcome).toBe('reported') - expect(report.graph.debt).toBe(graphDebt) - - await brain.close() - }) - - it('an empty ProviderMaintenanceDebt object (every field omitted) is still honestly "reported" — presence of the hook, not the payload\'s richness, drives the outcome', async () => { - const brain = await freshBrain() - - const emptyDebt: ProviderMaintenanceDebt = {} - ;(brain as any).graphIndex.maintenanceDebt = async () => emptyDebt - - const report = await brain.maintenanceDebt() - - expect(report.graph.outcome).toBe('reported') - expect(report.graph.debt).toEqual({}) - - await brain.close() - }) -}) diff --git a/tests/unit/brainy/warm.test.ts b/tests/unit/brainy/warm.test.ts index 8a0bb4da..ce213696 100644 --- a/tests/unit/brainy/warm.test.ts +++ b/tests/unit/brainy/warm.test.ts @@ -307,92 +307,4 @@ describe('brain.warm()', () => { expect(report.totalDurationMs).toBeGreaterThanOrEqual(0) await brain.close() }) - - // --- Metadata leg routes through the ACTIVE provider (8.10.1) ----------- - // - // `MetadataIndexProvider` is a ~50-method interface (src/plugin.ts) — far - // too large to hand-write a compliant fake class the way `FakeVectorProvider` - // fakes the ~8-method `VectorIndexProvider` above. Test (c) already - // establishes this file's pattern for the metadata leg: exercise the REAL - // `MetadataIndexManager` instance and shape just the probe points brain.ts - // reads (`typeof provider.warm === 'function'` / - // `typeof provider.hydrateAll === 'function'`) directly on that instance. - // Shadowing an own property on the live object stands in for "a different - // provider implementation" without needing a hand-rolled full fake — the - // rest of the real manager (used by add()/init() above) is untouched. - describe('metadata leg — warm() routes through the active provider', () => { - it('(f) calls the ACTIVE metadata provider\'s warm() when present and reports "warmed", never falling back to hydrateAll', async () => { - const brain = new Brainy({ - requireSubtype: false, - storage: { type: 'memory' }, - silent: true - }) - await brain.init() - await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) }) - - const metadataIndex = (brain as any).metadataIndex - let warmCalls = 0 - let hydrateAllCalls = 0 - const origHydrateAll = metadataIndex.hydrateAll.bind(metadataIndex) - metadataIndex.hydrateAll = async (...args: unknown[]) => { - hydrateAllCalls++ - return origHydrateAll(...args) - } - // Simulates a native metadata provider declaring the optional `warm()` - // hook added to `MetadataIndexProvider` (src/plugin.ts) in 8.10.1. - metadataIndex.warm = async () => { - warmCalls++ - } - - const report = await brain.warm() - - expect(warmCalls).toBe(1) - expect(hydrateAllCalls).toBe(0) // warm() ran — no hydrateAll fallback - expect(report.metadata.outcome).toBe('warmed') - await brain.close() - }) - - it('reports "unavailable" when the active metadata provider implements neither warm() nor hydrateAll() (the honest branch a native provider without either hook must hit)', async () => { - const brain = new Brainy({ - requireSubtype: false, - storage: { type: 'memory' }, - silent: true - }) - await brain.init() - await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) }) - - const metadataIndex = (brain as any).metadataIndex - // Shadow away BOTH optional hooks — models a genuinely native provider - // that (unlike the built-in JS manager) offers neither seam. This must - // never fall back to calling init() as a stand-in for warmth. - metadataIndex.warm = undefined - metadataIndex.hydrateAll = undefined - - const report = await brain.warm() - - expect(report.metadata.outcome).toBe('unavailable') - await brain.close() - }) - - it('the built-in JS manager (no warm()) still reports "warmed" via its existing hydrateAll() duck-type — unchanged by the new provider hook', async () => { - const brain = new Brainy({ - requireSubtype: false, - storage: { type: 'memory' }, - silent: true - }) - await brain.init() - await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) }) - - // No patching at all — the default built-in MetadataIndexManager has - // hydrateAll() but no warm(), exactly as it did before this change. - const metadataIndex = (brain as any).metadataIndex - expect(typeof metadataIndex.warm).not.toBe('function') - expect(typeof metadataIndex.hydrateAll).toBe('function') - - const report = await brain.warm() - - expect(report.metadata.outcome).toBe('warmed') - await brain.close() - }) - }) }) diff --git a/tests/unit/transaction/timeout-never-internally-retried.test.ts b/tests/unit/transaction/timeout-never-internally-retried.test.ts deleted file mode 100644 index a43a81a8..00000000 --- a/tests/unit/transaction/timeout-never-internally-retried.test.ts +++ /dev/null @@ -1,141 +0,0 @@ -/** - * @module tests/unit/transaction/timeout-never-internally-retried - * @description Regression pin for the no-hot-retry contract (8.10.1). - * - * A production incident: a native-provider op ground 38-40s inside a - * transaction, blew the ~32s budget, was rolled back, and a CONSUMER pipeline - * hot-retried the identical operation into a 6-minute 100%-CPU storm. - * Investigation established brainy itself never auto-retries a - * `TransactionTimeoutError` — the storm was entirely the consumer's hot-retry - * loop, driven by a "retryable" doc-prose claim with no machine-readable - * contract. This file pins the brainy-side half of that story so it can never - * regress silently: - * - * (i) the underlying engine (`TransactionManager.executeTransaction()` → - * `Transaction.execute()`) — the exact machinery every single-record - * write (`add`/`update`/`remove`/...) drives via - * `Brainy.persistSingleOp()` — never internally re-executes a timed-out - * operation, and the error it surfaces carries `retryable === true` and - * `hotRetryUnsafe === true` (see `src/transaction/errors.ts`). - * - * Constructed directly (mirrors the existing - * `tests/unit/transaction/timeout-rollback.test.ts` pattern) rather than - * through a real `brain.add()` call: `transactTimeoutBudget()` floors - * every single-op write's budget at `opCount * 2000`ms with NO override - * seam (`transactionBudgetFloorMs` only RAISES that floor — it cannot - * lower it below the per-op-count term), so getting a real `add()` to - * time out requires a multi-second sleep. The engine-level - * `options.timeout` override used here is the exact same - * `TransactionManager`/`Transaction` code `persistSingleOp` calls — - * pinning it here pins add()'s guarantee without paying that wall-clock - * cost. - * - * (ii) `Brainy.add()`'s upsert-race retry loop (src/brainy.ts, - * `MAX_UPSERT_ATTEMPTS = 10`) — proving the loop's `catch` treats a - * `TransactionTimeoutError` as terminal (immediate rethrow) rather than - * the `InsertPreconditionExistsSignal` it retries on, so a mid-flight - * timeout can never be silently swallowed and re-attempted up to 10 - * times. - */ -import { describe, it, expect } from 'vitest' -import { TransactionManager } from '../../../src/transaction/TransactionManager.js' -import type { Operation, RollbackAction } from '../../../src/transaction/types.js' -import { TransactionTimeoutError } from '../../../src/transaction/errors.js' -import { Brainy } from '../../../src/brainy.js' -import { NounType } from '../../../src/types/graphTypes.js' - -const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)) - -// Brainy's ValidationConfig fixes vectors at exactly 384 dimensions -// (src/utils/paramValidation.ts) — match it so add() doesn't reject test data. -const DIM = 384 -const V = (): number[] => Array(DIM).fill(0.1) - -/** An operation that counts every `execute()` invocation — the re-drive detector. */ -function countingOp(opts: { delayMs?: number; name: string }): Operation & { calls: number } { - const op = { - name: opts.name, - calls: 0, - async execute(): Promise { - op.calls++ - if (opts.delayMs) await sleep(opts.delayMs) - return async () => {} - } - } - return op -} - -describe('transaction timeouts are never internally re-driven (8.10.1 no-hot-retry contract)', () => { - it('(i) the single-op engine (TransactionManager.executeTransaction / Transaction.execute — what add() drives via persistSingleOp) runs the overrun operation EXACTLY once and surfaces ONE retryable+hotRetryUnsafe error', async () => { - const manager = new TransactionManager() - const op0 = countingOp({ name: 'op0-overruns-budget', delayMs: 30 }) - const op1 = countingOp({ name: 'op1-must-never-start' }) - - let caught: unknown - try { - await manager.executeTransaction( - async (tx) => { - tx.addOperation(op0) - tx.addOperation(op1) - }, - // Tiny explicit override — the same override seam `transact()` - // exposes as `options.timeoutMs`; wins outright over the - // opCount*2000 floor that gates every real single-op write - // (transactTimeoutBudget()'s override semantics). - { timeout: 5 } - ) - } catch (err) { - caught = err - } - - expect(caught).toBeInstanceOf(TransactionTimeoutError) - const err = caught as TransactionTimeoutError - // The machine-readable contract callers branch on instead of parsing - // message text (src/transaction/errors.ts). - expect(err.retryable).toBe(true) - expect(err.hotRetryUnsafe).toBe(true) - - // The re-drive assertion: op0 (the one that overran) executed EXACTLY - // once — nothing inside TransactionManager/Transaction looped back and - // re-ran it — and op1 never started at all (the budget gate stopped it - // before it began, per Transaction.execute()'s per-operation loop). - expect(op0.calls).toBe(1) - expect(op1.calls).toBe(0) - }) - - it('(ii) add()\'s upsert-race retry loop (MAX_UPSERT_ATTEMPTS=10) exits on the FIRST TransactionTimeoutError — attempt counter stays at 1, never mistaken for the lost-insert-race signal it retries on', async () => { - const brain = new Brainy({ - requireSubtype: false, - storage: { type: 'memory' }, - silent: true - }) - await brain.init() - - let persistSingleOpCalls = 0 - const timeoutError = new TransactionTimeoutError(5, 1, { - elapsedMs: 6, - totalOperations: 2, - operationName: 'SaveNounMetadata' - }) - // Stub the private commit seam add() drives (persistSingleOp) to throw - // the exact error type the real engine surfaces on a mid-flight timeout. - // This test pins the upsert loop's EXCEPTION-HANDLING contract (does it - // retry a TransactionTimeoutError like it retries - // InsertPreconditionExistsSignal?), not the timing mechanics of a real - // timeout — those are pinned by test (i) and by - // tests/unit/transaction/timeout-rollback.test.ts. - ;(brain as any).persistSingleOp = async (): Promise => { - persistSingleOpCalls++ - throw timeoutError - } - - await expect( - brain.add({ data: 'a', type: NounType.Thing, vector: V() }) - ).rejects.toBe(timeoutError) - - // The loop's attempt counter: exactly one call, never retried up to - // MAX_UPSERT_ATTEMPTS. - expect(persistSingleOpCalls).toBe(1) - await brain.close() - }) -})