Compare commits

...

5 commits

Author SHA1 Message Date
d9cc7b9024 chore(release): 8.10.1 2026-07-24 16:09:47 -07:00
edf123a5e2 refactor: remove the orphaned transaction-result type left behind by the dead-path removal 2026-07-24 16:04:41 -07:00
5b2cbf74e5 fix: warm() metadata surface routes through the active provider (warm hook added to the metadata contract); add maintenanceDebt() observability surface
A production deployment's warm report showed metadata: 'unavailable' under a
native metadata provider. brain.warm()'s metadata leg only duck-typed the
built-in JS manager's hydrateAll() method, which a native provider has no
reason to implement.

- MetadataIndexProvider (src/plugin.ts) gains an optional warm?(): Promise<void>
  hook, mirroring the existing vector and graph provider hooks. brain.warm()
  now checks the active provider's own warm() FIRST, falls back to the JS
  manager's hydrateAll() when absent, and reports 'unavailable' only when
  neither exists -- never init() as a stand-in, since a native provider's
  init() may be a cheap verify rather than a real warm.
- Tests (tests/unit/brainy/warm.test.ts): a live provider instance shaped to
  have warm() reports 'warmed' and the hook called with no hydrateAll
  fallback; shaped to have neither hook reports 'unavailable' (pins the
  honest branch); the unmodified built-in JS manager still reports 'warmed'
  via hydrateAll(), unchanged.

Additive scope agreed mid-flight with the native-provider team: a
maintenance-debt observability seam so an operator sees a grind coming
instead of discovering it as a CPU storm.

- New optional maintenanceDebt?(): Promise<ProviderMaintenanceDebt> hook on
  all three provider contracts (vector, metadata, graph -- the same three
  warm?() lives on). ProviderMaintenanceDebt is fields-all-optional: a
  provider reports only what it truly measures (pendingBytes, pendingItems,
  lastPassCompletedAt, lastPassOutcome, converging), never an estimate
  dressed as fact.
- New public brain.maintenanceDebt(): a pure passthrough -- for each surface
  it calls only the active provider's own hook and reports the payload
  verbatim, or 'unavailable' when absent. No thresholds, no polling, no
  JS-side estimation; the provider owns the numbers, the operator owns the
  policy.
- ProviderMaintenanceDebt, MaintenanceDebtReport, and MaintenanceDebtOutcome
  are exported from the package root.
- Tests (tests/unit/brainy/maintenance-debt.test.ts): hook present reports
  'reported' with the exact payload passed through; hook absent reports
  'unavailable' on every surface; mixed surfaces resolve independently of
  each other.

RELEASES.md gains the 8.10.1 entry covering both fixes above and this
feature, including the no-hot-retry contract from the prior commit.
2026-07-24 16:02:01 -07:00
003e2a74ea fix: transaction timeouts are a typed no-hot-retry contract; engine-side non-retry pinned; dead transaction path removed
A production incident: a native-provider op ground 38-40s inside a transaction,
blew the apply-phase budget, rolled back, and a downstream pipeline hot-retried
the identical operation into a 6-minute CPU storm. Brainy itself never
auto-retried the timeout; the gap was that TransactionTimeoutError only said
"retryable" in prose, with nothing machine-readable for a caller to branch on.

- TransactionTimeoutError gains two typed, always-true fields: retryable
  (a later attempt may succeed once the slowness resolves or the budget is
  raised) and hotRetryUnsafe (an immediate identical retry re-pays the full
  cost that just timed out and can cascade into a CPU storm -- callers must
  latch and back off, never loop). context's existing telemetry fields
  (timeoutMs, operationIndex, elapsedMs, totalOperations, operationName) are
  now documented as the caller's backoff inputs.
- Updated the "retryable" doc-prose sites (transact()'s timeoutMs option,
  transactionBudgetFloorMs, Transaction.execute()'s contract) to point at
  the new fields instead of bare prose.
- Regression pin (tests/unit/transaction/timeout-never-internally-retried.test.ts):
  an execution counter proves the engine never re-drives a timed-out
  operation, through both the single-op engine TransactionManager/Transaction
  drives for every single-record write, and add()'s upsert-race retry loop
  (which must exit on the first TransactionTimeoutError, never treat it like
  the lost-insert-race signal it retries on).
- Removed TransactionManager.executeTransactionWithResult -- zero callers
  anywhere in the codebase.
2026-07-24 16:01:41 -07:00
22702b81c0 chore: the forge is the address — retire the archived mirror from every live surface
Ruled today: the project's one public home is source.soulcraft.com. The
old public repo is archived history and no longer part of any release.

- package.json repository/homepage/bugs now point at the forge (this is
  what the npm page links as Repository/Homepage/Issues)
- README CI badge reads the forge pipeline; CONTRIBUTING drops the
  mirror paragraph (forge account or email patch were already the ruled
  contribution paths)
- release.sh: mirror push + external release step removed; publishes go
  forge-first (box-held write token, temp userconfig so the token never
  hits argv; a forge-publish failure aborts before the storefront so the
  pair can never diverge), then npmjs with the scope-override pin (the
  fleet npmrc maps @soulcraft to the forge and scope mappings beat
  --registry); release page created via forge API when a token is
  present, loud skip otherwise; changelog compare links point home
- dead external CI workflow removed (.forgejo/workflows/ci.yml is the
  live pipeline)

Historical CHANGELOG links to the archive stay as written - history is
history and the archive serves them read-only.
2026-07-24 15:43:48 -07:00
21 changed files with 729 additions and 180 deletions

View file

@ -1,40 +0,0 @@
name: CI
on:
push:
pull_request:
jobs:
node:
name: Node ${{ matrix.node-version }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node-version: ['22', '24']
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: npm
- run: npm ci
- run: npm run test:unit
bun:
name: Bun (latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '22'
cache: npm
- uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- run: npm ci
# test:bun imports the built dist/, so build first.
- run: npm run build
# Bun as a runtime is the supported Bun story (`bun add` / `bun run`).
- run: npm run test:bun

View file

@ -2,6 +2,14 @@
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines. All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
### [8.10.1](https://source.soulcraft.com/soulcraft/brainy/compare/v8.10.0...v8.10.1) (2026-07-24)
- refactor: remove the orphaned transaction-result type left behind by the dead-path removal (edf123a5)
- fix: warm() metadata surface routes through the active provider (warm hook added to the metadata contract); add maintenanceDebt() observability surface (5b2cbf74)
- fix: transaction timeouts are a typed no-hot-retry contract; engine-side non-retry pinned; dead transaction path removed (003e2a74)
- chore: the forge is the address — retire the archived mirror from every live surface (22702b81)
### [8.10.0](https://github.com/soulcraftlabs/brainy/compare/v8.9.0...v8.10.0) (2026-07-23) ### [8.10.0](https://github.com/soulcraftlabs/brainy/compare/v8.9.0...v8.10.0) (2026-07-23)
- docs: adoption storefront — contributing guide, security policy, README support + cor section (9a99a7b) - docs: adoption storefront — contributing guide, security policy, README support + cor section (9a99a7b)

View file

@ -10,10 +10,6 @@ The source of truth is a self-hosted forge: **source.soulcraft.com/soulcraft/bra
It's anonymously readable and cloneable — no account needed to browse, clone, It's anonymously readable and cloneable — no account needed to browse, clone,
or build. or build.
**github.com/soulcraftlabs/brainy** is a public read-only mirror. It's a fine
place to read code or star the project, but issues and pull requests opened
there won't be picked up — please use one of the paths below instead.
## How to contribute ## How to contribute
**Found a bug, or have an idea?** Email **brainy@soulcraft.com**. No account, **Found a bug, or have an idea?** Email **brainy@soulcraft.com**. No account,

View file

@ -13,7 +13,7 @@
<p align="center"> <p align="center">
<a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/v/@soulcraft/brainy.svg" alt="npm version"></a> <a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/v/@soulcraft/brainy.svg" alt="npm version"></a>
<a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/dm/@soulcraft/brainy.svg" alt="npm downloads"></a> <a href="https://www.npmjs.com/package/@soulcraft/brainy"><img src="https://img.shields.io/npm/dm/@soulcraft/brainy.svg" alt="npm downloads"></a>
<a href="https://github.com/soulcraftlabs/brainy/actions/workflows/ci.yml"><img src="https://github.com/soulcraftlabs/brainy/actions/workflows/ci.yml/badge.svg" alt="CI"></a> <a href="https://source.soulcraft.com/soulcraft/brainy/actions"><img src="https://source.soulcraft.com/soulcraft/brainy/actions/workflows/ci.yml/badge.svg?branch=main" alt="CI"></a>
<a href="https://soulcraft.com/docs"><img src="https://img.shields.io/badge/docs-soulcraft.com-blue.svg" alt="Documentation"></a> <a href="https://soulcraft.com/docs"><img src="https://img.shields.io/badge/docs-soulcraft.com-blue.svg" alt="Documentation"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License"></a> <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License"></a>
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/%3C%2F%3E-TypeScript-%230074c1.svg" alt="TypeScript"></a> <a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/%3C%2F%3E-TypeScript-%230074c1.svg" alt="TypeScript"></a>

View file

@ -31,6 +31,68 @@ is sometimes cited as a 7.x removal — those methods never existed on 7.x; the
--- ---
## v8.10.1 — 2026-07-24 (the no-hot-retry contract + warm()'s metadata surface under native providers)
From a production incident: a native-provider op ground 38-40s inside a transaction,
blew the ~32s apply-phase budget, was rolled back (zero loss, by design), and a
downstream pipeline hot-retried the identical operation into a 6-minute, 100%-CPU
storm. Investigation confirmed Brainy itself never auto-retries a timed-out
transaction — the storm was entirely the consumer's own retry loop, driven by a
"retryable" doc-prose claim with no machine-readable contract to branch on. This
release closes that contract gap and, separately, fixes a real `warm()` reporting gap
surfaced by the same investigation.
- **`TransactionTimeoutError` is now a machine-readable no-hot-retry contract.** Two
new typed, always-`true` fields replace prose-only guidance:
- `retryable: true` — the operation MAY succeed on a later attempt, once the
underlying slowness resolves or the budget is deliberately raised
(`transactionBudgetFloorMs`, or a batch's own `timeoutMs` override).
- `hotRetryUnsafe: true` — an immediate, identical retry re-pays the FULL cost of
the work that just timed out (it does not resume partway) and can cascade into
exactly the CPU storm above. **Never loop on this error.** The documented pattern
is a latch, not a retry loop:
```
on TransactionTimeoutError:
record { at: Date.now(), error }
rethrow loudly to your own caller
hold a cooldown window before any re-attempt
clear the latch only on a subsequent success
```
- `context` (unchanged, now fully documented) carries the backoff inputs:
`timeoutMs`, `operationIndex`, `elapsedMs`, `totalOperations`, `operationName`.
- Every "retryable" doc-prose site referencing this error (`transact()`'s
`timeoutMs` option, `transactionBudgetFloorMs`, `Transaction.execute()`) now
points at these fields instead of bare prose.
- Regression-pinned: the engine never internally re-drives a timed-out operation
(verified via an execution counter through both the single-op write path and
`add()`'s upsert-race retry loop), so this has always been true — it is now
provable and typed.
- **Dead code removed**: `TransactionManager.executeTransactionWithResult()` had zero
callers in this codebase and is deleted.
- **`brain.warm()`'s metadata surface now routes through the ACTIVE provider.** A
production deployment's warm report showed `metadata: 'unavailable'` under a native
metadata provider — the previous logic only duck-typed the built-in JS manager's
`hydrateAll()` method, which a native provider has no reason to implement. The
metadata provider contract (`MetadataIndexProvider`, `src/plugin.ts`) gains an
optional `warm?(): Promise<void>` hook, mirroring the existing vector and graph
provider hooks. `brain.warm()` now checks the active provider's own `warm()` FIRST,
falls back to the JS manager's `hydrateAll()` when absent, and only reports
`'unavailable'` when neither exists — never `init()` as a stand-in, since a native
provider's `init()` may be a cheap verify rather than a real warm. A native
provider lights this surface up the same way `@soulcraft/cor` already lights the
vector and graph surfaces: implement `warm()` on its metadata provider.
- **New: `brain.maintenanceDebt()`** — the observability seam so an operator sees a
provider's outstanding background maintenance work (pending bytes/items, last pass
outcome, whether it's converging) BEFORE it grinds into the kind of budget-busting
op this release's timeout contract exists for, instead of discovering it as a CPU
storm. It is a pure passthrough: brainy applies no thresholds, no polling, and no
estimation — it calls each active provider's own optional `maintenanceDebt?()` hook
(vector, metadata, graph — the same three contracts `warm?()` lives on) and reports
the payload verbatim, or `'unavailable'` when a surface's provider doesn't track
debt. Useful as a pre-warm/post-warm check or a boot gate. `@soulcraft/cor` does not
yet implement the hook as of this release — expect it on cor's next release; until
then all three surfaces honestly report `'unavailable'`.
## Unreleased (the warm contract: cold-restart writes stop paying demand-load latency) ## Unreleased (the warm contract: cold-restart writes stop paying demand-load latency)
From a production deployment's cold-restart incident: the FIRST writes after every From a production deployment's cold-restart incident: the FIRST writes after every

4
package-lock.json generated
View file

@ -1,12 +1,12 @@
{ {
"name": "@soulcraft/brainy", "name": "@soulcraft/brainy",
"version": "8.10.0", "version": "8.10.1",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "@soulcraft/brainy", "name": "@soulcraft/brainy",
"version": "8.10.0", "version": "8.10.1",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@msgpack/msgpack": "^3.1.2", "@msgpack/msgpack": "^3.1.2",

View file

@ -1,6 +1,6 @@
{ {
"name": "@soulcraft/brainy", "name": "@soulcraft/brainy",
"version": "8.10.0", "version": "8.10.1",
"description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. Stage 3 CANONICAL: 42 nouns × 127 verbs covering 96-97% of all human knowledge.", "description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. Stage 3 CANONICAL: 42 nouns × 127 verbs covering 96-97% of all human knowledge.",
"main": "dist/index.js", "main": "dist/index.js",
"module": "dist/index.js", "module": "dist/index.js",
@ -128,13 +128,13 @@
"publishConfig": { "publishConfig": {
"access": "public" "access": "public"
}, },
"homepage": "https://github.com/soulcraftlabs/brainy", "homepage": "https://source.soulcraft.com/soulcraft/brainy",
"bugs": { "bugs": {
"url": "https://github.com/soulcraftlabs/brainy/issues" "url": "https://source.soulcraft.com/soulcraft/brainy/issues"
}, },
"repository": { "repository": {
"type": "git", "type": "git",
"url": "git+https://github.com/soulcraftlabs/brainy.git" "url": "git+https://source.soulcraft.com/soulcraft/brainy.git"
}, },
"files": [ "files": [
"dist/**/*.js", "dist/**/*.js",

View file

@ -142,7 +142,7 @@ else
fi fi
# Create new changelog entry # Create new changelog entry
CHANGELOG_ENTRY="### [${NEW_VERSION}](https://github.com/soulcraftlabs/brainy/compare/v${CURRENT_VERSION}...v${NEW_VERSION}) ($(date +%Y-%m-%d)) CHANGELOG_ENTRY="### [${NEW_VERSION}](https://source.soulcraft.com/soulcraft/brainy/compare/v${CURRENT_VERSION}...v${NEW_VERSION}) ($(date +%Y-%m-%d))
${COMMITS} ${COMMITS}
" "
@ -175,42 +175,59 @@ echo -e "${BLUE}7⃣ Creating git tag v${NEW_VERSION}...${NC}"
git tag -a "v${NEW_VERSION}" -m "Release v${NEW_VERSION}" git tag -a "v${NEW_VERSION}" -m "Release v${NEW_VERSION}"
echo -e "${GREEN}✅ Tag created${NC}\n" echo -e "${GREEN}✅ Tag created${NC}\n"
# Step 9: Push to origin (source of truth) and the public GitHub mirror # Step 9: Push to origin — the forge is the one home (ruled 2026-07-23; the
# old public GitHub repo is archived history, no longer part of any release).
echo -e "${BLUE}8⃣ Pushing to origin...${NC}" echo -e "${BLUE}8⃣ Pushing to origin...${NC}"
git push --follow-tags origin "$CURRENT_BRANCH" git push --follow-tags origin "$CURRENT_BRANCH"
echo -e "${GREEN}✅ Pushed to origin${NC}\n" echo -e "${GREEN}✅ Pushed to origin${NC}\n"
# The public GitHub repo is a mirror of origin with an unknown sync cadence. # Step 10: Publish — forge FIRST (home), npmjs second (the world's storefront).
# `gh release create` below targets GitHub directly: if the new tag hasn't # The fleet-wide ~/.npmrc maps the @soulcraft scope to the forge registry, and
# reached GitHub yet, gh would CREATE it — pointed at GitHub's default-branch # a scope mapping BEATS `--registry` on the command line — so each publish
# head, i.e. the wrong commit. Push branch+tag to GitHub explicitly, then # names its registry via the scope override explicitly. Nothing implicit.
# verify the tag resolves there to the same commit before any release is cut. FORGE_NPM_REG="https://source.soulcraft.com/api/packages/soulcraft/npm/"
GITHUB_URL="https://github.com/soulcraftlabs/brainy.git" FORGE_NPM_TOKEN_FILE="$HOME/.config/soulcraft/npm-publish-brainy.token"
echo -e "${BLUE}8⃣½ Pushing to the public GitHub mirror...${NC}" echo -e "${BLUE}9⃣ Publishing to the forge registry (home)...${NC}"
git push --follow-tags "$GITHUB_URL" "$CURRENT_BRANCH" if [ -f "$FORGE_NPM_TOKEN_FILE" ]; then
LOCAL_TAG_SHA="$(git rev-parse "v${NEW_VERSION}^{}")" TMPRC="$(mktemp)"
GITHUB_TAG_SHA="$(git ls-remote --tags "$GITHUB_URL" "v${NEW_VERSION}^{}" | cut -f1)" chmod 600 "$TMPRC"
if [ "$LOCAL_TAG_SHA" != "$GITHUB_TAG_SHA" ]; then {
echo -e "${RED}❌ Tag v${NEW_VERSION} on GitHub (${GITHUB_TAG_SHA:-absent}) does not match local (${LOCAL_TAG_SHA}) — aborting before npm publish. Fix the mirror, then re-run.${NC}" echo "@soulcraft:registry=${FORGE_NPM_REG}"
echo "//source.soulcraft.com/api/packages/soulcraft/npm/:_authToken=$(cat "$FORGE_NPM_TOKEN_FILE")"
} > "$TMPRC"
if npm publish --tag "$NPM_TAG" --userconfig "$TMPRC"; then
echo -e "${GREEN}✅ Published to the forge${NC}\n"
else
rm -f "$TMPRC"
echo -e "${RED}❌ Forge publish FAILED — aborting before npmjs so the pair never diverges. Fix and re-run.${NC}"
exit 1 exit 1
fi fi
echo -e "${GREEN}✅ GitHub mirror has the tag at the right commit${NC}\n" rm -f "$TMPRC"
# Step 10: Publish to npm
echo -e "${BLUE}9⃣ Publishing to npm (dist-tag: ${NPM_TAG})...${NC}"
npm publish --tag "$NPM_TAG"
# Brainy is the only PUBLIC @soulcraft package — verify visibility after every publish.
npm access get status @soulcraft/brainy || true
echo -e "${GREEN}✅ Published to npm${NC}\n"
# Step 11: Create GitHub release
echo -e "${BLUE}🔟 Creating GitHub release...${NC}"
if [ "$PRERELEASE" = true ]; then
gh release create "v${NEW_VERSION}" --generate-notes --prerelease
else else
gh release create "v${NEW_VERSION}" --generate-notes echo -e "${RED}❌ Forge publish token missing (${FORGE_NPM_TOKEN_FILE}) — aborting. The forge is home; publish it first or restage the token.${NC}"
exit 1
fi
echo -e "${BLUE}9⃣½ Publishing to npmjs (storefront, dist-tag: ${NPM_TAG})...${NC}"
npm publish --tag "$NPM_TAG" "--@soulcraft:registry=https://registry.npmjs.org/"
# Brainy is the only PUBLIC @soulcraft package — verify visibility after every publish.
npm access get status @soulcraft/brainy "--@soulcraft:registry=https://registry.npmjs.org/" || true
echo -e "${GREEN}✅ Published to npmjs${NC}\n"
# Step 11: Release object on the forge (presentational — the tag, CHANGELOG,
# and RELEASES.md are the record; this just gives the forge UI a release page).
echo -e "${BLUE}🔟 Creating forge release...${NC}"
if [ -n "${FORGEJO_RELEASE_TOKEN:-}" ]; then
if curl -sf -X POST "https://source.soulcraft.com/api/v1/repos/soulcraft/brainy/releases" \
-H "Authorization: token ${FORGEJO_RELEASE_TOKEN}" -H "Content-Type: application/json" \
-d "{\"tag_name\":\"v${NEW_VERSION}\",\"name\":\"v${NEW_VERSION}\",\"prerelease\":${PRERELEASE}}" >/dev/null; then
echo -e "${GREEN}✅ Forge release created${NC}\n"
else
echo -e "${RED}⚠️ Forge release API call failed — tag + CHANGELOG remain the record; create the release page via the forge UI if wanted${NC}\n"
fi
else
echo -e "${RED}⚠️ FORGEJO_RELEASE_TOKEN unset — no release page created; tag + CHANGELOG remain the record${NC}\n"
fi fi
echo -e "${GREEN}✅ GitHub release created${NC}\n"
# Step 12: Push public docs to the soulcraft.com docs ingest door # Step 12: Push public docs to the soulcraft.com docs ingest door
# (VENUE-DOCS-RELEASE-PUSH). Skips with a loud warning when # (VENUE-DOCS-RELEASE-PUSH). Skips with a loud warning when
@ -229,4 +246,4 @@ echo -e "${GREEN}🎉 Release ${NEW_VERSION} complete!${NC}"
echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
echo "" echo ""
echo -e "📦 npm: ${BLUE}https://www.npmjs.com/package/@soulcraft/brainy/v/${NEW_VERSION}${NC}" echo -e "📦 npm: ${BLUE}https://www.npmjs.com/package/@soulcraft/brainy/v/${NEW_VERSION}${NC}"
echo -e "🐙 GitHub: ${BLUE}https://github.com/soulcraftlabs/brainy/releases/tag/v${NEW_VERSION}${NC}" echo -e "🏠 Forge: ${BLUE}https://source.soulcraft.com/soulcraft/brainy/releases/tag/v${NEW_VERSION}${NC}"

View file

@ -65,7 +65,8 @@ import type {
OpaqueIdSet, OpaqueIdSet,
AtGenerationVectors, AtGenerationVectors,
VectorIndexProvider, VectorIndexProvider,
GraphIndexProvider GraphIndexProvider,
ProviderMaintenanceDebt
} from './plugin.js' } from './plugin.js'
import type { import type {
BrainyPlugin, BrainyPlugin,
@ -424,6 +425,30 @@ export interface WarmReport {
totalDurationMs: number totalDurationMs: number
} }
/**
* @description Result of {@link Brainy.maintenanceDebt}: one outcome per
* index surface, mirroring {@link WarmReport}'s shape.
* - `'reported'` the active provider for this surface implements
* `maintenanceDebt?()` and its {@link ProviderMaintenanceDebt} payload is
* attached verbatim under `debt`.
* - `'unavailable'` the active provider does not implement the hook, so
* nothing is known; brainy never estimates or infers a payload on its
* behalf.
*/
export type MaintenanceDebtOutcome = 'reported' | 'unavailable'
/**
* @description Per-surface result of {@link Brainy.maintenanceDebt}. Brainy
* performs no thresholding, polling, or estimation over this data it is a
* pure passthrough of each active provider's own self-report (the provider
* owns the numbers; the operator owns the policy).
*/
export interface MaintenanceDebtReport {
vector: { outcome: MaintenanceDebtOutcome; debt?: ProviderMaintenanceDebt }
metadata: { outcome: MaintenanceDebtOutcome; debt?: ProviderMaintenanceDebt }
graph: { outcome: MaintenanceDebtOutcome; debt?: ProviderMaintenanceDebt }
}
/** /**
* How long a failed aggregation-backfill walk suppresses fresh walk attempts. * How long a failed aggregation-backfill walk suppresses fresh walk attempts.
* Within the window, queries rethrow the recorded failure instantly (loud, * Within the window, queries rethrow the recorded failure instantly (loud,
@ -14313,9 +14338,16 @@ export class Brainy<T = any> implements BrainyInterface<T> {
* *some* backing storage as a side effect but is reported honestly as * *some* backing storage as a side effect but is reported honestly as
* `'probed'`, never `'warmed'`. An empty index or unknown dimension has * `'probed'`, never `'warmed'`. An empty index or unknown dimension has
* nothing to probe (`'unavailable'`). * nothing to probe (`'unavailable'`).
* - **Metadata**: full hydration every persisted field's sparse index is * - **Metadata**: calls the provider's own `warm?()` when the active
* loaded from storage (`MetadataIndexManager.hydrateAll()`), not just the * `'metadataIndex'` provider implements it (`'warmed'`) the seam a
* heuristic common-fields subset `init()` warms. * native metadata provider lights up so it is not duck-typed against the
* JS manager's method. Otherwise falls back to full hydration on the
* built-in JS manager every persisted field's sparse index is loaded
* from storage (`MetadataIndexManager.hydrateAll()`), not just the
* heuristic common-fields subset `init()` warms and reports `'warmed'`.
* Neither seam present `'unavailable'` (honest: `init()` is never used
* as a substitute here, since a native provider's `init()` may be a
* cheap verify rather than a real warm).
* - **Graph**: calls the provider's own `warm?()` when the active graph * - **Graph**: calls the provider's own `warm?()` when the active graph
* provider implements it; otherwise re-runs its existing eager cold-load * provider implements it; otherwise re-runs its existing eager cold-load
* `init()` seam (idempotent the JS adjacency index's `init()` already * `init()` seam (idempotent the JS adjacency index's `init()` already
@ -14371,12 +14403,23 @@ export class Brainy<T = any> implements BrainyInterface<T> {
// --- Metadata -------------------------------------------------------- // --- Metadata --------------------------------------------------------
const metadataStart = Date.now() const metadataStart = Date.now()
let metadataOutcome: WarmOutcome let metadataOutcome: WarmOutcome
const metadataProvider = this.metadataIndex as unknown as MetadataIndexProvider
const metadataWithHydrate = this.metadataIndex as unknown as { hydrateAll?: () => Promise<void> } const metadataWithHydrate = this.metadataIndex as unknown as { hydrateAll?: () => Promise<void> }
if (typeof metadataWithHydrate.hydrateAll === 'function') { if (typeof metadataProvider.warm === 'function') {
// Active provider (e.g. a native metadata index) declares its own warm
// seam — route through it FIRST so a native provider's warmth is
// reported honestly instead of being duck-typed against the JS
// manager's hydrateAll(), which a native provider does not implement.
await metadataProvider.warm()
metadataOutcome = 'warmed'
} else if (typeof metadataWithHydrate.hydrateAll === 'function') {
// Built-in JS manager path — full sparse-index hydration.
await metadataWithHydrate.hydrateAll() await metadataWithHydrate.hydrateAll()
metadataOutcome = 'warmed' metadataOutcome = 'warmed'
} else { } else {
// No hydration seam on this metadata provider — nothing to run. // No hydration seam on this metadata provider — nothing to run. (No
// init() fallback here: init() on a native provider may be a cheap
// verify, and reporting that as warmth would lie.)
metadataOutcome = 'unavailable' metadataOutcome = 'unavailable'
} }
const metadataDurationMs = Date.now() - metadataStart const metadataDurationMs = Date.now() - metadataStart
@ -14410,6 +14453,63 @@ export class Brainy<T = any> implements BrainyInterface<T> {
} }
} }
/**
* Read each index surface's self-reported outstanding maintenance work
* the observability seam so an operator sees a grind coming (rising
* pending bytes/items, a stalled background pass) instead of discovering
* it as a CPU storm or a transaction blowing its budget mid-flight (see
* {@link TransactionTimeoutError}).
*
* PURE PASSTHROUGH: for each of vector/metadata/graph, this calls ONLY the
* ACTIVE provider's own `maintenanceDebt?()` hook (the same per-surface
* provider resolution {@link Brainy.warm} uses) and reports its
* {@link ProviderMaintenanceDebt} payload verbatim. There is no JS-side
* fallback computation, no threshold evaluation, and no polling brainy
* surfaces the truth the provider measured; the provider owns the numbers
* and the operator owns the policy (what threshold matters, what action to
* take). A surface whose active provider does not implement the hook
* reports `'unavailable'` never a guessed or zeroed payload.
*
* @returns A {@link MaintenanceDebtReport}: per-surface outcome + payload.
* @example
* ```typescript
* const debt = await brain.maintenanceDebt()
* if (debt.metadata.outcome === 'reported' && debt.metadata.debt?.pendingBytes) {
* console.log('metadata pending bytes:', debt.metadata.debt.pendingBytes)
* }
* ```
*/
async maintenanceDebt(): Promise<MaintenanceDebtReport> {
await this.ensureInitialized({ needs: ['vector', 'metadata', 'graph'] })
// --- Vector ---------------------------------------------------------
const vectorProvider = this.index as VectorIndexProvider & {
maintenanceDebt?: () => Promise<ProviderMaintenanceDebt>
}
const vector =
typeof vectorProvider.maintenanceDebt === 'function'
? { outcome: 'reported' as const, debt: await vectorProvider.maintenanceDebt() }
: { outcome: 'unavailable' as const }
// --- Metadata --------------------------------------------------------
const metadataProvider = this.metadataIndex as unknown as MetadataIndexProvider
const metadata =
typeof metadataProvider.maintenanceDebt === 'function'
? { outcome: 'reported' as const, debt: await metadataProvider.maintenanceDebt() }
: { outcome: 'unavailable' as const }
// --- Graph -------------------------------------------------------------
const graphProvider = this.graphIndex as GraphIndexProvider & {
maintenanceDebt?: () => Promise<ProviderMaintenanceDebt>
}
const graph =
typeof graphProvider.maintenanceDebt === 'function'
? { outcome: 'reported' as const, debt: await graphProvider.maintenanceDebt() }
: { outcome: 'unavailable' as const }
return { vector, metadata, graph }
}
/** /**
* Explicitly warm up the embedding engine * Explicitly warm up the embedding engine
* *

View file

@ -121,9 +121,13 @@ export interface TransactOptions {
* with the batch: `max(30 000, opCount × 2 000)` production imports on * with the batch: `max(30 000, opCount × 2 000)` production imports on
* network-attached disks measure ~2 s per operation, so a flat 30 s budget * network-attached disks measure ~2 s per operation, so a flat 30 s budget
* silently capped honest bulk work at ~15 operations. A tripped budget * silently capped honest bulk work at ~15 operations. A tripped budget
* rolls the whole batch back and throws a retryable * rolls the whole batch back and throws a `TransactionTimeoutError` naming
* `TransactionTimeoutError` naming the operation it stopped at, the batch * the operation it stopped at, the batch size, and the elapsed/budget
* size, and the elapsed/budget times. * times. That error is retryable-with-latch, never hot-retry: its
* `retryable` field says a later attempt may succeed, its
* `hotRetryUnsafe` field says an immediate identical retry re-pays the
* full cost that just timed out callers must latch and back off, never
* loop.
*/ */
timeoutMs?: number timeoutMs?: number
} }

View file

@ -31,6 +31,11 @@ export type { DiagnosticsResult } from './brainy.js'
// brain.warm() — eager index/storage readiness report (per-surface honest // brain.warm() — eager index/storage readiness report (per-surface honest
// outcome + timing). See the WarmReport JSDoc in brainy.ts. // outcome + timing). See the WarmReport JSDoc in brainy.ts.
export type { WarmReport, WarmOutcome } from './brainy.js' export type { WarmReport, WarmOutcome } from './brainy.js'
// brain.maintenanceDebt() — per-surface passthrough of each active
// provider's self-reported background maintenance debt. See the
// MaintenanceDebtReport JSDoc in brainy.ts and ProviderMaintenanceDebt in
// plugin.ts for the measure-only-what-you-track contract.
export type { MaintenanceDebtReport, MaintenanceDebtOutcome } from './brainy.js'
export type { export type {
GraphAuditReport, GraphAuditReport,
GraphAuditDiscrepancy GraphAuditDiscrepancy
@ -227,6 +232,11 @@ export type { FamilyStamp, StampMembers, StampVerdict } from './db/familyStamp.j
export { isVersionedIndexProvider } from './plugin.js' export { isVersionedIndexProvider } from './plugin.js'
export type { VersionedIndexProvider } from './plugin.js' export type { VersionedIndexProvider } from './plugin.js'
export type { ProviderInvariantReport, InvariantResult, InvariantHeal } from './plugin.js' export type { ProviderInvariantReport, InvariantResult, InvariantHeal } from './plugin.js'
// Optional provider self-report of outstanding background maintenance work
// (compaction, deferred writes, etc.) — the payload type for
// brain.maintenanceDebt(). See the measure-only-what-you-track contract on
// ProviderMaintenanceDebt in plugin.ts.
export type { ProviderMaintenanceDebt } from './plugin.js'
// Optional native graph-acceleration engine (cor 3.0) — the published provider // Optional native graph-acceleration engine (cor 3.0) — the published provider
// contract + its columnar wire types. Brainy feature-detects an implementation // contract + its columnar wire types. Brainy feature-detects an implementation
// and falls back to its pure-TS adjacency when absent. // and falls back to its pure-TS adjacency when absent.

View file

@ -171,6 +171,37 @@ export interface ProviderInvariantReport {
durationMs: number durationMs: number
} }
/**
* @description A provider's self-report of its own outstanding background
* maintenance work (compaction, deferred writes, a build-newverifyswap in
* flight, etc.) the observability seam so an operator sees a grind coming
* (rising pending bytes/items, a stalled pass) instead of discovering it as a
* CPU storm or a timeout under transaction budget pressure. Every field is
* OPTIONAL and every field is a MEASUREMENT: a provider reports ONLY what it
* actually tracks, never an estimate dressed up as a fact. Absence of the
* {@link VectorIndexProvider.maintenanceDebt} /
* {@link GraphIndexProvider.maintenanceDebt} /
* {@link MetadataIndexProvider.maintenanceDebt} hook itself means the
* provider does not track debt at all brainy reports that surface
* `'unavailable'` rather than inventing zeros. Brainy performs NO threshold
* checks, NO polling, and NO JS-side estimation over this payload it is a
* pure passthrough via {@link Brainy.maintenanceDebt}; the provider owns the
* numbers and the operator owns the policy (what threshold matters, what to
* do about it).
*/
export interface ProviderMaintenanceDebt {
/** Bytes of outstanding/unmerged work, if the provider measures it (e.g. unflushed writes, unmerged segments). */
pendingBytes?: number
/** Count of outstanding items (records, segments, nodes) awaiting the provider's background pass. */
pendingItems?: number
/** Epoch millis when the provider's last maintenance pass finished, if it tracks one. */
lastPassCompletedAt?: number
/** How the last pass ended, if the provider tracks pass outcomes. */
lastPassOutcome?: 'completed' | 'partial' | 'failed'
/** `true` if the provider's own measurements show debt trending down (making progress); `false` if flat or growing; omitted if the provider can't tell. */
converging?: boolean
}
/** /**
* The `'metadataIndex'` provider a drop-in for `MetadataIndexManager`. * The `'metadataIndex'` provider a drop-in for `MetadataIndexManager`.
* Brainy calls this surface via `this.metadataIndex.*` (see `brainy.ts`) and * Brainy calls this surface via `this.metadataIndex.*` (see `brainy.ts`) and
@ -181,6 +212,34 @@ export interface MetadataIndexProvider {
flush(): Promise<void> flush(): Promise<void>
rebuild(): Promise<void> rebuild(): Promise<void>
/**
* @description OPTIONAL. Eagerly load/fault-in backing storage (e.g. mmap
* pretouch, full sparse-index hydration) so first queries run at
* steady-state cost. Optional; absence means the provider demand-loads.
* Mirrors {@link GraphIndexProvider.warm} / the vector provider's `warm?()`
* (`src/plugin.ts` VectorIndexProvider). Distinct from `init()`: `init` is
* required and runs once automatically during brain startup; `warm` is a
* separate, explicit step a caller opts into via `brain.warm()` (or
* `warmOnOpen`) to pre-pay demand-load cost `init` left lazy. Idempotent
* calling it more than once must be safe and cheap on a brain that is
* already warm. A provider that already loads everything eagerly in
* `init()` may implement `warm` as a no-op or omit it `brain.warm()`
* falls back to the built-in JS manager's `hydrateAll()` duck-type when
* absent, and to an honest `'unavailable'` when neither exists.
*/
warm?(): Promise<void>
/**
* @description OPTIONAL self-reported {@link ProviderMaintenanceDebt}
* the observability seam so an operator sees outstanding background
* maintenance work (e.g. unmerged postings) BEFORE it grinds a transaction
* into a budget-busting op. Absence means this provider does not track
* debt; `brain.maintenanceDebt()` reports this surface `'unavailable'`
* rather than guessing. See {@link ProviderMaintenanceDebt} for the
* measure-only-what-you-track contract.
*/
maintenanceDebt?(): Promise<ProviderMaintenanceDebt>
/** /**
* @description OPTIONAL honest durability signal (readiness contract, * @description OPTIONAL honest durability signal (readiness contract,
* mirrors `isReady?()` on the graph and vector providers). `true` the * mirrors `isReady?()` on the graph and vector providers). `true` the
@ -395,6 +454,18 @@ export interface GraphIndexProvider {
*/ */
warm?(): Promise<void> warm?(): Promise<void>
/**
* @description OPTIONAL self-reported {@link ProviderMaintenanceDebt}
* the observability seam so an operator sees outstanding background
* maintenance work (e.g. a build-newverifyswap in flight, unmerged
* adjacency segments) BEFORE it grinds a transaction into a
* budget-busting op. Absence means this provider does not track debt;
* `brain.maintenanceDebt()` reports this surface `'unavailable'` rather
* than guessing. See {@link ProviderMaintenanceDebt} for the
* measure-only-what-you-track contract.
*/
maintenanceDebt?(): Promise<ProviderMaintenanceDebt>
/** /**
* @description OPTIONAL. A native provider returns true from the moment its * @description OPTIONAL. A native provider returns true from the moment its
* `init()` detects a large epoch-drift until its background * `init()` detects a large epoch-drift until its background
@ -1057,6 +1128,18 @@ export interface VectorIndexProvider {
*/ */
warm?(): Promise<void> warm?(): Promise<void>
/**
* @description OPTIONAL self-reported {@link ProviderMaintenanceDebt}
* the observability seam so an operator sees outstanding background
* maintenance work (e.g. unflushed writes, a pending rebuild) BEFORE it
* grinds a transaction into a budget-busting op. Absence means this
* provider does not track debt; `brain.maintenanceDebt()` reports this
* surface `'unavailable'` rather than guessing. See
* {@link ProviderMaintenanceDebt} for the measure-only-what-you-track
* contract.
*/
maintenanceDebt?(): Promise<ProviderMaintenanceDebt>
/** /**
* @description OPTIONAL honest durability signal (readiness contract, * @description OPTIONAL honest durability signal (readiness contract,
* mirrors {@link GraphIndexProvider.isReady}). `true` the persisted * mirrors {@link GraphIndexProvider.isReady}). `true` the persisted

View file

@ -62,9 +62,10 @@ const DEFAULT_BUDGET_FLOOR_MS = 30_000
* NEXT operation may start (see {@link Transaction.execute}), never whether * NEXT operation may start (see {@link Transaction.execute}), never whether
* already-completed work is rolled back after the fact. A trip mid-batch * already-completed work is rolled back after the fact. A trip mid-batch
* still rolls back every operation applied so far, atomically, and throws a * still rolls back every operation applied so far, atomically, and throws a
* retryable, fully-labeled TransactionTimeoutError that zero-loss guarantee * fully-labeled `TransactionTimeoutError` retryable-with-latch, never
* doesn't change; only the point at which the clock stops mattering does (at * hot-retry (see its `retryable` and `hotRetryUnsafe` fields) that
* the last operation, not one check later). * zero-loss guarantee doesn't change; only the point at which the clock
* stops mattering does (at the last operation, not one check later).
* *
* @param opCount - Number of operations in the batch. * @param opCount - Number of operations in the batch.
* @param override - A full override for this call; wins over everything else. * @param override - A full override for this call; wins over everything else.

View file

@ -19,7 +19,6 @@
import { Transaction } from './Transaction.js' import { Transaction } from './Transaction.js'
import { import {
TransactionFunction, TransactionFunction,
TransactionResult,
TransactionOptions TransactionOptions
} from './types.js' } from './types.js'
import { TransactionError } from './errors.js' import { TransactionError } from './errors.js'
@ -105,34 +104,6 @@ export class TransactionManager {
} }
} }
/**
* Execute a transaction and return detailed result
*/
async executeTransactionWithResult<T>(
fn: TransactionFunction<T>,
options?: TransactionOptions
): Promise<TransactionResult<T>> {
const startTime = Date.now()
const transaction = new Transaction(options)
try {
const value = await fn(transaction)
await transaction.execute()
const executionTimeMs = Date.now() - startTime
return {
value,
operationCount: transaction.getOperationCount(),
executionTimeMs
}
} catch (error) {
// Transaction failed
throw error
}
}
/** /**
* Get transaction statistics * Get transaction statistics
*/ */

View file

@ -73,14 +73,47 @@ export class InvalidTransactionStateError extends TransactionError {
/** /**
* Error for transaction timeout * Error for transaction timeout
*
* Machine-readable no-hot-retry contract: {@link retryable} and
* {@link hotRetryUnsafe} are both always `true` on this class they exist
* so a caller can branch on the *shape* of the error instead of parsing
* message text. Read them together: the operation may eventually succeed,
* but never by looping on it immediately.
*
* `context` (inherited from {@link TransactionError}) carries the caller's
* backoff inputs see the field docs below.
*/ */
export class TransactionTimeoutError extends TransactionError { export class TransactionTimeoutError extends TransactionError {
/**
* The failed operation MAY succeed on a later attempt once the
* underlying slowness resolves (e.g. a cold page cache warms up) or the
* budget is deliberately raised (`transactionBudgetFloorMs`, or a larger
* `timeoutMs` override on the batch). This is a statement about eventual
* retryability, not a license to retry now see {@link hotRetryUnsafe}.
*/
public readonly retryable = true
/**
* An immediate, identical retry re-pays the FULL cost of the work that
* just timed out it does not resume partway. Looping on this error
* (hot-retrying) repeats that full cost every attempt and can cascade
* into a CPU/resource storm on the caller's side. Callers MUST latch: on
* this error, record `{ at: Date.now(), error }`, surface one loud
* failure to their own caller, and hold a cooldown window before any
* re-attempt (clearing the latch only on success). Never retry this error
* in a tight loop.
*/
public readonly hotRetryUnsafe = true
constructor( constructor(
timeoutMs: number, timeoutMs: number,
operationIndex: number, operationIndex: number,
telemetry?: { telemetry?: {
/** Milliseconds elapsed in the transaction when the budget tripped. */
elapsedMs?: number elapsedMs?: number
/** Total number of operations in the batch that timed out. */
totalOperations?: number totalOperations?: number
/** Name of the operation the batch was about to start when it tripped, if named. */
operationName?: string operationName?: string
} }
) { ) {
@ -93,8 +126,16 @@ export class TransactionTimeoutError extends TransactionError {
telemetry?.elapsedMs !== undefined ? `${telemetry.elapsedMs}ms elapsed, ` : '' telemetry?.elapsedMs !== undefined ? `${telemetry.elapsedMs}ms elapsed, ` : ''
super( super(
`Transaction timed out at operation ${progress}${name}${elapsed}budget ${timeoutMs}ms. ` + `Transaction timed out at operation ${progress}${name}${elapsed}budget ${timeoutMs}ms. ` +
`The batch rolled back atomically; retry with a higher timeoutMs or a smaller batch.`, `The batch rolled back atomically; retryable after the underlying slowness resolves or ` +
{ timeoutMs, operationIndex, ...telemetry } `the budget is raised, but hot-retry-unsafe — latch and back off, never loop.`,
{
// Caller backoff inputs — all present on every instance:
/** Configured budget (ms) that was exceeded. */
timeoutMs,
/** Index of the operation the batch was about to start when it tripped. */
operationIndex,
...telemetry
}
) )
this.name = 'TransactionTimeoutError' this.name = 'TransactionTimeoutError'
} }

View file

@ -66,26 +66,6 @@ export interface TransactionContext {
*/ */
export type TransactionFunction<T> = (ctx: TransactionContext) => Promise<T> export type TransactionFunction<T> = (ctx: TransactionContext) => Promise<T>
/**
* Transaction execution result
*/
export interface TransactionResult<T> {
/**
* Result value from user function
*/
value: T
/**
* Number of operations executed
*/
operationCount: number
/**
* Execution time in milliseconds
*/
executionTimeMs: number
}
/** /**
* Transaction execution options * Transaction execution options
*/ */

View file

@ -1658,8 +1658,10 @@ export interface BrainyConfig {
* **start** never whether already-completed work gets rolled back after * **start** never whether already-completed work gets rolled back after
* the fact (a single-op write can never time out post-hoc: it either runs * the fact (a single-op write can never time out post-hoc: it either runs
* or it commits). A trip mid-batch still rolls back every applied operation * or it commits). A trip mid-batch still rolls back every applied operation
* atomically and throws a retryable `TransactionTimeoutError`; only the * atomically and throws a `TransactionTimeoutError` that is
* floor of the formula is configurable here. * retryable-with-latch, never hot-retry (see its `retryable` and
* `hotRetryUnsafe` fields); only the floor of the formula is configurable
* here.
* *
* Raise this when a cold store's first writes after a restart legitimately * Raise this when a cold store's first writes after a restart legitimately
* take longer than 30s per operation (e.g. page-cache-cold canonical writes * take longer than 30s per operation (e.g. page-cache-cold canonical writes

View file

@ -5,7 +5,6 @@
* - High-level transaction API * - High-level transaction API
* - Statistics tracking * - Statistics tracking
* - Error handling * - Error handling
* - Result wrapping
*/ */
import { describe, it, expect, beforeEach } from 'vitest' import { describe, it, expect, beforeEach } from 'vitest'
@ -84,42 +83,6 @@ describe('TransactionManager', () => {
}) })
}) })
describe('executeTransactionWithResult', () => {
it('should return detailed result', async () => {
const result = await manager.executeTransactionWithResult(async (tx) => {
tx.addOperation({
execute: async () => {
await new Promise(resolve => setTimeout(resolve, 1))
return async () => {}
}
})
tx.addOperation({ execute: async () => undefined })
return 'success'
})
expect(result.value).toBe('success')
expect(result.operationCount).toBe(2)
expect(result.executionTimeMs).toBeGreaterThanOrEqual(0)
})
it('should measure execution time', async () => {
const result = await manager.executeTransactionWithResult(async (tx) => {
tx.addOperation({
execute: async () => {
await new Promise(resolve => setTimeout(resolve, 25))
return async () => {}
}
})
return 'done'
})
// Timer coalescing can fire a setTimeout up to a few ms EARLY under
// load, so assert well below the sleep — this tests that time is
// MEASURED, not the OS timer's precision.
expect(result.executionTimeMs).toBeGreaterThanOrEqual(20)
})
})
describe('Statistics Tracking', () => { describe('Statistics Tracking', () => {
it('should track total transactions', async () => { it('should track total transactions', async () => {
await manager.executeTransaction(async (tx) => { await manager.executeTransaction(async (tx) => {

View file

@ -0,0 +1,122 @@
/**
* @module tests/unit/brainy/maintenance-debt
* @description Coverage for `brain.maintenanceDebt()` (8.10.1) the
* observability seam so an operator sees a provider's outstanding background
* maintenance work (compaction, deferred writes, a build-newverifyswap in
* flight, ...) BEFORE it grinds a transaction into a budget-busting op, the
* same failure class documented on `TransactionTimeoutError`
* (src/transaction/errors.ts). Sibling to tests/unit/brainy/warm.test.ts,
* which establishes this file's technique: shape the probe points brain.ts
* reads (`typeof provider.maintenanceDebt === 'function'`) directly on the
* REAL, live provider instances rather than hand-rolling full fakes for the
* larger `MetadataIndexProvider` / `GraphIndexProvider` interfaces.
*
* `brain.maintenanceDebt()` is a PURE PASSTHROUGH: no thresholds, no
* polling, no JS-side estimation these tests pin exactly that by asserting
* the returned payload is the provider's object, verbatim.
*/
import { describe, it, expect } from 'vitest'
import { Brainy } from '../../../src/brainy.js'
import { NounType } from '../../../src/types/graphTypes.js'
import type { ProviderMaintenanceDebt } from '../../../src/plugin.js'
// Brainy's ValidationConfig fixes vectors at exactly 384 dimensions
// (src/utils/paramValidation.ts) — match it so add() doesn't reject test data.
const DIM = 384
const V = (seed = 1): number[] => Array.from({ length: DIM }, (_, i) => Math.sin(seed + i))
async function freshBrain(): Promise<Brainy<any>> {
const brain = new Brainy({
requireSubtype: false,
storage: { type: 'memory' },
silent: true
})
await brain.init()
await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
return brain
}
describe('brain.maintenanceDebt()', () => {
it('reports "unavailable" for every surface when no active provider implements maintenanceDebt() (the built-in JS stack today)', async () => {
const brain = await freshBrain()
const report = await brain.maintenanceDebt()
expect(report.vector.outcome).toBe('unavailable')
expect(report.vector.debt).toBeUndefined()
expect(report.metadata.outcome).toBe('unavailable')
expect(report.metadata.debt).toBeUndefined()
expect(report.graph.outcome).toBe('unavailable')
expect(report.graph.debt).toBeUndefined()
await brain.close()
})
it('reports "reported" + the exact payload when the active provider implements maintenanceDebt() (verbatim passthrough, no thresholding)', async () => {
const brain = await freshBrain()
const vectorDebt: ProviderMaintenanceDebt = {
pendingBytes: 4_096,
pendingItems: 12,
lastPassCompletedAt: 1_700_000_000_000,
lastPassOutcome: 'completed',
converging: true
}
;(brain as any).index.maintenanceDebt = async () => vectorDebt
const report = await brain.maintenanceDebt()
expect(report.vector.outcome).toBe('reported')
// Verbatim passthrough — the exact object, not a re-derived copy.
expect(report.vector.debt).toBe(vectorDebt)
// Untouched surfaces stay honestly 'unavailable'.
expect(report.metadata.outcome).toBe('unavailable')
expect(report.graph.outcome).toBe('unavailable')
await brain.close()
})
it('mixed surfaces: each surface\'s outcome depends ONLY on its OWN active provider — one surface reporting never leaks into another', async () => {
const brain = await freshBrain()
const metadataDebt: ProviderMaintenanceDebt = {
pendingItems: 3,
lastPassOutcome: 'partial',
converging: false
}
const graphDebt: ProviderMaintenanceDebt = {
pendingBytes: 0,
converging: true
}
;(brain as any).metadataIndex.maintenanceDebt = async () => metadataDebt
;(brain as any).graphIndex.maintenanceDebt = async () => graphDebt
// Vector is deliberately left unpatched.
const report = await brain.maintenanceDebt()
expect(report.vector.outcome).toBe('unavailable')
expect(report.vector.debt).toBeUndefined()
expect(report.metadata.outcome).toBe('reported')
expect(report.metadata.debt).toBe(metadataDebt)
expect(report.graph.outcome).toBe('reported')
expect(report.graph.debt).toBe(graphDebt)
await brain.close()
})
it('an empty ProviderMaintenanceDebt object (every field omitted) is still honestly "reported" — presence of the hook, not the payload\'s richness, drives the outcome', async () => {
const brain = await freshBrain()
const emptyDebt: ProviderMaintenanceDebt = {}
;(brain as any).graphIndex.maintenanceDebt = async () => emptyDebt
const report = await brain.maintenanceDebt()
expect(report.graph.outcome).toBe('reported')
expect(report.graph.debt).toEqual({})
await brain.close()
})
})

View file

@ -307,4 +307,92 @@ describe('brain.warm()', () => {
expect(report.totalDurationMs).toBeGreaterThanOrEqual(0) expect(report.totalDurationMs).toBeGreaterThanOrEqual(0)
await brain.close() await brain.close()
}) })
// --- Metadata leg routes through the ACTIVE provider (8.10.1) -----------
//
// `MetadataIndexProvider` is a ~50-method interface (src/plugin.ts) — far
// too large to hand-write a compliant fake class the way `FakeVectorProvider`
// fakes the ~8-method `VectorIndexProvider` above. Test (c) already
// establishes this file's pattern for the metadata leg: exercise the REAL
// `MetadataIndexManager` instance and shape just the probe points brain.ts
// reads (`typeof provider.warm === 'function'` /
// `typeof provider.hydrateAll === 'function'`) directly on that instance.
// Shadowing an own property on the live object stands in for "a different
// provider implementation" without needing a hand-rolled full fake — the
// rest of the real manager (used by add()/init() above) is untouched.
describe('metadata leg — warm() routes through the active provider', () => {
it('(f) calls the ACTIVE metadata provider\'s warm() when present and reports "warmed", never falling back to hydrateAll', async () => {
const brain = new Brainy({
requireSubtype: false,
storage: { type: 'memory' },
silent: true
})
await brain.init()
await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
const metadataIndex = (brain as any).metadataIndex
let warmCalls = 0
let hydrateAllCalls = 0
const origHydrateAll = metadataIndex.hydrateAll.bind(metadataIndex)
metadataIndex.hydrateAll = async (...args: unknown[]) => {
hydrateAllCalls++
return origHydrateAll(...args)
}
// Simulates a native metadata provider declaring the optional `warm()`
// hook added to `MetadataIndexProvider` (src/plugin.ts) in 8.10.1.
metadataIndex.warm = async () => {
warmCalls++
}
const report = await brain.warm()
expect(warmCalls).toBe(1)
expect(hydrateAllCalls).toBe(0) // warm() ran — no hydrateAll fallback
expect(report.metadata.outcome).toBe('warmed')
await brain.close()
})
it('reports "unavailable" when the active metadata provider implements neither warm() nor hydrateAll() (the honest branch a native provider without either hook must hit)', async () => {
const brain = new Brainy({
requireSubtype: false,
storage: { type: 'memory' },
silent: true
})
await brain.init()
await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
const metadataIndex = (brain as any).metadataIndex
// Shadow away BOTH optional hooks — models a genuinely native provider
// that (unlike the built-in JS manager) offers neither seam. This must
// never fall back to calling init() as a stand-in for warmth.
metadataIndex.warm = undefined
metadataIndex.hydrateAll = undefined
const report = await brain.warm()
expect(report.metadata.outcome).toBe('unavailable')
await brain.close()
})
it('the built-in JS manager (no warm()) still reports "warmed" via its existing hydrateAll() duck-type — unchanged by the new provider hook', async () => {
const brain = new Brainy({
requireSubtype: false,
storage: { type: 'memory' },
silent: true
})
await brain.init()
await brain.add({ data: 'a', type: NounType.Thing, vector: V(1) })
// No patching at all — the default built-in MetadataIndexManager has
// hydrateAll() but no warm(), exactly as it did before this change.
const metadataIndex = (brain as any).metadataIndex
expect(typeof metadataIndex.warm).not.toBe('function')
expect(typeof metadataIndex.hydrateAll).toBe('function')
const report = await brain.warm()
expect(report.metadata.outcome).toBe('warmed')
await brain.close()
})
})
}) })

View file

@ -0,0 +1,141 @@
/**
* @module tests/unit/transaction/timeout-never-internally-retried
* @description Regression pin for the no-hot-retry contract (8.10.1).
*
* A production incident: a native-provider op ground 38-40s inside a
* transaction, blew the ~32s budget, was rolled back, and a CONSUMER pipeline
* hot-retried the identical operation into a 6-minute 100%-CPU storm.
* Investigation established brainy itself never auto-retries a
* `TransactionTimeoutError` the storm was entirely the consumer's hot-retry
* loop, driven by a "retryable" doc-prose claim with no machine-readable
* contract. This file pins the brainy-side half of that story so it can never
* regress silently:
*
* (i) the underlying engine (`TransactionManager.executeTransaction()`
* `Transaction.execute()`) the exact machinery every single-record
* write (`add`/`update`/`remove`/...) drives via
* `Brainy.persistSingleOp()` never internally re-executes a timed-out
* operation, and the error it surfaces carries `retryable === true` and
* `hotRetryUnsafe === true` (see `src/transaction/errors.ts`).
*
* Constructed directly (mirrors the existing
* `tests/unit/transaction/timeout-rollback.test.ts` pattern) rather than
* through a real `brain.add()` call: `transactTimeoutBudget()` floors
* every single-op write's budget at `opCount * 2000`ms with NO override
* seam (`transactionBudgetFloorMs` only RAISES that floor it cannot
* lower it below the per-op-count term), so getting a real `add()` to
* time out requires a multi-second sleep. The engine-level
* `options.timeout` override used here is the exact same
* `TransactionManager`/`Transaction` code `persistSingleOp` calls
* pinning it here pins add()'s guarantee without paying that wall-clock
* cost.
*
* (ii) `Brainy.add()`'s upsert-race retry loop (src/brainy.ts,
* `MAX_UPSERT_ATTEMPTS = 10`) proving the loop's `catch` treats a
* `TransactionTimeoutError` as terminal (immediate rethrow) rather than
* the `InsertPreconditionExistsSignal` it retries on, so a mid-flight
* timeout can never be silently swallowed and re-attempted up to 10
* times.
*/
import { describe, it, expect } from 'vitest'
import { TransactionManager } from '../../../src/transaction/TransactionManager.js'
import type { Operation, RollbackAction } from '../../../src/transaction/types.js'
import { TransactionTimeoutError } from '../../../src/transaction/errors.js'
import { Brainy } from '../../../src/brainy.js'
import { NounType } from '../../../src/types/graphTypes.js'
const sleep = (ms: number) => new Promise<void>((r) => setTimeout(r, ms))
// Brainy's ValidationConfig fixes vectors at exactly 384 dimensions
// (src/utils/paramValidation.ts) — match it so add() doesn't reject test data.
const DIM = 384
const V = (): number[] => Array(DIM).fill(0.1)
/** An operation that counts every `execute()` invocation — the re-drive detector. */
function countingOp(opts: { delayMs?: number; name: string }): Operation & { calls: number } {
const op = {
name: opts.name,
calls: 0,
async execute(): Promise<RollbackAction | undefined> {
op.calls++
if (opts.delayMs) await sleep(opts.delayMs)
return async () => {}
}
}
return op
}
describe('transaction timeouts are never internally re-driven (8.10.1 no-hot-retry contract)', () => {
it('(i) the single-op engine (TransactionManager.executeTransaction / Transaction.execute — what add() drives via persistSingleOp) runs the overrun operation EXACTLY once and surfaces ONE retryable+hotRetryUnsafe error', async () => {
const manager = new TransactionManager()
const op0 = countingOp({ name: 'op0-overruns-budget', delayMs: 30 })
const op1 = countingOp({ name: 'op1-must-never-start' })
let caught: unknown
try {
await manager.executeTransaction(
async (tx) => {
tx.addOperation(op0)
tx.addOperation(op1)
},
// Tiny explicit override — the same override seam `transact()`
// exposes as `options.timeoutMs`; wins outright over the
// opCount*2000 floor that gates every real single-op write
// (transactTimeoutBudget()'s override semantics).
{ timeout: 5 }
)
} catch (err) {
caught = err
}
expect(caught).toBeInstanceOf(TransactionTimeoutError)
const err = caught as TransactionTimeoutError
// The machine-readable contract callers branch on instead of parsing
// message text (src/transaction/errors.ts).
expect(err.retryable).toBe(true)
expect(err.hotRetryUnsafe).toBe(true)
// The re-drive assertion: op0 (the one that overran) executed EXACTLY
// once — nothing inside TransactionManager/Transaction looped back and
// re-ran it — and op1 never started at all (the budget gate stopped it
// before it began, per Transaction.execute()'s per-operation loop).
expect(op0.calls).toBe(1)
expect(op1.calls).toBe(0)
})
it('(ii) add()\'s upsert-race retry loop (MAX_UPSERT_ATTEMPTS=10) exits on the FIRST TransactionTimeoutError — attempt counter stays at 1, never mistaken for the lost-insert-race signal it retries on', async () => {
const brain = new Brainy({
requireSubtype: false,
storage: { type: 'memory' },
silent: true
})
await brain.init()
let persistSingleOpCalls = 0
const timeoutError = new TransactionTimeoutError(5, 1, {
elapsedMs: 6,
totalOperations: 2,
operationName: 'SaveNounMetadata'
})
// Stub the private commit seam add() drives (persistSingleOp) to throw
// the exact error type the real engine surfaces on a mid-flight timeout.
// This test pins the upsert loop's EXCEPTION-HANDLING contract (does it
// retry a TransactionTimeoutError like it retries
// InsertPreconditionExistsSignal?), not the timing mechanics of a real
// timeout — those are pinned by test (i) and by
// tests/unit/transaction/timeout-rollback.test.ts.
;(brain as any).persistSingleOp = async (): Promise<never> => {
persistSingleOpCalls++
throw timeoutError
}
await expect(
brain.add({ data: 'a', type: NounType.Thing, vector: V() })
).rejects.toBe(timeoutError)
// The loop's attempt counter: exactly one call, never retried up to
// MAX_UPSERT_ATTEMPTS.
expect(persistSingleOpCalls).toBe(1)
await brain.close()
})
})