Compare commits

..

6 commits

Author SHA1 Message Date
4014e0f125 chore(release): 10.4.6
Some checks failed
Publish (The Source) / Publish to The Source registry (push) Successful in 12m31s
CI / Node 22 (push) Successful in 12m25s
CI / Node 24 (push) Successful in 12m20s
CI / Bun (latest) (push) Successful in 12m29s
CI / Integration + conformance (Node 22) (push) Failing after 16m56s
2026-08-31 14:50:45 -07:00
73500e7d10 fix(transact): metadata-index ops take their JSON-safe view at the crossing, not at construction
Some checks failed
CI / Node 24 (push) Successful in 12m30s
CI / Node 22 (push) Successful in 12m35s
CI / Integration + conformance (Node 22) (push) Failing after 16m54s
CI / Bun (latest) (push) Successful in 12m26s
transact()'s delete legs (direct unrelate and the noun-remove cascade) hand
the SAME verb object to the graph-retraction op and the metadata-retraction
op. The metadata leg sanitized at PLAN time, when the verb was still clean,
so the wrap returned the same reference — then the graph op's execute-time
endpoint resolution (deliberately deferred for same-batch forward refs)
mirrored BigInt sourceInt/targetInt onto the shared object, and the metadata
op crossed the seam with them. A strict provider rightly refuses that
crossing, so every transact-wrapped edge delete aborted; direct unrelate()
resolves ints at build time, before its sanitize, which is why no existing
gate saw it.

The JSON-safe view now lives in a shared leaf (utils/jsonSafeIndexMetadata)
and is applied INSIDE AddToMetadataIndexOperation and
RemoveFromMetadataIndexOperation at execute and rollback time — the one
place no plan-vs-execute ordering can bypass. Pins: the fleet repro, the
cascade shape, a mixed batch, and unit pins that mutate the entity after
construction against a strict seam (5 red before, 5 green after).
2026-08-31 12:59:40 -07:00
0f0022b1c9 chore(release): 10.4.5
Some checks failed
CI / Node 24 (push) Successful in 12m32s
CI / Node 22 (push) Successful in 12m35s
Publish (The Source) / Publish to The Source registry (push) Successful in 13m3s
CI / Bun (latest) (push) Successful in 12m29s
CI / Integration + conformance (Node 22) (push) Failing after 16m45s
2026-08-31 12:34:36 -07:00
d6bcb14f69 build(release): the docs-push step retires — this engine documents itself in its own repository
Some checks failed
CI / Node 22 (push) Successful in 12m27s
CI / Node 24 (push) Successful in 12m26s
CI / Integration + conformance (Node 22) (push) Failing after 17m1s
CI / Bun (latest) (push) Successful in 12m26s
The one-doc-set ruling (2026-08-31) gives soulcraft.com/docs to the paid
product alone; the site serves redirects for the slugs this rail used to
push. The push script stays in the tree as history; the rail stops calling
it.

(cherry picked from commit 655aa13ea7)
2026-08-31 10:47:08 -07:00
a963a744cc fix(generations): a sealed segment may only declare the generations it holds
Diagnosis of the "packed history is damaged" narration that fires on every
run of the affected stores. It is a WRITER defect, and the reader's refusal
was the symptom rather than the cause.

A sealed segment declares one contiguous range [firstGeneration,
lastGeneration], and every reader treats that range as containment:
coveringSegment is an interval test, hasGeneration returns true for anything
inside it, and open() seeds committedRanges from it.

repackHistory handed fold() a SPARSE batch. Three filters punch holes in its
candidate list mid-run — a generation absent from committedRanges never
appears, one still in the pending buffer is skipped, one whose tx.json will
not read is skipped — and fold() then computed the range from the first and
last survivor, claiming every generation in between. The next open merged
that mis-declared range back into committedRanges, re-admitting the hole as
committed history, so the following auto-compaction pass asked the packed
tier for a frame that was never written and failed. Re-merged at every open,
which is why it repeated on every run.

Confirmed against a forensic fixture: generation directories 1..2503 present
except exactly one, 1416; and its fact-log segment already showed the tell —
seg-...1410.bfl declaring 1410..1940 (531 generations) while recording 530
facts.

Three changes:

  - repackHistory folds each contiguous RUN as its own segment
    (`contiguousRuns`), so ranges describe exactly what the segments contain.
  - fold() REFUSES a non-contiguous batch, naming the gap and its width. The
    density law is now mechanical, so no future caller can reintroduce it. A
    refusal loses nothing: the generations stay live and readable.
  - Stores already carrying the damage heal instead of wedging. A segment
    whose declared span exceeds its frame count is SPARSE; `actualRanges()`
    reads the real generation list from its sidecar so open() never re-admits
    the holes, and readFrame reports such a hole as unpacked with a narration
    naming the segment, rather than throwing. A DENSE segment missing a frame
    is still loud damage — that one means the manifest and sidecar disagree.

Pins: nine unit cases (refusal and its message, honest ranges for separately
folded runs, a reconstructed pre-fix sparse segment serving its real frames
while reporting holes as unpacked, holes excluded from actualRanges, and the
dense-segment damage path still throwing) plus an end-to-end case that
deletes a generation directory and drives the real sequence — ordinary
close()-time repacking folds over the hole, then reopen and compact must both
complete. Verified red without the fix: the segment declared an
11-generation span while holding 10 frames.

(cherry picked from commit 9a888c37e9)
2026-08-31 10:47:08 -07:00
David Snelling
c99308710a fix(recovery): a torn generation-log tail is a terminal verdict, never a wait
Two halves of one defect, found by a seeded-SIGKILL crash lane.

THE FALSE POSITIVE. stampEntityTree() recorded generationStore.generation()
— the ALLOCATED counter, a number a write in flight has claimed and may
never commit — while the JSDoc beside it already said the source is the
committed generation. Every crash inside a write window therefore produced
a spurious verdict at the next open: either 'sourceGeneration N is ahead of
the log head N-1' (the allocated generation died with the process) or
'rollup invariant nounCount: stamped X, observed Y' (the recovery fold
folded facts the stamp's counts predate). Both told the operator to run
repairIndex() — a whole-store recount — for a store that was coherent.
Measured before this commit: 4 of 11 SIGKILL cycles on a healthy store
raised one of the two. The stamp and the open now both read
committedGeneration(), which is what every other open-time watermark in the
class already reasons about.

THE TERMINAL VERDICT. A stamp still ahead of committed truth after the
recovery fold witnesses a generation that is not in the log — the stamp's
fsync outlived the tail's, and there is nothing to arrive. That is its own
verdict state now ('torn'), never folded in with 'incoherent': the two have
opposite cures. A writer open demotes it — the unusable stamped surface is
re-derived at the committed generation from the live counters, O(1),
straight-line, no loop and no await on external progress, narrated with
both count sets, the stamp's path and its committedAt. A read-only open
cannot re-stamp, so it says so and names the cure instead of guessing, and
still serves. Neither branch waits, and neither locks an owner out of a
canonical tree the stamp only describes.

Pins: the verifier returns the torn verdict with both generations; a
fabricated head-behind-source store narrates precisely, demotes inside a
bounded open, serves its rows, and is quiet at the next open (the demotion
converges); a read-only open narrates the same verdict and leaves the bytes
untouched.

(cherry picked from commit 298cb6daca)
2026-08-31 10:47:08 -07:00
10 changed files with 279 additions and 217 deletions

View file

@ -2,6 +2,18 @@
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines. All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
### [10.4.6](https://source.soulcraft.com/soulcraftlabs/open-brainy/compare/v10.4.5...v10.4.6) (2026-08-31)
- fix(transact): metadata-index ops take their JSON-safe view at the crossing, not at construction (73500e7d)
### [10.4.5](https://source.soulcraft.com/soulcraftlabs/open-brainy/compare/v10.4.4...v10.4.5) (2026-08-31)
- build(release): the docs-push step retires — this engine documents itself in its own repository (d6bcb14f)
- fix(generations): a sealed segment may only declare the generations it holds (a963a744)
- fix(recovery): a torn generation-log tail is a terminal verdict, never a wait (c9930871)
### [10.4.4](https://source.soulcraft.com/soulcraftlabs/open-brainy/compare/v10.4.3...v10.4.4) (2026-08-28) ### [10.4.4](https://source.soulcraft.com/soulcraftlabs/open-brainy/compare/v10.4.3...v10.4.4) (2026-08-28)
- fix(vfs): the old-root sweep narrates only when it has something to say (d49148e1) - fix(vfs): the old-root sweep narrates only when it has something to say (d49148e1)

4
package-lock.json generated
View file

@ -1,12 +1,12 @@
{ {
"name": "@soulcraftlabs/brainy", "name": "@soulcraftlabs/brainy",
"version": "10.4.4", "version": "10.4.6",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "@soulcraftlabs/brainy", "name": "@soulcraftlabs/brainy",
"version": "10.4.4", "version": "10.4.6",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@msgpack/msgpack": "^3.1.2", "@msgpack/msgpack": "^3.1.2",

View file

@ -1,6 +1,6 @@
{ {
"name": "@soulcraftlabs/brainy", "name": "@soulcraftlabs/brainy",
"version": "10.4.4", "version": "10.4.6",
"brainyContract": 1, "brainyContract": 1,
"description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. Stage 3 CANONICAL: 42 nouns × 127 verbs covering 96-97% of all human knowledge.", "description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. Stage 3 CANONICAL: 42 nouns × 127 verbs covering 96-97% of all human knowledge.",
"main": "dist/index.js", "main": "dist/index.js",

View file

@ -1,76 +0,0 @@
{
"product": "brainy",
"entries": [
{
"version": "11.0.5",
"date": "2026-09-02",
"headline": "Graph-first finds in production, and opens that stop rescanning history",
"items": [
"find({ connected, where }) now walks the neighbours first and filters only those rows through a native door — correct at every page and O(neighbours), never the whole store.",
"related() with a list of verb types returns every requested kind (a fast path had silently kept only the first).",
"Deferred-embedding recovery resumes from a low-water mark instead of rescanning the whole generation log at every open — measured at two minutes on a large brain, now milliseconds."
],
"url": null,
"thumb": null
},
{
"version": "11.0.4",
"date": "2026-09-01",
"headline": "Closes in milliseconds, index rebuilds without the disk-sync storm",
"items": [
"close() no longer pays deferred compaction or waits out an in-flight rebuild — measured 8 ms against the 4-minute closes it replaces; deferred work resumes at the next open, in the background.",
"The metadata index's rebuild syncs to disk per shard instead of per row, and the durability point moved to the publish step — the same guarantee, a fraction of the disk traffic.",
"A new native filter door evaluates queries over exactly the candidate rows a graph walk found, never the whole store."
],
"url": null,
"thumb": null
},
{
"version": "11.0.3",
"date": "2026-09-01",
"headline": "The embedding upgrade ceremony runs on every brain",
"items": [
"A brain opened through the standard plugin now carries its embedding-model identity, so the full-precision upgrade ceremony can run on it.",
"A one-fix release; nothing else changed."
],
"url": null,
"thumb": null
},
{
"version": "11.0.2",
"date": "2026-08-31",
"headline": "One embedding quality everywhere, 34× faster imports",
"items": [
"Every runtime embeds with the same full-precision model — search quality no longer depends on where you run.",
"Bulk embedding measured 3.14.2× faster, and an online re-embed ceremony upgrades existing stores without downtime.",
"The engine's change feed is documented, with the SSE/WebSocket fan-out pattern for realtime surfaces."
],
"url": null,
"thumb": null
},
{
"version": "11.0.1",
"date": "2026-08-31",
"headline": "Deletes inside transactions are safe",
"items": [
"Deleting relations inside a transact() no longer corrupts index bookkeeping.",
"A store that deletes its last relation keeps serving instead of refusing."
],
"url": null,
"thumb": null
},
{
"version": "11.0.0",
"date": "2026-08-28",
"headline": "One install, one engine — Brainy",
"items": [
"The former two-package pair is one package: the native engine under the familiar API. One import is the whole install.",
"A missing native build refuses loudly with its cures named; nothing falls back silently.",
"Stores open in place — no migration."
],
"url": null,
"thumb": null
}
],
"history": "The version line continues from the 4.3.x native-engine releases; their record lives in the product repository's CHANGELOG.md."
}

View file

@ -1,110 +0,0 @@
{
"product": "open-brainy",
"entries": [
{
"version": "10.4.9",
"date": "2026-09-02",
"headline": "Graph-first finds, honest verb arrays, and opens that stop rescanning history",
"items": [
"find({ connected, where }) now walks the neighbours first and filters only those rows — correct at every page, and O(neighbours) instead of O(store).",
"related() with a list of verb types (or sources, or targets) returns every requested kind — four fast paths silently kept only the first.",
"Deferred-embedding recovery resumes from a low-water mark instead of rescanning the whole generation log at every open — measured at two minutes on a large brain, now milliseconds."
],
"url": "https://source.soulcraft.com/soulcraftlabs/open-brainy/releases/tag/v10.4.9",
"thumb": null
},
{
"version": "10.4.7",
"date": "2026-09-01",
"headline": "Count ledgers can no longer race themselves",
"items": [
"Concurrent count flushes coalesce into one writer with a trailing pass — parallel flushes can no longer corrupt a store's count ledger.",
"Atomic writes carry a per-process sequence, so two processes' temp files can never collide."
],
"url": "https://source.soulcraft.com/soulcraftlabs/open-brainy/releases/tag/v10.4.7",
"thumb": null
},
{
"version": "10.4.6",
"date": "2026-08-31",
"headline": "Transactions cross the index seam safely",
"items": [
"Deleting relations inside a transact() no longer fails against the metadata index — operations take a JSON-safe view at the moment they execute.",
"Fixes a class of transaction failures on stores with integer-mapped relation endpoints."
],
"url": "https://source.soulcraft.com/soulcraftlabs/open-brainy/releases/tag/v10.4.6",
"thumb": null
},
{
"version": "10.4.5",
"date": "2026-08-31",
"headline": "Recovery tells the truth, docs live at home",
"items": [
"A torn generation-log tail is a terminal verdict with a named cure — never an endless wait at open.",
"A sealed segment declares only the generations it actually holds.",
"The engine's documentation now publishes from its own repository."
],
"url": "https://source.soulcraft.com/soulcraftlabs/open-brainy/releases/tag/v10.4.5",
"thumb": null
},
{
"version": "10.4.4",
"date": "2026-08-28",
"headline": "Faster opens, quieter idle",
"items": [
"Opening a store discovers generations from directory names instead of walking the log, and answers \"any entities?\" with one directory read.",
"The flush-request watch is event-driven; idle stores stop paying a polling heartbeat.",
"A slow open now names the exact step it is in, so operators see what is being paid and why."
],
"url": "https://source.soulcraft.com/soulcraftlabs/open-brainy/releases/tag/v10.4.4",
"thumb": null
},
{
"version": "10.4.3",
"date": "2026-08-27",
"headline": "Open Brainy, under its own name",
"items": [
"The same engine as 10.4.2, now published as @soulcraftlabs/brainy — the MIT reference engine, on The Source.",
"No code changes; your imports change once and everything else stays put."
],
"url": "https://source.soulcraft.com/soulcraftlabs/open-brainy/releases/tag/v10.4.3",
"thumb": null
},
{
"version": "10.4.2",
"date": "2026-08-27",
"headline": "Vectors that lie are refused, counts that drift are caught",
"items": [
"A zero-norm vector is not a vector: the index refuses them, rebuilds skip them, and a sanctioned unvector door removes them cleanly.",
"The canonical count ledger derives from identity records and marks legacy-derived ledgers suspect at load.",
"Plugin activation failures keep their original error as cause, so the real frame reaches your logs."
],
"url": "https://source.soulcraft.com/soulcraftlabs/open-brainy/releases/tag/v10.4.2",
"thumb": null
},
{
"version": "10.4.1",
"date": "2026-08-26",
"headline": "Writes that change nothing cost nothing",
"items": [
"The read gate is per index family, and a write carrying unchanged data never re-embeds.",
"The vectored-row count joins the ledger, so vector coverage is a number you can read, not a guess."
],
"url": "https://source.soulcraft.com/soulcraftlabs/open-brainy/releases/tag/v10.4.1",
"thumb": null
},
{
"version": "10.4.0",
"date": "2026-08-26",
"headline": "Repair routing, the vector ledger, and honest empties",
"items": [
"Repairs route to the index that owns the damage, and the open gate closes the vector leg until coverage is proven.",
"An empty string is real data, not a missing field.",
"The metadata crossing never carries raw integer relation endpoints — a whole class of serialization faults closed."
],
"url": "https://source.soulcraft.com/soulcraftlabs/open-brainy/releases/tag/v10.4.0",
"thumb": null
}
],
"history": "Earlier releases are recorded in CHANGELOG.md in this repository."
}

View file

@ -237,7 +237,7 @@ fi
# and RELEASES.md are the record; this just gives The Source's UI a release page). # and RELEASES.md are the record; this just gives The Source's UI a release page).
echo -e "${BLUE}🔟 Creating release page on The Source...${NC}" echo -e "${BLUE}🔟 Creating release page on The Source...${NC}"
if [ -n "${FORGEJO_RELEASE_TOKEN:-}" ]; then if [ -n "${FORGEJO_RELEASE_TOKEN:-}" ]; then
if curl -sf -X POST "https://source.soulcraft.com/api/v1/repos/soulcraftlabs/open-brainy/releases" \ if curl -sf -X POST "https://source.soulcraft.com/api/v1/repos/soulcraft/brainy/releases" \
-H "Authorization: token ${FORGEJO_RELEASE_TOKEN}" -H "Content-Type: application/json" \ -H "Authorization: token ${FORGEJO_RELEASE_TOKEN}" -H "Content-Type: application/json" \
-d "{\"tag_name\":\"v${NEW_VERSION}\",\"name\":\"v${NEW_VERSION}\",\"prerelease\":${PRERELEASE}}" >/dev/null; then -d "{\"tag_name\":\"v${NEW_VERSION}\",\"name\":\"v${NEW_VERSION}\",\"prerelease\":${PRERELEASE}}" >/dev/null; then
echo -e "${GREEN}✅ Release page created on The Source${NC}\n" echo -e "${GREEN}✅ Release page created on The Source${NC}\n"

View file

@ -15,6 +15,7 @@ import { JsHnswVectorIndex } from './hnsw/hnswIndex.js'
import { createStorage, resolveFilesystemRoot } from './storage/storageFactory.js' import { createStorage, resolveFilesystemRoot } from './storage/storageFactory.js'
import type { StorageOptions } from './storage/storageFactory.js' import type { StorageOptions } from './storage/storageFactory.js'
import { rebuildCounts } from './utils/rebuildCounts.js' import { rebuildCounts } from './utils/rebuildCounts.js'
import { jsonSafeIndexMetadata } from './utils/jsonSafeIndexMetadata.js'
import type { MetadataWriteBuffer } from './utils/metadataWriteBuffer.js' import type { MetadataWriteBuffer } from './utils/metadataWriteBuffer.js'
import { BaseStorage } from './storage/baseStorage.js' import { BaseStorage } from './storage/baseStorage.js'
import { import {
@ -4203,32 +4204,19 @@ export class Brainy<T = any> implements BrainyInterface<T> {
*/ */
/** /**
* @description A JSON-safe view of a record bound for the metadata-index * @description A JSON-safe view of a record bound for the metadata-index
* crossing. The seam's metadata is JSON-safe BY CONTRACT (a native provider * crossing delegates to the shared {@link jsonSafeIndexMetadata} leaf,
* serializes it; u64 ints as Number corrupt above 2^53) but * which the metadata-index transaction operations ALSO apply at execute
* {@link resolveVerbEndpointInts} MIRRORS the resolved endpoint ints onto * and rollback time. This plan-time wrap alone proved insufficient: it
* the verb object itself as BigInt (`verb.sourceInt`/`targetInt`), so a * returns the same reference when the record is clean, and `transact()`'s
* verb object reused as index metadata carried BigInts into * delete legs share that reference with a graph-retraction op whose
* JSON.stringify, which throws, aborting the whole transaction (found by * execute-time endpoint resolution mirrors BigInt ints onto it (the full
* the first joint pair gate). Endpoint ints ride their OWN op params on the * aliasing story lives on the leaf module's doc).
* graph legs the metadata crossing drops every BigInt-valued top-level
* key instead of guessing at a lossy numeric encoding.
* @param metadata - The candidate index-metadata record. * @param metadata - The candidate index-metadata record.
* @returns The same object when already JSON-safe, else a shallow copy * @returns The same object when already JSON-safe, else a shallow copy
* without the BigInt-valued keys. * without the BigInt-valued keys.
*/ */
private static jsonSafeIndexMetadata(metadata: unknown): unknown { private static jsonSafeIndexMetadata(metadata: unknown): unknown {
if (metadata === null || typeof metadata !== 'object') return metadata return jsonSafeIndexMetadata(metadata)
const rec = metadata as Record<string, unknown>
let hasBigint = false
for (const k in rec) {
if (typeof rec[k] === 'bigint') { hasBigint = true; break }
}
if (!hasBigint) return metadata
const out: Record<string, unknown> = {}
for (const k in rec) {
if (typeof rec[k] !== 'bigint') out[k] = rec[k]
}
return out
} }
private metadataIndexRetractionOp( private metadataIndexRetractionOp(

View file

@ -14,6 +14,7 @@ import type { MetadataIndexManager } from '../../utils/metadataIndex.js'
import type { GraphVerb } from '../../coreTypes.js' import type { GraphVerb } from '../../coreTypes.js'
import type { Operation, RollbackAction } from '../types.js' import type { Operation, RollbackAction } from '../types.js'
import { isZeroNormVector } from '../../utils/distance.js' import { isZeroNormVector } from '../../utils/distance.js'
import { jsonSafeIndexMetadata } from '../../utils/jsonSafeIndexMetadata.js'
import { prodLog } from '../../utils/logger.js' import { prodLog } from '../../utils/logger.js'
/** /**
@ -390,13 +391,21 @@ export class AddToMetadataIndexOperation implements Operation {
// rollback so add + undo reference the same watermark. // rollback so add + undo reference the same watermark.
const generation = this.generationFn?.() const generation = this.generationFn?.()
// Add to metadata index (skipFlush=true for transaction atomicity) // The JSON-safe view is taken HERE, per crossing, never at construction:
await this.index.addToIndex(this.id, this.entity, true, false, generation) // the entity reference this op holds can be mutated between plan and
// execute (a graph op's execute-time endpoint-int resolution mirrors
// BigInts onto a shared verb object) — see jsonSafeIndexMetadata's
// module doc.
await this.index.addToIndex(
this.id, jsonSafeIndexMetadata(this.entity), true, false, generation
)
// Return rollback action // Return rollback action
return async () => { return async () => {
// Remove from metadata index // Remove from metadata index
await this.index.removeFromIndex(this.id, this.entity, generation) await this.index.removeFromIndex(
this.id, jsonSafeIndexMetadata(this.entity), generation
)
} }
} }
} }
@ -432,13 +441,21 @@ export class RemoveFromMetadataIndexOperation implements Operation {
// Resolve the removal generation once; reuse it for the rollback re-add. // Resolve the removal generation once; reuse it for the rollback re-add.
const generation = this.generationFn?.() const generation = this.generationFn?.()
// Remove from metadata index // Sanitized per crossing, never at construction — transact()'s delete
await this.index.removeFromIndex(this.id, this.entity, generation) // legs hand this op the SAME verb object the graph-retraction op's
// execute-time endpoint resolution mutates (BigInt sourceInt/targetInt),
// so a plan-time view aliases the pollution. See jsonSafeIndexMetadata's
// module doc.
await this.index.removeFromIndex(
this.id, jsonSafeIndexMetadata(this.entity), generation
)
// Return rollback action // Return rollback action
return async () => { return async () => {
// Re-add with original metadata (skipFlush=true) // Re-add with original metadata (skipFlush=true)
await this.index.addToIndex(this.id, this.entity, true, false, generation) await this.index.addToIndex(
this.id, jsonSafeIndexMetadata(this.entity), true, false, generation
)
} }
} }
} }

View file

@ -0,0 +1,47 @@
/**
* @module utils/jsonSafeIndexMetadata
* @description The metadata-index crossing's JSON-safety law, as a leaf
* function both the coordinator and the transaction operations share.
*
* The seam's metadata is JSON-safe BY CONTRACT (a native provider serializes
* it; u64 ints as Number corrupt above 2^53) but `resolveVerbEndpointInts`
* MIRRORS the resolved endpoint ints onto the verb object itself as BigInt
* (`verb.sourceInt`/`targetInt`), so a verb object reused as index metadata
* carries BigInts into JSON.stringify, which throws, aborting the whole
* transaction. Endpoint ints ride their OWN op params on the graph legs the
* metadata crossing drops every BigInt-valued top-level key instead of
* guessing at a lossy numeric encoding.
*
* WHY THIS IS A LEAF MODULE, ENFORCED AT THE CROSSING: sanitizing only at
* operation-construction time is not enough. `transact()`'s delete legs pass
* the SAME verb object to both the graph-retraction op (whose endpoint-int
* thunk deliberately resolves at EXECUTE time, for same-batch forward refs)
* and the metadata-retraction op. At plan time the verb is still clean, so a
* plan-time sanitize returns the same reference then the graph op executes
* first, mirrors the BigInt ints onto the shared object, and the metadata op
* crosses the seam with them (found by the first fleet adoption of the native
* pair: every transact-wrapped edge delete aborted). The crossing itself is
* the only place ordering cannot bypass.
*/
/**
* A JSON-safe view of a record bound for the metadata-index crossing.
*
* @param metadata - The candidate index-metadata record.
* @returns The same object when already JSON-safe, else a shallow copy
* without the BigInt-valued keys.
*/
export function jsonSafeIndexMetadata(metadata: unknown): unknown {
if (metadata === null || typeof metadata !== 'object') return metadata
const rec = metadata as Record<string, unknown>
let hasBigint = false
for (const k in rec) {
if (typeof rec[k] === 'bigint') { hasBigint = true; break }
}
if (!hasBigint) return metadata
const out: Record<string, unknown> = {}
for (const k in rec) {
if (typeof rec[k] !== 'bigint') out[k] = rec[k]
}
return out
}

View file

@ -0,0 +1,184 @@
/**
* @module tests/integration/transact-edge-delete-bigint-aliasing
* @description Regression for a fleet-adoption blocker: ANY edge delete
* inside `transact()` a direct unrelate or a noun-remove's cascade
* aborted with the metadata seam's BigInt JSON-guard error on a strict
* (native) metadata provider.
*
* The aliasing chain: `planTxUnrelate`/the remove-cascade pass the SAME verb
* object to the graph-retraction op and the metadata-retraction op. The
* metadata leg's JSON-safe wrap ran at PLAN time, when the verb was still
* clean so it returned the same reference. At EXECUTE time the graph op
* runs first and `resolveVerbEndpointInts` mirrors BigInt
* `sourceInt`/`targetInt` onto the shared object (deliberately deferred for
* same-batch forward refs see transact-forward-ref-graph.test.ts); the
* metadata op then crossed the seam with the polluted object. Direct
* `unrelate()` resolves ints at BUILD time, before its sanitize, which is why
* only the transact() shapes ever hit it.
*
* Fix under pin: the JSON-safe view is taken AT THE CROSSING inside the
* metadata-index operations' execute/rollback so no plan-vs-execute
* ordering can bypass it. The JS baseline index tolerates BigInts (it would
* mask the bug), so these pins SPY on the seam and assert what actually
* crossed, exactly as a strict native provider would judge it.
*/
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
import * as fs from 'node:fs'
import * as os from 'node:os'
import * as path from 'node:path'
import { Brainy } from '../../src/brainy.js'
import { NounType, VerbType } from '../../src/types/graphTypes.js'
import {
AddToMetadataIndexOperation,
RemoveFromMetadataIndexOperation
} from '../../src/transaction/operations/index.js'
let seq = 0
const freshId = (): string =>
`00000000-0000-4000-8000-${(++seq).toString(16).padStart(12, '0')}`
/** Top-level BigInt-valued keys of a candidate seam crossing (the guard's law). */
const bigintKeys = (metadata: unknown): string[] => {
if (metadata === null || typeof metadata !== 'object') return []
return Object.entries(metadata as Record<string, unknown>)
.filter(([, v]) => typeof v === 'bigint')
.map(([k]) => k)
}
describe('transact() edge deletes never carry BigInt across the metadata seam', () => {
let dir: string
let brain: any
let crossings: Array<{ door: string; id: string; keys: string[] }>
beforeEach(async () => {
process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-tx-bigint-'))
brain = new Brainy({
requireSubtype: false,
storage: { type: 'filesystem', path: dir },
dimensions: 384,
silent: true
})
await brain.init()
// Spy on the seam the way a strict native provider judges it: record the
// BigInt-valued top-level keys of every metadata argument that crosses.
// The JS baseline index tolerates BigInts, so without this the baseline
// run would green a shape the native pair aborts on.
crossings = []
const index = brain.metadataIndex
for (const door of ['addToIndex', 'removeFromIndex'] as const) {
const real = index[door].bind(index)
index[door] = (id: string, metadata: unknown, ...rest: unknown[]) => {
crossings.push({ door, id, keys: bigintKeys(metadata) })
return real(id, metadata, ...rest)
}
}
})
afterEach(async () => {
await brain.close()
fs.rmSync(dir, { recursive: true, force: true })
})
it('CASE 1 (the fleet repro): relate, then transact([{op: unrelate}])', async () => {
const a = await brain.add({ id: freshId(), data: 'a', type: NounType.Thing })
const b = await brain.add({ id: freshId(), data: 'b', type: NounType.Thing })
const verbId = await brain.relate({ from: a, to: b, type: VerbType.RelatedTo })
crossings.length = 0
await brain.transact([{ op: 'unrelate', id: verbId }])
const polluted = crossings.filter((c) => c.keys.length > 0)
expect(polluted).toEqual([])
expect(await brain.storage.getVerb(verbId)).toBeFalsy()
})
it('CASE 2 (the cascade shape): transact([{op: remove}]) cascading edge deletes', async () => {
const a = await brain.add({ id: freshId(), data: 'a', type: NounType.Thing })
const b = await brain.add({ id: freshId(), data: 'b', type: NounType.Thing })
const c = await brain.add({ id: freshId(), data: 'c', type: NounType.Thing })
const ab = await brain.relate({ from: a, to: b, type: VerbType.RelatedTo })
const ca = await brain.relate({ from: c, to: a, type: VerbType.RelatedTo })
crossings.length = 0
await brain.transact([{ op: 'remove', id: a }])
const polluted = crossings.filter((c2) => c2.keys.length > 0)
expect(polluted).toEqual([])
expect(await brain.get(a)).toBeFalsy()
expect(await brain.storage.getVerb(ab)).toBeFalsy()
expect(await brain.storage.getVerb(ca)).toBeFalsy()
})
it('CASE 3 (one batch, both legs): adds + relate + unrelate of a pre-existing edge', async () => {
const a = await brain.add({ id: freshId(), data: 'a', type: NounType.Thing })
const b = await brain.add({ id: freshId(), data: 'b', type: NounType.Thing })
const old = await brain.relate({ from: a, to: b, type: VerbType.RelatedTo })
const x = freshId()
crossings.length = 0
await brain.transact([
{ op: 'add', id: x, data: 'x', type: NounType.Thing },
{ op: 'relate', from: a, to: x, type: VerbType.RelatedTo },
{ op: 'unrelate', id: old }
])
const polluted = crossings.filter((c) => c.keys.length > 0)
expect(polluted).toEqual([])
expect(await brain.storage.getVerb(old)).toBeFalsy()
const edges = await brain.related({ from: a })
expect(edges.length).toBe(1)
expect(edges[0].id).not.toBe(old)
})
})
describe('the metadata-index operations sanitize at the crossing, not at construction', () => {
/** A strict seam: refuses BigInts exactly as the native provider does. */
const strictIndex = () => {
const seen: Array<{ door: string; keys: string[] }> = []
const judge = (door: string, metadata: unknown) => {
const keys = bigintKeys(metadata)
seen.push({ door, keys })
if (keys.length > 0) {
throw new Error(
`${door}: the metadata object violates the provider seam's JSON ` +
`contract — BigInt at ${keys.join(', ')}.`
)
}
}
return {
seen,
addToIndex: async (_id: string, metadata: unknown) => judge('addToIndex', metadata),
removeFromIndex: async (_id: string, metadata: unknown) => judge('removeFromIndex', metadata)
}
}
it('RemoveFromMetadataIndexOperation: entity mutated AFTER construction still crosses clean', async () => {
const index = strictIndex()
const verb: Record<string, unknown> = { id: 'v1', sourceId: 'a', targetId: 'b' }
const op = new RemoveFromMetadataIndexOperation(index as any, 'v1', verb, () => 7n)
// The graph leg's execute-time endpoint resolution, simulated: the shared
// object is polluted between plan and execute.
verb.sourceInt = 800_000n
verb.targetInt = 800_001n
const rollback = await op.execute()
await rollback()
expect(index.seen.map((s) => s.keys)).toEqual([[], []])
})
it('AddToMetadataIndexOperation: same law on the add leg and its rollback', async () => {
const index = strictIndex()
const verb: Record<string, unknown> = { id: 'v2', sourceId: 'a', targetId: 'b' }
const op = new AddToMetadataIndexOperation(index as any, 'v2', verb, () => 7n)
verb.sourceInt = 800_000n
verb.targetInt = 800_001n
const rollback = await op.execute()
await rollback()
expect(index.seen.map((s) => s.keys)).toEqual([[], []])
})
})