soulcraftlabs/open-brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 343 Packages Wiki Activity Actions 6
open-brainy/tests/integration
History
David Snelling 258176e6e6
Some checks are pending
CI / Node 22 (push) Waiting to run
Details
CI / Node 24 (push) Waiting to run
Details
CI / Integration + conformance (Node 22) (push) Waiting to run
Details
CI / Bun (latest) (push) Waiting to run
Details
perf(idle): the flush-request watch is event-driven; the heartbeat is observability 
Three idle-burn items from the steady-state audit, and one correction.

THE FLUSH-REQUEST WATCH (the strongest of them). It readdir'd the request
directory every 500 ms, per brain, for the life of every writer — armed on
every non-reader brain whether or not any inspector process existed. On a
process holding 21 brains that is 42 directory reads per second on a
completely idle service, plus a stale-request GC on every one of them. It now
uses fs.watch, so the arrival itself wakes it and a request is seen SOONER
than the poll saw it. Two concessions ride along, both stated in the code: a
30s safety sweep (fs.watch drops events on some network and fuse filesystems,
and the GC needs a tick of its own — 0.7 reads/s across 21 brains where the
poll cost 42), and a fall back to the original 500 ms poll, narrated, on a
filesystem that cannot watch at all, because an inspector whose request is
never seen waits forever.

THE WRITER HEARTBEAT goes 10s → 60s. It is observability ONLY — staleness is
decided by pid liveness and the fence compares pid + hostname, so no decision
anywhere reads the timestamp — and at 10s it was a lock-file write every ten
seconds per brain forever, for a value nothing computes with. An operator
still sees a heartbeat inside the minute.

THE HEALTH NARRATION dedupes by CONTENT, not by the provider's generation
counter. That counter bumps on every ledger mutation and rebuild boundary, so
a provider bumping it on routine work re-emitted the same unchanged line on
every read, while one that never bumped could suppress a line whose reasons
had genuinely changed. The generation is still reported; it no longer decides
whether the line is worth saying.

CORRECTION, and it is against my own earlier claim: the idle-flush commit read
the field's "21 brains, no writes, a flush every ~35s, 1.26 cores" as caused by
the flush path. That does not follow — this engine's cadence is write-driven
(every trigger runs through noteWriteForPersistence, which only a committed
write calls), so something was CALLING flush() on those brains and the caller
is still unidentified. The clean-flush gate makes such a call free; it does not
account for it. The code comments and the idle lane now say exactly that.

Pins: tests/integration/flush-watcher-event-driven.test.ts — an idle writer
makes at most one request-directory read in 8 seconds (the old poll made ~16),
and a dropped request is still acked well inside the safety sweep.
2026-08-28 11:25:47 -07:00
..
adopt-drift-cure.test.ts fix(adoption): the baseline backfill cures hydration-law drift — existing brains reach the crash-safe default with zero operator steps 2026-08-12 11:48:17 -07:00
adopt-large-baseline.test.ts fix(adoption): the baseline backfill runs to completion — one call adopts a pre-log baseline of any size 2026-08-17 12:53:04 -07:00
advanced-apis-regression.test.ts feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law 2026-08-03 16:59:32 -07:00
aggregate-reserved-fields.test.ts feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law 2026-08-03 16:59:32 -07:00
aggregation-delete.test.ts feat(8.0): id-normalization (#18) + aggregation min/max delete-safety + RC-safe release 2026-06-20 14:46:40 -07:00
aggregation-lifecycle-catchup.test.ts fix(aggregation): the lifecycle cluster — flush stamps, behind-stamp catches up incrementally, the native rebuild finally gets invoked, deletes are never silently skipped 2026-08-05 15:49:12 -07:00
aggregation-state-persistence.test.ts fix: exception-safe aggregation backfill + generation-verified adoption + loud open-path guards 2026-07-17 16:00:11 -07:00
aggregation.test.ts chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
all-apis-comprehensive.test.ts feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law 2026-08-03 16:59:32 -07:00
api-parameter-validation.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
asof-semantic-recall.test.ts feat(temporal): as-of semantic recall joins the release contract — past vectors byte-exact, pinned 2026-08-10 09:42:08 -07:00
background-dedup-lifecycle.test.ts fix: import dedup off-switch honesty + brain-owned lifecycle for the background pass 2026-07-18 10:40:45 -07:00
batchImportWithRelations.test.ts fix(storage): an unknown nested storage config can never silently land on the shared default root 2026-08-25 10:47:51 -07:00
brain-relocation.test.ts feat(engine): the wiring wave — stamps ride every flush, provider generations, waitForIndexed, adopt-backfill, match-all serves 2026-08-10 10:55:11 -07:00
brainy-complete.integration.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
brainy-core.integration.test.ts fix(add): empty string is real data, not a missing field 2026-08-25 10:10:19 -07:00
brainy-phase1c-integration.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
bun-runtime-test.ts chore(8.0): modernize toolchain + position Bun as a runtime 2026-07-01 09:16:46 -07:00
canonical-count-ledger.test.ts fix(vfs): the VFS root never persists a zero-norm vector 2026-08-27 09:28:44 -07:00
clear-persistence.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
clear-vfs.test.ts docs(8.0): consistency-model concept + snapshots guide — Db API replaces branching docs 2026-06-11 08:37:26 -07:00
cold-graph-connected-8.0.test.ts feat(health): the gate reads the named report — reads refuse loudly, never rebuild; open serves before it returns; the ceremony door 2026-08-24 12:45:51 -07:00
cortex-compat.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
count-invariant.test.ts fix: spine hardening pass 1 (part) — count symmetry, honest partial-load, flush durability, read-fault propagation 2026-07-13 08:50:07 -07:00
count-ledger-identity-record.test.ts fix(storage): a suspect count ledger heals itself, and counts.json is written atomically 2026-08-28 10:28:25 -07:00
count-synchronization.test.ts test(8.0): close brains in afterEach (count-sync, get-relations teardown) 2026-06-16 13:18:14 -07:00
counter-recount.test.ts fix: honest counters — removal never re-reads the removed record + repairIndex recounts and persists all rollups 2026-07-14 11:51:44 -07:00
db-asof-vector-defer.test.ts feat(8.0): #35 part-3 — supply at-gen candidate vectors for the native exact-rerank 2026-06-23 16:50:41 -07:00
db-mvcc.test.ts feat(log): log authority is the fleet default — adopt-at-open, oracle-gated; plus the power-cut throw-site cures and the loud torn-record contract 2026-08-11 08:37:38 -07:00
db-temporal.test.ts feat(log): log authority is the fleet default — adopt-at-open, oracle-gated; plus the power-cut throw-site cures and the loud torn-record contract 2026-08-11 08:37:38 -07:00
deferred-embedding.test.ts feat(embedding): MT5 — deferred embedding with durable markers; write acks never wait on a neural net 2026-08-05 16:26:43 -07:00
delete-full-removal.test.ts fix: full-removal canonical deletes + family-scoped migration gate 2026-07-14 10:11:53 -07:00
durability-kill-matrix.test.ts feat(log): log authority is the fleet default — adopt-at-open, oracle-gated; plus the power-cut throw-site cures and the loud torn-record contract 2026-08-11 08:37:38 -07:00
embed-markers-in-log.test.ts feat(embedding): deferred-embed markers become log records — the sidecar recovery path is deleted 2026-08-10 11:27:07 -07:00
entity-confidence-weight.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
entity-tree-stamp.test.ts feat: provider access to the fact log + shared stamp verifier via internals 2026-07-15 12:36:27 -07:00
enumeration-population-law.test.ts fix(storage): enumeration re-keys on the identity record, not the vector leg 2026-08-27 12:38:52 -07:00
fact-log-contracts.test.ts feat(log): log authority is the fleet default — adopt-at-open, oracle-gated; plus the power-cut throw-site cures and the loud torn-record contract 2026-08-11 08:37:38 -07:00
fact-log-dual-write.test.ts feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law 2026-08-03 16:59:32 -07:00
fact-log-v2-cutover.test.ts feat(embedding): deferred-embed markers become log records — the sidecar recovery path is deleted 2026-08-10 11:27:07 -07:00
filter-operator-conformance.test.ts feat(contract): declare contract 1, serve three operators, refuse four by name 2026-08-28 10:57:43 -07:00
find-limits.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
find-matchall-cold.test.ts feat(engine): the wiring wave — stamps ride every flush, provider generations, waitForIndexed, adopt-backfill, match-all serves 2026-08-10 10:55:11 -07:00
find-triple-composition.test.ts test(8.0): asOf() error-path spot-checks + find() triple-composition correctness + scale-bench harness 2026-06-15 11:55:26 -07:00
find-unified-integration.test.ts test(budgets): iron-honest wall-clock budgets — 3x the worst honest-iron measurement 2026-08-18 09:36:21 -07:00
find-where-zero.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
flush-watcher-event-driven.test.ts perf(idle): the flush-request watch is event-driven; the heartbeat is observability 2026-08-28 11:25:47 -07:00
fold-checkpoint-bound.test.ts test(fold-checkpoint): the ARM-AT-FLIP pin arms its crash instead of racing the pending-flush timer 2026-08-24 09:58:45 -07:00
get-relations-fix.test.ts test(8.0): close brains in afterEach (count-sync, get-relations teardown) 2026-06-16 13:18:14 -07:00
graph-audit.test.ts feat: brain.auditGraph() — read-only graph-truth audit 2026-07-17 16:34:45 -07:00
graphIndex-pagination.test.ts chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
health-gate.test.ts fix(reads): the read gate is per-family; a write carrying unchanged data never re-embeds 2026-08-26 13:55:11 -07:00
history-repacking.test.ts feat: two-tier history reads + the repacker + generationDigest — D1+D3 wired end-to-end 2026-07-19 16:26:10 -07:00
hnsw-rebuild.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
hybrid-search-vfs.test.ts feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API 2026-06-10 15:22:47 -07:00
id-normalization.test.ts feat(8.0): id-normalization (#18) + aggregation min/max delete-safety + RC-safe release 2026-06-20 14:46:40 -07:00
idle-costs-nothing.test.ts perf(idle): the flush-request watch is event-driven; the heartbeat is observability 2026-08-28 11:25:47 -07:00
ifabsent-upsert-blob-concurrency.test.ts feat: temporal VFS — file content joins the Model-B immutability model 2026-07-10 16:43:48 -07:00
ifrev-concurrent-cas.test.ts fix: ifRev CAS is atomic — the revision check now runs under the commit mutex 2026-07-08 14:53:34 -07:00
index-skips-unvectored.test.ts fix(hnsw): skip unvectored rows on rebuild; refuse empty vectors in the index 2026-08-27 13:29:11 -07:00
ledger-derivation-identity.test.ts fix(storage): a suspect count ledger heals itself, and counts.json is written atomically 2026-08-28 10:28:25 -07:00
lens-consistency.test.ts feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law 2026-08-03 16:59:32 -07:00
level-field-shadow.test.ts feat(namespace): aggregation reads under the law + epoch 3 (the key-split rebuild) + THE ARMING COMMIT — the capability constant, the law module, and the typed refusals export from the package root; both engines' conformance suites light on this signal 2026-08-03 15:53:13 -07:00
log-authority-adopt.test.ts feat(log): log authority is the fleet default — adopt-at-open, oracle-gated; plus the power-cut throw-site cures and the loud torn-record contract 2026-08-11 08:37:38 -07:00
log-authority.test.ts feat(log): log authority is the fleet default — adopt-at-open, oracle-gated; plus the power-cut throw-site cures and the loud torn-record contract 2026-08-11 08:37:38 -07:00
memory-enhancements-v5.11.0.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
metadata-online-rebuild.test.ts feat(recovery): the catchup verdict is consumed; verb rows go live; the metadata rebuild goes online 2026-08-25 10:01:56 -07:00
metadata-only-comprehensive.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
metadata-vector-exclusion.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
migration-7x-to-8x.test.ts fix: 7→8 migration preserves branch-scoped non-entity state instead of deleting it 2026-07-07 16:07:10 -07:00
migration-backup.test.ts fix(8.0): byte-copy _id_mapper/* in the pre-upgrade backup (cor's write-new nuance) 2026-07-02 13:35:43 -07:00
migration.test.ts feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law 2026-08-03 16:59:32 -07:00
multi-process-safety.test.ts fix: release drains in-flight writer-lock heartbeat — no phantom lock after unlink 2026-07-19 12:52:24 -07:00
null-metadata-delete.test.ts fix(delete): the null-metadata skip closes — index legs run id-keyed or narrate, never silently strand postings 2026-08-20 11:53:58 -07:00
onchange-feed.test.ts feat: brain.onChange — the in-process change feed for every committed mutation 2026-07-10 11:24:31 -07:00
open-does-not-wait-for-a-rebuilding-provider.test.ts fix(open): a provider rebuilding itself is a third state, not a CRITICAL 2026-08-28 10:50:26 -07:00
open-narration.test.ts feat(open): the open narrates itself, on a channel production cannot clamp 2026-08-28 10:19:55 -07:00
orderby-sort-bug.test.ts feat(namespace): NO SPECIAL NAMES + storage fidelity — the ruled completion of the field-addressing law 2026-08-03 16:59:32 -07:00
read-gate-scope-and-no-reembed.test.ts fix(reads): the read gate is per-family; a write carrying unchanged data never re-embeds 2026-08-26 13:55:11 -07:00
read-surface-readiness.test.ts fix(reads): the readiness gate guards every index read surface — serving empty from a not-ready provider is unrepresentable 2026-08-20 11:49:03 -07:00
read-your-writes-contract.test.ts test: pin the read-your-writes contract under the single writer 2026-07-12 18:24:31 -07:00
readAfterWrite.test.ts fix(storage): an unknown nested storage config can never silently land on the shared default root 2026-08-25 10:47:51 -07:00
readdir-no-duplicate-replace.test.ts fix: full-removal canonical deletes + family-scoped migration gate 2026-07-14 10:11:53 -07:00
recovery-walk-tolerance.test.ts fix(recovery): walks are healers — the typed/tolerant boundary redrawn where block-layer fault injection proved it belonged 2026-08-11 09:20:30 -07:00
relationship-intelligence.test.ts chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
remaining-apis.test.ts test(budgets): iron-honest wall-clock budgets — 3x the worst honest-iron measurement 2026-08-18 09:36:21 -07:00
repair-narration.test.ts feat(repair): repairIndex narrates every phase and its receipt carries the walls 2026-08-28 10:31:42 -07:00
repair-report.test.ts feat(repair): a heal:'repair' verdict routes to the provider's own incremental repair() 2026-08-25 10:47:51 -07:00
reprojection-doors-open.test.ts feat(reprojection): the one doors-open machinery — budget-capped, yielding, foreground-preempted, atomic-swap; poison records quarantine typed 2026-08-10 11:39:27 -07:00
reserved-root-mint.test.ts fix(adoption): the reserved-root mint exemption — int 0 is legitimate for exactly one id 2026-08-12 08:55:12 -07:00
restore-nondestructive.test.ts fix: non-destructive, crash-resumable restore 2026-07-12 09:10:35 -07:00
rev-and-ifabsent.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
rollback-trapdoor.test.ts fix: honest response when a transaction rollback cannot complete 2026-07-12 12:19:09 -07:00
savebinaryblob-concurrent-rename.test.ts fix: saveBinaryBlob unique tmp suffix + ENOENT swallow on rename 2026-06-09 10:36:02 -07:00
smart-import.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
storage-batch-operations.test.ts feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API 2026-06-10 15:22:47 -07:00
strict-mode-self-test.test.ts feat(8.0): brain.fillSubtypes migration helper + pre-RC1 gap closure 2026-06-11 10:53:55 -07:00
subtype-and-facets.test.ts chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
sync-fail-compensation.test.ts fix(log): pad-frame construction is total; the at-ack sync-failure compensation splits by phase — a production adoption's two write-path defects, cured at their roots 2026-08-12 16:09:48 -07:00
temporal-vfs.test.ts feat: temporal VFS — file content joins the Model-B immutability model 2026-07-10 16:43:48 -07:00
transact-durability-barrier.test.ts feat(log): log authority is the fleet default — adopt-at-open, oracle-gated; plus the power-cut throw-site cures and the loud torn-record contract 2026-08-11 08:37:38 -07:00
transact-forward-ref-graph.test.ts fix: transact forward references resolve graph endpoint ints at execute time 2026-07-10 18:06:37 -07:00
txlog-origin-and-reconcile.test.ts feat(log): system commits carry their origin; the attested per-id reconcile door 2026-08-17 16:21:25 -07:00
update-asymmetry.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
update-write-granularity.test.ts fix: metadata-only update() never rewrites the noun record — the unconditional whole-vector save turned per-entity stat touches into full rewrites+fsync, amplifying read-heavy sweeps into disk saturation on a production deployment 2026-07-29 10:42:50 -07:00
vector-leg-open-build.test.ts fix(vfs): the VFS root never persists a zero-norm vector 2026-08-27 09:28:44 -07:00
vector-recall.test.ts test(8.0): A/B benchmark harness (open leg) — generic corpus+metrics lib, brainy-alone scaling bench, boundary guard, real-embedding recall guard 2026-06-15 15:51:17 -07:00
verb-metadata-rows.test.ts fix(update-seam): the metadata crossing never carries BigInt endpoint ints 2026-08-25 12:07:28 -07:00
verb-subtype-and-enforcement.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
vfs-and-graph-entities.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
vfs-api-wiring.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
vfs-cow-blob-adoption.test.ts fix: recover VFS content blobs stranded by a 7→8 upgrade, in place on open 2026-07-07 12:23:36 -07:00
vfs-debug.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
vfs-import-verification.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
vfs-knowledge-separation.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
vfs-performance-v5.11.1.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
vfs-rename-containment.test.ts fix: VFS rename moves the containment edge — no ghost in the old directory 2026-07-15 09:25:21 -07:00
vfs-root-sweep-once.test.ts perf(vfs): the old-root sweep runs once per store, not once per open 2026-08-28 11:01:43 -07:00
vfs-root-zero-norm.test.ts fix(vectors): a zero-norm vector is not a vector, canonical side included, plus the sanctioned unvector door 2026-08-27 13:53:09 -07:00
wait-for-indexed.test.ts feat(engine): the wiring wave — stamps ride every flush, provider generations, waitForIndexed, adopt-backfill, match-all serves 2026-08-10 10:55:11 -07:00
wasm-embeddings.test.ts feat: migrate embeddings to Candle WASM + remove semantic type inference 2026-01-06 12:52:34 -08:00
watermark-adopt-reopen.test.ts feat(index): watermark stamps on every TS projection — adopt/catchup/rescan verdicts at load, stamp-after-data 2026-08-10 10:55:11 -07:00
write-flow-production-shape.test.ts feat(recovery): the fold-checkpoint bound — crash folds (checkpoint, head], never the whole log twice 2026-08-12 16:56:08 -07:00
writer-lock-clean-close.test.ts fix(storage): a clean close is recorded, and the writer lock is always given up 2026-08-28 10:17:20 -07:00
writer-lock-fencing.test.ts fix(locks): the fence keys ownership on pid+hostname — a same-process re-open never fences its predecessor 2026-08-18 10:11:30 -07:00
zero-norm-unvector-door.test.ts fix(vectors): a zero-norm vector is not a vector, canonical side included, plus the sanctioned unvector door 2026-08-27 13:53:09 -07:00