brainy/tests/integration
David Snelling 867939ed50 fix: atomic ifAbsent/upsert inserts + exact blob reference counts under concurrency
The fast-follow to the ifRev CAS fix: the sweep for the same check-then-act
class found two more instances, both now closed with the same discipline —
the decision runs at the serialization point that guards the apply.

add({ifAbsent}) / add({upsert}): the absence check ran before the commit
mutex, so N concurrent same-id creates could all pass it and all write — the
second silently overwriting the first, violating ifAbsent's "return the
existing id WITHOUT writing" contract and upsert's merge-never-clobber
contract. The insert leg now carries a must-be-absent commit precondition
(the same conditional-commit primitive ifRev uses), verified under the commit
mutex against the authoritative before-image. A losing caller takes its
documented resolution instead of overwriting: ifAbsent returns the existing
id with zero writes; upsert merges into the now-existing entity via update()
(shared param mapping in upsertMergeParams so the planning-time branch and
the conflict retry can never drift), with a bounded retry if a concurrent
delete lands between the conflict and the merge. The planning-time checks
remain as fast-fails. transact()'s batch-level ifAbsent/upsert keep
planning-time semantics (documented, converges to a valid entity).

BlobStorage: write()'s dedup decision (exists → increment refCount, absent →
create at 1) and delete()'s decrement-then-remove were unserialized
read-modify-writes over blob-meta:<hash>. Concurrent writes of identical
content could lose references, so a later delete removed bytes another file
still referenced (data loss), or leaked unreferenced blobs. All
reference-count-bearing mutations now serialize through a per-hash
InMemoryMutex — distinct content never contends; incrementRefCount/
decrementRefCount are documented lock-assumed internals.

Both fixes are complete by construction in-process: storage enforces
single-writer-per-directory, so the process is the whole concurrency domain.

Tests (tests/integration/ifabsent-upsert-blob-concurrency.test.ts): 8-way
ifAbsent storm advances the store by exactly one generation with _rev 1;
8-way upsert storm on an absent id yields one create + seven merges
(_rev === 8); sequential ifAbsent/upsert semantics pinned unchanged; N
identical concurrent blob writes → refCount === N; the blob survives until
the true last reference drops; interleaved write/delete storm never loses a
landed reference.
2026-07-08 15:13:26 -07:00
..
advanced-apis-regression.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
aggregation-delete.test.ts feat(8.0): id-normalization (#18) + aggregation min/max delete-safety + RC-safe release 2026-06-20 14:46:40 -07:00
aggregation.test.ts chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
all-apis-comprehensive.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
api-parameter-validation.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
batchImportWithRelations.test.ts feat(8.0): reserved-field enforcement — reservedFieldPolicy defaults to throw 2026-06-20 15:37:21 -07:00
brainy-complete.integration.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
brainy-core.integration.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
brainy-phase1c-integration.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
bun-runtime-test.ts chore(8.0): modernize toolchain + position Bun as a runtime 2026-07-01 09:16:46 -07:00
clear-persistence.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
clear-vfs.test.ts docs(8.0): consistency-model concept + snapshots guide — Db API replaces branching docs 2026-06-11 08:37:26 -07:00
cold-graph-connected-8.0.test.ts fix(8.0): never serve a silent [] from find({connected}) on a cold-loaded graph 2026-06-30 09:30:11 -07:00
cortex-compat.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
count-synchronization.test.ts test(8.0): close brains in afterEach (count-sync, get-relations teardown) 2026-06-16 13:18:14 -07:00
db-asof-vector-defer.test.ts feat(8.0): #35 part-3 — supply at-gen candidate vectors for the native exact-rerank 2026-06-23 16:50:41 -07:00
db-mvcc.test.ts fix(8.0): restore() reloads a native entity-id mapper before graphIndex.rebuild() 2026-06-23 12:02:17 -07:00
db-temporal.test.ts refactor(8.0): API-surface + quality polish from the readiness audit 2026-06-29 10:04:19 -07:00
entity-confidence-weight.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
find-limits.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
find-triple-composition.test.ts test(8.0): asOf() error-path spot-checks + find() triple-composition correctness + scale-bench harness 2026-06-15 11:55:26 -07:00
find-unified-integration.test.ts feat(8.0): id-normalization (#18) + aggregation min/max delete-safety + RC-safe release 2026-06-20 14:46:40 -07:00
find-where-zero.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
get-relations-fix.test.ts test(8.0): close brains in afterEach (count-sync, get-relations teardown) 2026-06-16 13:18:14 -07:00
graphIndex-pagination.test.ts chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
hnsw-rebuild.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
hybrid-search-vfs.test.ts feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API 2026-06-10 15:22:47 -07:00
id-normalization.test.ts feat(8.0): id-normalization (#18) + aggregation min/max delete-safety + RC-safe release 2026-06-20 14:46:40 -07:00
ifabsent-upsert-blob-concurrency.test.ts fix: atomic ifAbsent/upsert inserts + exact blob reference counts under concurrency 2026-07-08 15:13:26 -07:00
ifrev-concurrent-cas.test.ts fix: ifRev CAS is atomic — the revision check now runs under the commit mutex 2026-07-08 14:53:34 -07:00
memory-enhancements-v5.11.0.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
metadata-only-comprehensive.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
metadata-vector-exclusion.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
migration-7x-to-8x.test.ts fix: 7→8 migration preserves branch-scoped non-entity state instead of deleting it 2026-07-07 16:07:10 -07:00
migration-backup.test.ts fix(8.0): byte-copy _id_mapper/* in the pre-upgrade backup (cor's write-new nuance) 2026-07-02 13:35:43 -07:00
migration.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
multi-process-safety.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
orderby-sort-bug.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
readAfterWrite.test.ts chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
relationship-intelligence.test.ts chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
remaining-apis.test.ts feat(8.0): reserved-field enforcement — reservedFieldPolicy defaults to throw 2026-06-20 15:37:21 -07:00
rev-and-ifabsent.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
savebinaryblob-concurrent-rename.test.ts fix: saveBinaryBlob unique tmp suffix + ENOENT swallow on rename 2026-06-09 10:36:02 -07:00
smart-import.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
storage-batch-operations.test.ts feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API 2026-06-10 15:22:47 -07:00
strict-mode-self-test.test.ts feat(8.0): brain.fillSubtypes migration helper + pre-RC1 gap closure 2026-06-11 10:53:55 -07:00
subtype-and-facets.test.ts chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
update-asymmetry.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
vector-recall.test.ts test(8.0): A/B benchmark harness (open leg) — generic corpus+metrics lib, brainy-alone scaling bench, boundary guard, real-embedding recall guard 2026-06-15 15:51:17 -07:00
verb-subtype-and-enforcement.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
vfs-and-graph-entities.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
vfs-api-wiring.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
vfs-cow-blob-adoption.test.ts fix: recover VFS content blobs stranded by a 7→8 upgrade, in place on open 2026-07-07 12:23:36 -07:00
vfs-debug.test.ts feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1) 2026-06-09 14:58:25 -07:00
vfs-import-verification.test.ts test(8.0): integration rot pass — 77→17 failures (parallel per-file hardening) 2026-06-17 13:11:41 -07:00
vfs-knowledge-separation.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
vfs-performance-v5.11.1.test.ts feat(8.0): API simplification — remove neural()/Db.search, one storage path key, integration→0 2026-06-20 13:31:11 -07:00
wasm-embeddings.test.ts feat: migrate embeddings to Candle WASM + remove semantic type inference 2026-01-06 12:52:34 -08:00