feat: per-entity _rev + update({ ifRev }) CAS + add({ ifAbsent })
Optimistic concurrency for multi-writer coordination — the read-then-CAS
lock pattern + idempotent-bootstrap singleton inserts. Lands the surface the
SDK scheduler asked for in BRAINY-EXPOSE-TRANSACTIONS, scoped to what ships
cleanly without painting the 8.0 transact() API into a corner.
A. PER-ENTITY _rev FIELD
- src/coreTypes.ts — HNSWNounWithMetadata gains optional `_rev?: number`;
STANDARD_ENTITY_FIELDS includes it so resolveEntityField routes correctly.
- src/types/brainy.types.ts — Entity<T> gains optional `_rev?: number`;
Result<T> mirrors it on the convenience flatten layer.
- src/brainy.ts add() — initializes `_rev: 1` in storageMetadata. New entities
always start at 1.
- src/brainy.ts update() — reads currentRev off the persisted metadata
(falls back to 1 for pre-7.31.0 entities with no _rev), writes
`_rev: currentRev + 1` into the updated metadata. Every successful update()
bumps by exactly 1.
- src/brainy.ts convertNounToEntity + convertMetadataToEntity — both pull
_rev out of the storage metadata and surface it at the top level of Entity.
Pre-7.31.0 entities (no _rev in storage) read as `_rev: 1` so consumers
see a consistent value.
- src/storage/baseStorage.ts — six destructure sites updated to pull _rev
out of the metadata bag so it doesn't leak into customMetadata. Noun-side
returns surface _rev; verb-side destructures correctly but doesn't expose
it on HNSWVerbWithMetadata (verb CAS is future work).
- src/brainy.ts createResult() — flattens entity._rev onto the Result for
backward compat with the existing convenience-field layer.
B. update({ ifRev }) OPTIMISTIC CONCURRENCY
- src/types/brainy.types.ts — UpdateParams<T> gains optional `ifRev?: number`.
- src/transaction/RevisionConflictError.ts (NEW) — carries `{ id, expected,
actual }`. Message names the recipe: refetch with brain.get() and retry
with the latest _rev. Subclass of Error.
- src/brainy.ts update() — when params.ifRev is provided, compares against
currentRev (the rev we just read) and throws RevisionConflictError on
mismatch before any storage write. Omitting ifRev keeps the prior
unconditional-update behavior; existing consumers see no change.
- src/transaction/index.ts — exports RevisionConflictError.
- src/index.ts — public export of RevisionConflictError.
C. add({ ifAbsent }) BY-ID IDEMPOTENT INSERT
- src/types/brainy.types.ts — AddParams<T> gains optional `ifAbsent?: boolean`;
AddManyParams<T> mirrors it as a batch-level flag.
- src/brainy.ts add() — when params.id AND params.ifAbsent, pre-reads
storage.getNounMetadata(id); if present, returns the existing id without
writing. No throw, no overwrite. Ignored when id is omitted (a fresh UUID
can never collide).
- src/brainy.ts addMany() — propagates the batch-level ifAbsent to each
item's add() call. Per-item ifAbsent takes precedence so callers can
override individual rows.
WHAT'S NOT SHIPPED (and why)
A public brain.transaction(fn) wrapper was on the table but was cut. The
internal TransactionManager exposes raw Operation classes (SaveNounMetadata,
etc.) that take StorageAdapter as a constructor argument. A clean high-level
facade in 7.31.0 would have meant either:
(a) Delegate to brain.add() / update() / relate(). Each of those opens its
own internal transaction and commits before the closure returns. Looks
atomic, isn't — a footgun.
(b) Thread an optional `tx?` through every internal write site in
brainy.ts (8 transactionManager.executeTransaction sites). ~2 days of
real refactor with regression surface, and the API shape changes again
in 8.0 anyway.
The SDK scheduler's actual ask (BRAINY-EXPOSE-TRANSACTIONS) is the
read-then-CAS lock pattern. _rev + ifRev solves it completely. Multi-write
atomicity is the 8.0 brain.transact() use case where the Datomic-style
immutable Db makes it atomic by construction — that's the right home.
TESTS
- New tests/integration/rev-and-ifabsent.test.ts (18 tests):
- _rev initialization (1 on add) and surface on get fast/full paths + find
- _rev auto-bump on update across multiple writes
- update({ ifRev }) pass / fail / message format / omitted / legacy-no-rev
- add({ ifAbsent }) writes when absent / no-op when present / id-required
- addMany({ ifAbsent }) propagation + per-item override
- SDK-scheduler scenario: two concurrent CAS updates, one wins one throws
- Unit suite unchanged: 1468/1468.
- All integration subtype + verb + strict + find-limits + new rev suites
pass: 96/96.
DOCS
- New docs/guides/optimistic-concurrency.md (public: true) — full reference:
the lock pattern, read-modify-write retry, idempotent bootstrap, how _rev
interacts with brain.versions, branches/fork, and VFS, what's coming in 8.0.
- docs/api/README.md — add() and update() entries get the new params + tips
pointing at the new guide.
- RELEASES.md v7.31.0 entry.
CORTEX COMPATIBILITY
Zero changes required. _rev is a metadata column already supported by
NativeColumnStore. The auto-bump runs in Brainy JS before any storage call;
Cortex never sees the per-entity counter.
8.0 FORWARD-COMPAT
_rev, ifRev, RevisionConflictError, and ifAbsent survive the 8.0 Db redesign
unchanged. 8.0 layers brain.transact(tx, { ifAtGeneration }) for whole-tx CAS
on top of the same per-entity mechanism — per-entity for single-record
patterns (job locks, idempotent state machines), generation-based for
"did the world move under me." Locked in .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md § C-6.
Verification
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit
- All integration suites including new rev-and-ifabsent: 96/96
- npm run build: clean
- Closed-source product reference audit: clean
This commit is contained in:
parent
ccc1d74953
commit
bafb4e4caa
11 changed files with 726 additions and 9 deletions
175
docs/guides/optimistic-concurrency.md
Normal file
175
docs/guides/optimistic-concurrency.md
Normal file
|
|
@ -0,0 +1,175 @@
|
|||
---
|
||||
title: Optimistic concurrency with _rev
|
||||
slug: guides/optimistic-concurrency
|
||||
public: true
|
||||
category: guides
|
||||
template: guide
|
||||
order: 8
|
||||
description: Use the per-entity `_rev` counter and `update({ ifRev })` to coordinate concurrent writes safely. Covers the lock pattern, idempotent inserts with `ifAbsent`, and recovery on conflict.
|
||||
next:
|
||||
- guides/find-limits
|
||||
- api/README
|
||||
---
|
||||
|
||||
# Optimistic concurrency with `_rev`
|
||||
|
||||
Brainy 7.31.0 adds a per-entity revision counter so multiple writers can coordinate without a global lock or external coordinator. The pattern is the same one CouchDB, PouchDB, and ETag-based HTTP caches use: read the current revision, do your work, write back with `ifRev: <whatRevYouSaw>`. If the revision moved, your write is rejected and you retry against the latest state.
|
||||
|
||||
## What gets added
|
||||
|
||||
| Surface | Behavior |
|
||||
|---|---|
|
||||
| `entity._rev: number` | Returned on every `get()`, `find()`, `search()`. Initialized to `1` on `add()`. Bumped by `1` on every successful `update()`. Pre-7.31.0 entities without `_rev` are surfaced as `1`. |
|
||||
| `update({ id, ..., ifRev: number })` | If the persisted `_rev` does not equal `ifRev`, throws `RevisionConflictError`. Omitting `ifRev` keeps the prior (unconditional) update behavior. |
|
||||
| `RevisionConflictError` | Carries `{ id, expected, actual }` for principled recovery. |
|
||||
| `add({ id, ifAbsent: true })` | By-ID idempotent insert. Returns the existing `id` if one is already present; no throw, no overwrite. |
|
||||
| `addMany({ items, ifAbsent: true })` | Applies `ifAbsent` to every item. Per-item `ifAbsent` overrides the batch flag. |
|
||||
|
||||
`_rev` is independent of `brain.versions.*` (named snapshots), of `brain.fork()` / branches (COW), and of VFS file versioning. They all coexist; `_rev` is the per-write counter used for CAS.
|
||||
|
||||
## The lock pattern
|
||||
|
||||
Every distributed-job scheduler eventually wants this exact loop:
|
||||
|
||||
```ts
|
||||
import { Brainy, RevisionConflictError } from '@soulcraft/brainy'
|
||||
|
||||
const LOCK_ID = '...uuid for this job slot...'
|
||||
|
||||
// Bootstrap the lock document once (idempotent).
|
||||
await brain.add({
|
||||
id: LOCK_ID,
|
||||
type: NounType.Document,
|
||||
data: { owner: null, expiresAt: 0 },
|
||||
ifAbsent: true
|
||||
})
|
||||
|
||||
async function tryAcquireLock(workerId: string, ttlMs: number) {
|
||||
const lock = await brain.get(LOCK_ID)
|
||||
if (!lock) throw new Error('lock document missing')
|
||||
|
||||
const state = lock.data as { owner: string | null; expiresAt: number }
|
||||
const now = Date.now()
|
||||
|
||||
// Only take the lock if it's free or expired.
|
||||
if (state.owner && state.expiresAt > now) return false
|
||||
|
||||
try {
|
||||
await brain.update({
|
||||
id: LOCK_ID,
|
||||
data: { owner: workerId, expiresAt: now + ttlMs },
|
||||
ifRev: lock._rev
|
||||
})
|
||||
return true
|
||||
} catch (err) {
|
||||
if (err instanceof RevisionConflictError) {
|
||||
// Another worker grabbed it between our read and write.
|
||||
return false
|
||||
}
|
||||
throw err
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
No external lock service, no Redis SETNX, no Cloud Tasks. The CAS check is the lock.
|
||||
|
||||
## Read-modify-write with retry
|
||||
|
||||
The other common shape is "update a counter / config object" with bounded retries on conflict:
|
||||
|
||||
```ts
|
||||
async function incrementCounter(id: string, by: number) {
|
||||
for (let attempt = 0; attempt < 5; attempt++) {
|
||||
const entity = await brain.get(id)
|
||||
if (!entity) throw new Error('counter does not exist')
|
||||
const current = (entity.data as { value: number }).value
|
||||
try {
|
||||
await brain.update({
|
||||
id,
|
||||
data: { value: current + by },
|
||||
ifRev: entity._rev
|
||||
})
|
||||
return
|
||||
} catch (err) {
|
||||
if (err instanceof RevisionConflictError) continue // refetch + retry
|
||||
throw err
|
||||
}
|
||||
}
|
||||
throw new Error('counter update conflict after 5 attempts')
|
||||
}
|
||||
```
|
||||
|
||||
The retry bound matters — without one, two unlucky writers can ping-pong forever.
|
||||
|
||||
## Idempotent bootstrap with `ifAbsent`
|
||||
|
||||
For singletons (config rows, well-known seed entities, job-state documents) where the natural ID is deterministic:
|
||||
|
||||
```ts
|
||||
await brain.add({
|
||||
id: 'config:singleton',
|
||||
type: NounType.Document,
|
||||
data: { tenantQuota: 1000 },
|
||||
ifAbsent: true
|
||||
})
|
||||
```
|
||||
|
||||
- First caller writes; gets back `'config:singleton'`.
|
||||
- Every subsequent caller short-circuits at the pre-read; gets back the same `'config:singleton'` without touching the existing entity.
|
||||
- `_rev` is **not** bumped on the no-op path (no write happened).
|
||||
|
||||
`ifAbsent` is only meaningful when you supply an `id`. With no `id`, Brainy generates a fresh UUID that can never collide, so the flag is silently ignored.
|
||||
|
||||
`addMany({ items, ifAbsent: true })` applies the flag to every item. Mixing per-item overrides with the batch flag works as you'd expect: per-item `ifAbsent: false` opts an individual row out, per-item `ifAbsent: true` opts it in.
|
||||
|
||||
### Why no `addIfMissing({ match, add })` for attribute-based dedup?
|
||||
|
||||
You may want "create if no entity with this email exists" (lookup by attribute, not by ID). That's a different operation:
|
||||
|
||||
```ts
|
||||
// What we DID NOT ship in 7.31.0 — the attribute-based variant.
|
||||
await brain.addIfMissing({ // ← not a real API
|
||||
match: { type: 'Person', where: { email: 'x@y.com' } },
|
||||
add: { data: '...', metadata: { email: 'x@y.com' } }
|
||||
})
|
||||
```
|
||||
|
||||
It's race-prone outside a transaction: two concurrent imports both see "not found," both insert, you get duplicates. Without a unique-index primitive (which Brainy doesn't have today), this pattern needs to live inside a transaction:
|
||||
|
||||
```ts
|
||||
// The race-safe shape, available once brain.transact() ships in 8.0.
|
||||
await brain.transact(async tx => {
|
||||
const existing = await tx.find({
|
||||
type: 'Person',
|
||||
where: { email: 'x@y.com' },
|
||||
limit: 1
|
||||
})
|
||||
if (existing.length === 0) {
|
||||
await tx.add({ type: 'Person', data: '...', metadata: { email: 'x@y.com' } })
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
For 7.31.0, lean on `ifAbsent` when you control the ID, and accept the inherent dedup race when you don't. The 8.0 `brain.transact()` is the natural home for the attribute-based variant.
|
||||
|
||||
## How `_rev` interacts with the other versioning systems
|
||||
|
||||
Brainy has several persistence primitives that all touch the word "version" in different ways. `_rev` is independent of every one of them:
|
||||
|
||||
| System | What it tracks | When it advances |
|
||||
|---|---|---|
|
||||
| **`_rev`** (new in 7.31.0) | Per-entity write counter | Auto, on every successful `update()` |
|
||||
| **`brain.versions.save()`** | Named snapshots per entity | Explicit — you call `save()` with a tag |
|
||||
| **`brain.fork()` / branches** | Whole-brain copy-on-write | Explicit — you call `fork(name)` |
|
||||
| **VFS file versioning** | Per-VFS-file snapshots (Document entity) | Same as `brain.versions.save()` |
|
||||
|
||||
If you're on a branch, each branch has its own copy of every entity (that's what COW means), so each branch has its own `_rev` per entity — same as every other field. Snapshots taken via `brain.versions.save()` capture the entity at that moment including its `_rev` at that time; the snapshot's own `version: number` is the snapshot index, distinct from `_rev`.
|
||||
|
||||
## What's coming in 8.0
|
||||
|
||||
Brainy 8.0 ships a Datomic-style immutable `Db` API where `brain.transact(tx)` is the public multi-write atomicity primitive. Two things change for code written against 7.31.0's surface:
|
||||
|
||||
- `_rev` survives unchanged. It's part of the 8.0 entity shape; the auto-bump moves to the new `transact()` write path. Code that uses `update({ ifRev })` keeps working.
|
||||
- A new `brain.transact(tx, { ifAtGeneration: prev.generation })` adds generation-based CAS for whole-transaction atomicity, layered on top of `_rev`. Per-entity for single-record patterns (the lock above), generation-based for "did the world move under me."
|
||||
|
||||
If you adopt `_rev` + `ifRev` in 7.31.0, no migration work is needed for 8.0.
|
||||
Loading…
Add table
Add a link
Reference in a new issue