feat: per-entity _rev + update({ ifRev }) CAS + add({ ifAbsent })

Optimistic concurrency for multi-writer coordination — the read-then-CAS lock pattern + idempotent-bootstrap singleton inserts. Lands the surface the SDK scheduler asked for in BRAINY-EXPOSE-TRANSACTIONS, scoped to what ships cleanly without painting the 8.0 transact() API into a corner. A. PER-ENTITY _rev FIELD - src/coreTypes.ts — HNSWNounWithMetadata gains optional `_rev?: number`; STANDARD_ENTITY_FIELDS includes it so resolveEntityField routes correctly. - src/types/brainy.types.ts — Entity<T> gains optional `_rev?: number`; Result<T> mirrors it on the convenience flatten layer. - src/brainy.ts add() — initializes `_rev: 1` in storageMetadata. New entities always start at 1. - src/brainy.ts update() — reads currentRev off the persisted metadata (falls back to 1 for pre-7.31.0 entities with no _rev), writes `_rev: currentRev + 1` into the updated metadata. Every successful update() bumps by exactly 1. - src/brainy.ts convertNounToEntity + convertMetadataToEntity — both pull _rev out of the storage metadata and surface it at the top level of Entity. Pre-7.31.0 entities (no _rev in storage) read as `_rev: 1` so consumers see a consistent value. - src/storage/baseStorage.ts — six destructure sites updated to pull _rev out of the metadata bag so it doesn't leak into customMetadata. Noun-side returns surface _rev; verb-side destructures correctly but doesn't expose it on HNSWVerbWithMetadata (verb CAS is future work). - src/brainy.ts createResult() — flattens entity._rev onto the Result for backward compat with the existing convenience-field layer. B. update({ ifRev }) OPTIMISTIC CONCURRENCY - src/types/brainy.types.ts — UpdateParams<T> gains optional `ifRev?: number`. - src/transaction/RevisionConflictError.ts (NEW) — carries `{ id, expected, actual }`. Message names the recipe: refetch with brain.get() and retry with the latest _rev. Subclass of Error. - src/brainy.ts update() — when params.ifRev is provided, compares against currentRev (the rev we just read) and throws RevisionConflictError on mismatch before any storage write. Omitting ifRev keeps the prior unconditional-update behavior; existing consumers see no change. - src/transaction/index.ts — exports RevisionConflictError. - src/index.ts — public export of RevisionConflictError. C. add({ ifAbsent }) BY-ID IDEMPOTENT INSERT - src/types/brainy.types.ts — AddParams<T> gains optional `ifAbsent?: boolean`; AddManyParams<T> mirrors it as a batch-level flag. - src/brainy.ts add() — when params.id AND params.ifAbsent, pre-reads storage.getNounMetadata(id); if present, returns the existing id without writing. No throw, no overwrite. Ignored when id is omitted (a fresh UUID can never collide). - src/brainy.ts addMany() — propagates the batch-level ifAbsent to each item's add() call. Per-item ifAbsent takes precedence so callers can override individual rows. WHAT'S NOT SHIPPED (and why) A public brain.transaction(fn) wrapper was on the table but was cut. The internal TransactionManager exposes raw Operation classes (SaveNounMetadata, etc.) that take StorageAdapter as a constructor argument. A clean high-level facade in 7.31.0 would have meant either: (a) Delegate to brain.add() / update() / relate(). Each of those opens its own internal transaction and commits before the closure returns. Looks atomic, isn't — a footgun. (b) Thread an optional `tx?` through every internal write site in brainy.ts (8 transactionManager.executeTransaction sites). ~2 days of real refactor with regression surface, and the API shape changes again in 8.0 anyway. The SDK scheduler's actual ask (BRAINY-EXPOSE-TRANSACTIONS) is the read-then-CAS lock pattern. _rev + ifRev solves it completely. Multi-write atomicity is the 8.0 brain.transact() use case where the Datomic-style immutable Db makes it atomic by construction — that's the right home. TESTS - New tests/integration/rev-and-ifabsent.test.ts (18 tests): - _rev initialization (1 on add) and surface on get fast/full paths + find - _rev auto-bump on update across multiple writes - update({ ifRev }) pass / fail / message format / omitted / legacy-no-rev - add({ ifAbsent }) writes when absent / no-op when present / id-required - addMany({ ifAbsent }) propagation + per-item override - SDK-scheduler scenario: two concurrent CAS updates, one wins one throws - Unit suite unchanged: 1468/1468. - All integration subtype + verb + strict + find-limits + new rev suites pass: 96/96. DOCS - New docs/guides/optimistic-concurrency.md (public: true) — full reference: the lock pattern, read-modify-write retry, idempotent bootstrap, how _rev interacts with brain.versions, branches/fork, and VFS, what's coming in 8.0. - docs/api/README.md — add() and update() entries get the new params + tips pointing at the new guide. - RELEASES.md v7.31.0 entry. CORTEX COMPATIBILITY Zero changes required. _rev is a metadata column already supported by NativeColumnStore. The auto-bump runs in Brainy JS before any storage call; Cortex never sees the per-entity counter. 8.0 FORWARD-COMPAT _rev, ifRev, RevisionConflictError, and ifAbsent survive the 8.0 Db redesign unchanged. 8.0 layers brain.transact(tx, { ifAtGeneration }) for whole-tx CAS on top of the same per-entity mechanism — per-entity for single-record patterns (job locks, idempotent state machines), generation-based for "did the world move under me." Locked in .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md § C-6. Verification - npx tsc --noEmit: clean - npm test: 1468 / 1468 unit - All integration suites including new rev-and-ifabsent: 96/96 - npm run build: clean - Closed-source product reference audit: clean
2026-06-09 10:04:24 -07:00 · 2026-06-09 10:04:24 -07:00 · bafb4e4caa
commit bafb4e4caa
parent ccc1d74953
11 changed files with 726 additions and 9 deletions
--- a/tests/integration/rev-and-ifabsent.test.ts
+++ b/tests/integration/rev-and-ifabsent.test.ts
@ -0,0 +1,291 @@
+/**
+ * @module tests/integration/rev-and-ifabsent
+ * @description End-to-end tests for the 7.31.0 optimistic-concurrency surface:
+ *   - `_rev` is initialized to 1 on add() and surfaced on get()/find()/search()
+ *   - update() auto-bumps `_rev`
+ *   - update({ ifRev }) accepts a matching rev and rejects a stale one with
+ *     RevisionConflictError carrying { id, expected, actual }
+ *   - add({ ifAbsent: true }) is a no-op when the id is already present
+ *   - addMany({ ifAbsent: true }) applies the flag to every item
+ */
+
+import { describe, it, expect, beforeEach } from 'vitest'
+import { Brainy } from '../../src/brainy.js'
+import { RevisionConflictError } from '../../src/transaction/RevisionConflictError.js'
+import { NounType } from '../../src/types/graphTypes.js'
+
+describe('7.31.0 — _rev CAS + ifAbsent', () => {
+  let brain: Brainy
+
+  beforeEach(async () => {
+    brain = new Brainy({ storage: { type: 'memory' as const } })
+    await brain.init()
+  })
+
+  describe('_rev initialization + surface', () => {
+    it('initializes _rev to 1 on add()', async () => {
+      const id = await brain.add({ data: 'hello', type: NounType.Document })
+      const entity = await brain.get(id)
+      expect(entity?._rev).toBe(1)
+    })
+
+    it('surfaces _rev on metadata-only get() fast path', async () => {
+      const id = await brain.add({ data: 'fast-path', type: NounType.Document })
+      const entity = await brain.get(id)  // metadata-only by default
+      expect(entity?._rev).toBe(1)
+    })
+
+    it('surfaces _rev on includeVectors get() full path', async () => {
+      const id = await brain.add({ data: 'full-path', type: NounType.Document })
+      const entity = await brain.get(id, { includeVectors: true })
+      expect(entity?._rev).toBe(1)
+    })
+
+    it('surfaces _rev on find() results', async () => {
+      const id = await brain.add({ data: 'findable', type: NounType.Document })
+      const results = await brain.find({ type: NounType.Document })
+      const hit = results.find((r) => r.id === id)
+      expect(hit?._rev).toBe(1)
+    })
+  })
+
+  describe('_rev auto-bump on update()', () => {
+    it('bumps _rev to 2 after first update()', async () => {
+      const id = await brain.add({ data: 'v1', type: NounType.Document })
+      await brain.update({ id, data: 'v2' })
+      const entity = await brain.get(id)
+      expect(entity?._rev).toBe(2)
+    })
+
+    it('bumps _rev monotonically across multiple updates', async () => {
+      const id = await brain.add({ data: 'v1', type: NounType.Document })
+      for (let i = 2; i <= 5; i++) {
+        await brain.update({ id, data: `v${i}` })
+        const entity = await brain.get(id)
+        expect(entity?._rev).toBe(i)
+      }
+    })
+
+    it('bumps _rev on metadata-only update', async () => {
+      const id = await brain.add({
+        data: 'doc',
+        type: NounType.Document,
+        metadata: { status: 'draft' }
+      })
+      await brain.update({ id, metadata: { status: 'published' } })
+      const entity = await brain.get(id)
+      expect(entity?._rev).toBe(2)
+      expect(entity?.metadata?.status).toBe('published')
+    })
+  })
+
+  describe('update({ ifRev }) optimistic concurrency', () => {
+    it('passes when ifRev matches the persisted _rev', async () => {
+      const id = await brain.add({ data: 'v1', type: NounType.Document })
+      await expect(brain.update({ id, data: 'v2', ifRev: 1 })).resolves.not.toThrow()
+      const after = await brain.get(id)
+      expect(after?._rev).toBe(2)
+    })
+
+    it('throws RevisionConflictError when ifRev is stale', async () => {
+      const id = await brain.add({ data: 'v1', type: NounType.Document })
+      await brain.update({ id, data: 'v2' })  // bumps to 2
+
+      let caught: unknown = null
+      try {
+        await brain.update({ id, data: 'v3-from-stale-reader', ifRev: 1 })
+      } catch (err) {
+        caught = err
+      }
+      expect(caught).toBeInstanceOf(RevisionConflictError)
+      const e = caught as RevisionConflictError
+      expect(e.id).toBe(id)
+      expect(e.expected).toBe(1)
+      expect(e.actual).toBe(2)
+    })
+
+    it('emits a message that points at the recovery recipe', async () => {
+      const id = await brain.add({ data: 'v1', type: NounType.Document })
+      await brain.update({ id, data: 'v2' })
+      try {
+        await brain.update({ id, ifRev: 1, data: 'v3' })
+        throw new Error('should have thrown')
+      } catch (err) {
+        const msg = (err as Error).message
+        expect(msg).toMatch(/persisted _rev is 2/)
+        expect(msg).toMatch(/brain\.get/)
+      }
+    })
+
+    it('omits the check entirely when ifRev is not provided', async () => {
+      const id = await brain.add({ data: 'v1', type: NounType.Document })
+      await brain.update({ id, data: 'v2' })
+      await brain.update({ id, data: 'v3' })  // no ifRev — must succeed
+      const after = await brain.get(id)
+      expect(after?._rev).toBe(3)
+    })
+
+    it('treats pre-7.31.0 metadata with no _rev as rev 1', async () => {
+      // Simulate a legacy entity by writing directly through the storage adapter
+      // with no _rev field, then verifying ifRev: 1 succeeds.
+      const id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa'
+      const storage = (brain as any).storage
+      await storage.saveNounMetadata(id, {
+        data: 'pre-7.31',
+        noun: NounType.Document,
+        createdAt: Date.now(),
+        updatedAt: Date.now()
+        // intentionally no _rev
+      })
+      // Also save the vector so get() doesn't return null on the full path.
+      const vector = await (brain as any).embed('pre-7.31')
+      await storage.saveNoun({
+        id,
+        vector,
+        connections: new Map(),
+        level: 0
+      })
+      const legacy = await brain.get(id)
+      expect(legacy?._rev).toBe(1)
+      await expect(brain.update({ id, data: 'updated', ifRev: 1 })).resolves.not.toThrow()
+      const after = await brain.get(id)
+      expect(after?._rev).toBe(2)
+    })
+  })
+
+  describe('add({ ifAbsent: true })', () => {
+    it('writes when no entity with the id exists', async () => {
+      const id = await brain.add({
+        id: '11111111-1111-4111-8111-111111111111',
+        data: 'first-write',
+        type: NounType.Document,
+        ifAbsent: true
+      })
+      expect(id).toBe('11111111-1111-4111-8111-111111111111')
+      const entity = await brain.get(id)
+      expect(entity?.data).toBe('first-write')
+      expect(entity?._rev).toBe(1)
+    })
+
+    it('returns the existing id without writing when the id is taken', async () => {
+      await brain.add({
+        id: '22222222-2222-4222-8222-222222222222',
+        data: 'original',
+        type: NounType.Document
+      })
+      const id = await brain.add({
+        id: '22222222-2222-4222-8222-222222222222',
+        data: 'attempted-overwrite',
+        type: NounType.Document,
+        ifAbsent: true
+      })
+      expect(id).toBe('22222222-2222-4222-8222-222222222222')
+      const entity = await brain.get(id)
+      expect(entity?.data).toBe('original')  // not overwritten
+      expect(entity?._rev).toBe(1)  // not bumped (no write happened)
+    })
+
+    it('is a no-op (writes anyway) when no id is supplied', async () => {
+      // ifAbsent without id can't mean anything — a fresh UUID can't collide.
+      // The contract says "ignored when id is omitted." Verify it writes.
+      const id = await brain.add({
+        data: 'no-id-uuid-generated',
+        type: NounType.Document,
+        ifAbsent: true
+      })
+      expect(id).toMatch(/^[0-9a-f]{8}-/)  // UUID v4
+      const entity = await brain.get(id)
+      expect(entity?.data).toBe('no-id-uuid-generated')
+    })
+  })
+
+  describe('addMany({ ifAbsent: true })', () => {
+    it('applies ifAbsent to every item', async () => {
+      // Seed one entity that should NOT be overwritten by the batch.
+      await brain.add({
+        id: '33333333-3333-4333-8333-333333333333',
+        data: 'original',
+        type: NounType.Document
+      })
+
+      const result = await brain.addMany({
+        ifAbsent: true,
+        items: [
+          { id: '33333333-3333-4333-8333-333333333333', data: 'should-not-overwrite', type: NounType.Document },
+          { id: '44444444-4444-4444-8444-444444444444', data: 'new', type: NounType.Document },
+          { id: '55555555-5555-4555-8555-555555555555', data: 'new', type: NounType.Document }
+        ]
+      })
+
+      expect(result.successful).toContain('33333333-3333-4333-8333-333333333333')
+      expect(result.successful).toContain('44444444-4444-4444-8444-444444444444')
+      expect(result.successful).toContain('55555555-5555-4555-8555-555555555555')
+
+      const seeded = await brain.get('33333333-3333-4333-8333-333333333333')
+      expect(seeded?.data).toBe('original')
+      const fresh1 = await brain.get('44444444-4444-4444-8444-444444444444')
+      expect(fresh1?.data).toBe('new')
+    })
+
+    it('per-item ifAbsent overrides the batch flag', async () => {
+      await brain.add({
+        id: '77777777-7777-4777-8777-777777777777',
+        data: 'keep',
+        type: NounType.Document
+      })
+
+      // Batch flag is false (default), but one item opts in.
+      const result = await brain.addMany({
+        items: [
+          { id: '77777777-7777-4777-8777-777777777777', data: 'overwrite-attempt', type: NounType.Document, ifAbsent: true },
+          { id: '66666666-6666-4666-8666-666666666666', data: 'new', type: NounType.Document }
+        ]
+      })
+
+      expect(result.successful).toContain('77777777-7777-4777-8777-777777777777')
+      const protectedEntity = await brain.get('77777777-7777-4777-8777-777777777777')
+      expect(protectedEntity?.data).toBe('keep')
+    })
+  })
+
+  describe('SDK scheduler use case — read-then-CAS lock pattern', () => {
+    it('two concurrent CAS updates: one wins, one throws RevisionConflictError', async () => {
+      const lockId = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb'
+      await brain.add({
+        id: lockId,
+        data: { owner: null, expiresAt: 0 },
+        type: NounType.Document,
+        metadata: { kind: 'job-state' },
+        ifAbsent: true
+      })
+
+      // Two readers see the same _rev
+      const a = await brain.get(lockId)
+      const b = await brain.get(lockId)
+      expect(a?._rev).toBe(b?._rev)
+
+      // A wins the race
+      await brain.update({
+        id: lockId,
+        data: { owner: 'worker-A', expiresAt: Date.now() + 300_000 },
+        ifRev: a!._rev
+      })
+
+      // B's CAS now fails because the rev has moved
+      let caught: unknown = null
+      try {
+        await brain.update({
+          id: lockId,
+          data: { owner: 'worker-B', expiresAt: Date.now() + 300_000 },
+          ifRev: b!._rev
+        })
+      } catch (err) {
+        caught = err
+      }
+      expect(caught).toBeInstanceOf(RevisionConflictError)
+      // Final state: A owns the lock
+      const final = await brain.get(lockId)
+      expect((final?.data as any)?.owner).toBe('worker-A')
+    })
+  })
+})