Reconciles the vector-index rename to the ruled three-layer naming: the
provider contract's identity field is now a REQUIRED readonly name (was
optional providerId), self-reported and truthful, rendered as
[vector-index:<name>] where the index identifies itself and stamped into
the op-name strings journals already parse (AddToVectorIndex(<name>)).
The built-in JS engine names itself hnsw-js. A runtime provider instance
compiled against the previous optional contract is tolerated - never
crashed on, never silently mislabeled: it stamps unknown-provider and
emits one loud warning naming the missing field. Graph index operations
keep their static names (they never interpolate provider identity), and
no public API exports a provider-routed hnsw-carrying name, so no
deprecation shim is required.
Three pieces addressing the cold-restart-write incident where a production
deployment's first writes after every restart (33-35s each on a cold page
cache) blew the op-count-scaled transact budget mid-batch: every write is
itself a multi-op transaction, so one cold operation consumed the whole
budget, the gate before the next operation tripped, and the write rolled
back atomically - refused, retried, and refused again until the page cache
warmed passively.
- The budget's start-gating contract is now explicit and pinned: it gates
STARTING the next operation, never rolling back completed work for
elapsed time (the shipped schedule since 8.7.0, now stated in contract
JSDoc, guarded by code for operation 0, and enforced by regression
tests). The 30s floor is configurable via transactionBudgetFloorMs for
stores whose cold operations legitimately run long.
- New brain.warm() eagerly loads the vector index, metadata index, and
graph adjacency so first operations after a cold restart run at
steady-state cost. Returns a WarmReport with an honest per-surface
outcome (warmed / probed / unavailable) - never reports a probe as a
warm. warmOnOpen: true runs it during init(). New optional provider hook
warm() on the vector and graph plugin contracts.
- Vector-index transaction op classes renamed from the backend-specific
AddToHNSWOperation / RemoveFromHNSWOperation to backend-neutral
AddToVectorIndexOperation / RemoveFromVectorIndexOperation, stamping the
active backend into the emitted op-name string (AddToVectorIndex(js-hnsw)
vs a native provider's own identity) so journals never misdirect an
operator toward an index that isn't running.
- The transact apply budget scales with the batch — max(30s, opCount x 2s) —
or is exactly the caller's new TransactOptions.timeoutMs. A flat 30s cap
silently limited honest bulk work to ~15 operations on network-attached
storage (~2s/op measured in a production import) while looking generous for
small batches. Internal batch paths (removeMany chunks) get the same
scaling via the shared transactTimeoutBudget helper.
- TransactionTimeoutError now reports the operation it stopped at as i/N with
the operation's name, elapsed vs budgeted time, and states the batch rolled
back atomically and is retryable; context carries the fields
programmatically.
- The transact operational envelope is documented (optimistic-concurrency
guide): budget math, chunking with ifAbsent idempotency, when to reach for
addMany vs transact, and the precompute pattern (embedBatch + per-op
vector) that keeps inference out of the commit path.
- Embedding claims made honest per an end-to-end probe of the built dist:
batch and single embedding paths produce bit-identical vectors and a
vector-supplied add is fully searchable, but batch throughput on the
default WASM engine measures comparable to sequential (~160ms/text) — the
5-10x speedup claim in the addMany JSDoc is replaced with the measured
reality and the actual win (inference outside the budgeted write path).
Persisted entity totals inflated permanently: a delete's count decrement
was sourced from re-reading the record being removed, so a null read
(replace race, or a ghost left by a pre-8.3.1 partial delete) silently
skipped the decrement while the paired add had counted — and
Math.max(persistedTotal, scanned) meant no disk cleanup could ever
lower the reported total. Reported by a production deployment with a
write→delete→re-create repro minting drift within hours.
- The caller's pre-delete read now rides through the entire delete path
(remove()/removeMany()/plan → DeleteNoun/DeleteVerb operations →
deleteNoun/deleteVerb → deleteNounMetadata/deleteVerbMetadata): a
null internal read falls back to the known prior record instead of
skipping the decrement. StorageAdapter.deleteNoun/deleteVerb gain an
optional priorMetadata parameter (additive).
- rebuildTypeCounts() rebuilt only the type-statistics arrays and
computed the total just to LOG it — the persisted scalar
(counts.json) survived every rebuild. One canonical walk now rebuilds
every counter rollup (scalar totals + per-type maps + type
statistics) and persists them; repairIndex() runs the recount
unconditionally, since counters can be inflated over clean shelves.
Two production-reported spine fixes:
- Canonical delete is a FULL removal: remove()/removeMany() deleted the
metadata leg but never the canonical vectors.json leg or the <id>/
directory, leaving ghost rows that inflated enumerated counts forever,
read as damage scars, and caused duplicate readdir entries for
re-created VFS paths (the stale-unpost path). The delete operation now
routes through storage.deleteNoun — both legs + the entity container —
with a full two-leg before-image rollback; deleteVerb symmetric; the
blind 'no metadata file' catch that masked real faults is gone. The
generation log still holds the delete's before-image, so asOf() history
is unchanged. repairIndex() gains a conservative, loud orphan-prune
sweep (containers with no metadata content leg) + count recompute for
stores damaged by earlier versions.
- Family-scoped migration gate: every read blocked on the whole-brain
migration lock even when the migrating index was irrelevant. The gate
now scopes to the index families a read actually consults — canonical
reads never wait; find() waits only on its query shape's families;
graph traversals wait only on the graph family. Writes keep the
conservative whole-brain wait. A read that needs the migrating family
still blocks (bounded) with the retryable MigrationInProgressError.
When a transaction failed and its rollback ALSO failed to undo a canonical
write (retries exhausted), the old path logged, continued, threw
TransactionRollbackError, and set state='rolled_back' — while the record it
could not undo stayed durable on disk. The caller got an error implying the
write was undone; a read-back showed the record. A failed rollback had no
truthful response (the post-commit response lie).
Give a failed rollback a two-branch honest contract, decided by observation:
both commit paths already hold byte-identical before-images, so after a failed
rollback the store compares current canonical state to them and classifies each
touched record as reconciled, an additive orphan (present when it should be
gone), or a restorative loss (gone/wrong when it should have been restored).
- Adopt-forward: a single-op write whose only damage is a durably-present
orphan — the record the caller asked for — is committed forward (its
generation buffered) and returns success with a loud warning that the derived
index may be incomplete for that id until the next rebuild/repairIndex(). No
error, no double-write; the record is durable and get-able immediately.
- Fail loud + quarantine: a multi-op batch, or ANY restorative loss, throws the
new StoreInconsistentError naming every unreconciled record and its
disposition, and puts the brain into write-quarantine (reads keep working,
writes refused via assertWritable) until repairIndex() reconciles the derived
indexes against canonical and lifts it. The counter is not advanced, and
Transaction.rollback now reports state 'inconsistent' instead of the
'rolled_back' lie when any undo failed.
New: StoreInconsistentError + UnreconciledRecord exported from the root;
repairIndex() forces a rebuild and clears the quarantine. Regression
tests/integration/rollback-trapdoor.test.ts injects index-add-throws +
canonical-delete-undo-fails and pins adopt-forward (durable, get-able, not
quarantined), fail-loud (StoreInconsistentError + quarantine + reads work +
repairIndex lifts it), and the error's record naming.
Transaction.execute() checked its time budget at the top of the operation
loop and threw TransactionTimeoutError from OUTSIDE the per-operation
try/catch, so a mid-flight timeout bypassed rollback entirely — only
per-operation failures rolled back. A bulk transact that crossed its 30s
budget mid-flight left the operations already applied to canonical storage
in place while the generation was never stamped: torn, generation-less
state. The generation-store commit path's abort cleanup explicitly assumes a
throw from execute() already restored the applied operations byte-identically
(it only discards the uncommitted staging directory), so the missing
rollback broke that invariant.
Give execute() a single rollback point: the operation loop is the sole
rollback-guarded region, and any error escaping it — an operation failure OR
a mid-flight timeout — rolls back every applied operation in reverse order,
then surfaces the original error (a rollback failure still supersedes it via
TransactionRollbackError). The per-exit-path rollback that let the timeout
throw slip past is gone; atomicity now holds by construction for every error
type. An aborted transaction leaves generation() unchanged and storage
byte-identical to its pre-transaction state.
Regression (tests/unit/transaction/timeout-rollback.test.ts): a mid-flight
timeout leaves an in-memory canonical store byte-identical with the tx in the
rolled_back terminal state; the operation-failure and rollback-failure paths
through the same single rollback point; and a clean transaction still commits.
transact() promises atomic forward references — add an entity and relate to
it in one batch — but the planner resolved relationship endpoint ints at
PLAN time, before the batch's add operations had applied. Asking the id
mapper about an entity that does not exist yet made the (correctly strict)
native mapper throw in EntityIdMapper.getOrAssign, so
transact([{op:'add', id:X}, {op:'relate', to:X}]) failed on native
deployments; the permissive JS mapper masked the bug and silently leaked an
id assignment whenever a batch was later rejected by a commit precondition
(normal control flow since the conditional-commit CAS landed).
Make endpoint resolution lazy: AddToGraphIndexOperation and
RemoveFromGraphIndexOperation take VerbEndpointInts — an eager
{sourceInt, targetInt} or a thunk evaluated when the operation EXECUTES,
mirroring the constructor's already-lazy generationFn. The four
transact-planner sites (relate, its bidirectional reverse edge, remove's
relationship cascade, unrelate) pass thunks, so resolution happens inside
the commit after same-batch adds have applied; the seven single-operation
sites keep eager objects (their endpoints provably pre-exist — relate
validates both, unrelate/updateRelation/remove read the existing verb).
The remove operation's rollback captures the ints resolved at execute so
its re-add uses the same mappings.
Byproduct fix: a rejected batch no longer pollutes the id mapper — the
regression suite pins this (mapper has no assignment for the phantom
entity after a precondition-rejected forward-ref batch), alongside the
exact reported shape, both-endpoints-in-batch, bidirectional,
add+relate+remove in one batch, and the split-transact control
(tests/integration/transact-forward-ref-graph.test.ts).
Consumer-invisible modernization pass — no public API or runtime-behavior
change; dist for the override-only files is byte-identical.
Toolchain:
- CI: GitHub Actions matrix — Node 22/24 (test:unit) + Bun latest (test:bun).
- engines: node ">=22" (was "22.x"), bun ">=1.1.0".
- tsconfig: isolatedModules + noImplicitOverride; add the 33 `override`
modifiers the flag requires across storage/integrations/vfs/transaction.
- deps: @types/node ^22; add prettier; drop dead standard-version,
@rollup/plugin-* and the redundant embedded eslintConfig (flat
eslint.config.js is the active config — verified identical lint output).
paramValidation: replace the top-level `await import('node:os'/'node:fs')`
with static ESM imports. The top-level-await form poisoned the module graph;
static imports also drop the browser/edge fallback branches no supported
runtime reaches (8.0 is Node/Bun/Deno-only).
Bun positioning: recommend Bun as a runtime (`bun add` / `bun run`), which is
green (test:bun 8/8). Drop single-binary `bun build --compile` as a target —
native addons cannot embed into it, and Bun 1.3.10 has a `--compile` codegen
regression around top-level await. Rename the Bun test to bun-runtime-test.ts
and correct docs that overclaimed single-binary support.
Gates: typecheck 0, build 0, test:unit 1743/1743, test:bun 8/8.
Pre-GA dead-code sweep. A deterministic import-reachability walk from the
package's real entry points (the exports map, the CLI bin, the conversation
surface) found 34 source modules that nothing reachable imports — they
compile and ship as dist output that no consumer or internal path can ever
reach. All were superseded duplicates, abandoned parallel implementations,
or built-but-never-wired features:
- superseded duplicates of live modules: an older "unified" entry, a
standalone neural-import variant, a static NLP processor + its matcher,
a parallel API-types module, a duplicate progress-types module
- an abandoned import path (orchestrator + entity deduplicator + barrels)
left behind when ingestion moved to the coordinator
- unwired feature modules (instance pool, import presets, cached
embeddings, relationship-confidence scorer) reachable only from tests
- dead leaf utilities (write buffer, deleted-items index, bounded
registry, two crypto shims, a cache manager, a structured logger, a
stale v5 type-migration helper, a browser-only FS type shim) and
several dead re-export barrels
- a CLI catalog command wired into no command
Also removed two tests that only exercised deleted modules, repointed two
VFS tests off a deleted barrel onto the implementation module, and deleted
one example that demoed removed features.
Verified: clean build (both tsc passes), unit 1505 + integration 607 green,
and a re-run of the reachability walk reports zero remaining orphans.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Graph time-travel needs an edge's existence recorded per generation so
db.asOf(g) hops resolve historically correct endpoints. The metadata layer
already threads brainy's commit generation per write; the graph write path did
not, leaving a versioned verb-endpoint store unable to answer "which edges
existed at generation g".
- GraphIndexProvider.addVerb/removeVerb gain a `generation: bigint` parameter
(the same watermark the storage layer stamps onto the record). A provider with
a per-generation edge chain stamps the edge at that generation; the JS baseline
has no such chain and accepts-and-ignores it — graph time-travel is a
native-provider capability, and the open-core path serves edges as-of-now (the
one documented graph time-travel limitation).
- The two graph transaction operations resolve the generation via a thunk at
EXECUTE time: the generation store assigns the batch generation only once the
commit begins executing, after the operations are planned. The same generation
is reused for an operation's rollback half.
- All graph-write call sites pass the in-flight generation.
Adds a spy-provider test proving the threading, execute-time resolution, and
forward/rollback generation reuse. The JS index ignores the value, so behaviour
is unchanged: unit 1402/1402, db-mvcc 25/25, bigint-contract relate/unrelate 10/10.
The COW version-control surface (fork, branches, checkout, commit,
getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with
its subsystems: src/versioning/, the COW object store (CommitLog,
CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the
TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/
with/persist/restore) is the one versioning model in 8.0.
Survivors and replacements:
- BlobStorage survives (the VFS stores file content through it), relocated
to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter
interface is now BlobStoreAdapter, slimmed to the consumed surface
(write/read/has/delete/getMetadata + MIME-aware compression policy).
- brain.migrate() backup branches are replaced by persist-before-migrate:
MigrateOptions.backupTo persists a hard-link snapshot of the current
generation before any transform runs; MigrationResult.backupPath reports
it, and brain.restore(path) brings it back wholesale.
- CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by
snapshot.ts — snapshot <path>, restore <path>, history (tx-log),
generation.
- New public read API: brain.transactionLog({limit}) exposes the reified
tx-log (generation/timestamp/meta, newest first) that backs the CLI
history command; TxLogEntry is exported.
Tests: superseded suites deleted; fork/commit blocks excised from shared
suites; BlobStorage tests relocated + reworked against the slimmed store;
migration tests now prove the backupTo snapshot/restore round trip; new
transactionLog coverage in db-mvcc.
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror):
- getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and
return entity/verb ints as bigint[]
- new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7
identity-fingerprint design — verb ids are UUIDs by contract, so the
provider-side interning is losslessly reversible)
- addVerb(verb, sourceInt, targetInt) returns the interned verb int;
removeVerb(verbId) joins the contract
Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on
writes, getInt on reads (unmapped UUID -> empty result without calling the
provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry
insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb
returns and resolver results. GraphVerb gains derived sourceInt/targetInt
(populated at add time, never persisted). findConnectedSubtype gains a
native fast path that routes single-type single-subtype outgoing BFS
through the provider when available.
JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed
internally: entity ints resolve through the shared entity-id mapper (threaded
in by the coordinator on init/fork/checkout), verb ints come from an
in-process append-only interning map re-derived from storage on
rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/
removeEntity take bigint, sortTopK/filteredSortTopK return bigint[].
relate() now rejects a caller-supplied id with a teaching error — verb ids
are brainy-generated UUIDs by contract in 8.0 (previously a passed id was
silently ignored). No Roaring64 provider-boundary decode site exists yet;
the JS-internal column store stays Roaring32 and the Treemap decoder lands
with the first consumer of provider-returned filter buffers.
Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
Step 3 of the brainy 8.0 rename scaffolding. The class name now matches
its role: the JS HNSW implementation of the VectorIndexProvider contract.
Per planning § 2.3: this is NOT cosmetic — leaving HNSWIndex when the
public contract is VectorIndexProvider creates a confusing read at the
implementation layer.
CHANGES
Repo-wide mechanical rename: HNSWIndex → JsHnswVectorIndex across 52
references in src/ + 5 references in tests/ via sed. Class declaration,
imports, JSDoc, comments, and identifiers all updated.
src/index.ts
- Primary public export: JsHnswVectorIndex (the new name).
- Backwards-compat alias: `export const HNSWIndex = JsHnswVectorIndex`
with @deprecated note. Removed in the final 8.0 cleanup commit.
NO-OP scope
No behavioural change. The class is exactly the same; only the name
moved. Tests + build green.
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit
Optimistic concurrency for multi-writer coordination — the read-then-CAS
lock pattern + idempotent-bootstrap singleton inserts. Lands the surface the
SDK scheduler asked for in BRAINY-EXPOSE-TRANSACTIONS, scoped to what ships
cleanly without painting the 8.0 transact() API into a corner.
A. PER-ENTITY _rev FIELD
- src/coreTypes.ts — HNSWNounWithMetadata gains optional `_rev?: number`;
STANDARD_ENTITY_FIELDS includes it so resolveEntityField routes correctly.
- src/types/brainy.types.ts — Entity<T> gains optional `_rev?: number`;
Result<T> mirrors it on the convenience flatten layer.
- src/brainy.ts add() — initializes `_rev: 1` in storageMetadata. New entities
always start at 1.
- src/brainy.ts update() — reads currentRev off the persisted metadata
(falls back to 1 for pre-7.31.0 entities with no _rev), writes
`_rev: currentRev + 1` into the updated metadata. Every successful update()
bumps by exactly 1.
- src/brainy.ts convertNounToEntity + convertMetadataToEntity — both pull
_rev out of the storage metadata and surface it at the top level of Entity.
Pre-7.31.0 entities (no _rev in storage) read as `_rev: 1` so consumers
see a consistent value.
- src/storage/baseStorage.ts — six destructure sites updated to pull _rev
out of the metadata bag so it doesn't leak into customMetadata. Noun-side
returns surface _rev; verb-side destructures correctly but doesn't expose
it on HNSWVerbWithMetadata (verb CAS is future work).
- src/brainy.ts createResult() — flattens entity._rev onto the Result for
backward compat with the existing convenience-field layer.
B. update({ ifRev }) OPTIMISTIC CONCURRENCY
- src/types/brainy.types.ts — UpdateParams<T> gains optional `ifRev?: number`.
- src/transaction/RevisionConflictError.ts (NEW) — carries `{ id, expected,
actual }`. Message names the recipe: refetch with brain.get() and retry
with the latest _rev. Subclass of Error.
- src/brainy.ts update() — when params.ifRev is provided, compares against
currentRev (the rev we just read) and throws RevisionConflictError on
mismatch before any storage write. Omitting ifRev keeps the prior
unconditional-update behavior; existing consumers see no change.
- src/transaction/index.ts — exports RevisionConflictError.
- src/index.ts — public export of RevisionConflictError.
C. add({ ifAbsent }) BY-ID IDEMPOTENT INSERT
- src/types/brainy.types.ts — AddParams<T> gains optional `ifAbsent?: boolean`;
AddManyParams<T> mirrors it as a batch-level flag.
- src/brainy.ts add() — when params.id AND params.ifAbsent, pre-reads
storage.getNounMetadata(id); if present, returns the existing id without
writing. No throw, no overwrite. Ignored when id is omitted (a fresh UUID
can never collide).
- src/brainy.ts addMany() — propagates the batch-level ifAbsent to each
item's add() call. Per-item ifAbsent takes precedence so callers can
override individual rows.
WHAT'S NOT SHIPPED (and why)
A public brain.transaction(fn) wrapper was on the table but was cut. The
internal TransactionManager exposes raw Operation classes (SaveNounMetadata,
etc.) that take StorageAdapter as a constructor argument. A clean high-level
facade in 7.31.0 would have meant either:
(a) Delegate to brain.add() / update() / relate(). Each of those opens its
own internal transaction and commits before the closure returns. Looks
atomic, isn't — a footgun.
(b) Thread an optional `tx?` through every internal write site in
brainy.ts (8 transactionManager.executeTransaction sites). ~2 days of
real refactor with regression surface, and the API shape changes again
in 8.0 anyway.
The SDK scheduler's actual ask (BRAINY-EXPOSE-TRANSACTIONS) is the
read-then-CAS lock pattern. _rev + ifRev solves it completely. Multi-write
atomicity is the 8.0 brain.transact() use case where the Datomic-style
immutable Db makes it atomic by construction — that's the right home.
TESTS
- New tests/integration/rev-and-ifabsent.test.ts (18 tests):
- _rev initialization (1 on add) and surface on get fast/full paths + find
- _rev auto-bump on update across multiple writes
- update({ ifRev }) pass / fail / message format / omitted / legacy-no-rev
- add({ ifAbsent }) writes when absent / no-op when present / id-required
- addMany({ ifAbsent }) propagation + per-item override
- SDK-scheduler scenario: two concurrent CAS updates, one wins one throws
- Unit suite unchanged: 1468/1468.
- All integration subtype + verb + strict + find-limits + new rev suites
pass: 96/96.
DOCS
- New docs/guides/optimistic-concurrency.md (public: true) — full reference:
the lock pattern, read-modify-write retry, idempotent bootstrap, how _rev
interacts with brain.versions, branches/fork, and VFS, what's coming in 8.0.
- docs/api/README.md — add() and update() entries get the new params + tips
pointing at the new guide.
- RELEASES.md v7.31.0 entry.
CORTEX COMPATIBILITY
Zero changes required. _rev is a metadata column already supported by
NativeColumnStore. The auto-bump runs in Brainy JS before any storage call;
Cortex never sees the per-entity counter.
8.0 FORWARD-COMPAT
_rev, ifRev, RevisionConflictError, and ifAbsent survive the 8.0 Db redesign
unchanged. 8.0 layers brain.transact(tx, { ifAtGeneration }) for whole-tx CAS
on top of the same per-entity mechanism — per-entity for single-record
patterns (job locks, idempotent state machines), generation-based for
"did the world move under me." Locked in .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md § C-6.
Verification
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit
- All integration suites including new rev-and-ifabsent: 96/96
- npm run build: clean
- Closed-source product reference audit: clean
Shutdown/close/flush now properly flushes all 4 components in parallel:
metadataIndex, graphIndex, HNSW dirty nodes, and storage counts. Previously
only counts were flushed, causing native provider data loss on restart.
Also:
- Wire roaring, msgpack, entityIdMapper provider consumption from plugins
- Fix allOf filter O(n²) intersection → O(n) Set-based
- Fix ne/exists negation filter to use Set-based exclusion
- Add setMsgpackImplementation() swap in SSTable for native msgpack
- Add setRoaringImplementation() swap for native CRoaring bitmaps
- Add getAllIntIds() to EntityIdMapper for bitmap operations
- Remove TypeAwareHNSWIndex from default index creation path
- Export memory detection utilities from internals
- Clean up 26 permanently-skipped dead tests
- Remove detectAndRepairCorruption from init() hot path — was loading all
metadata chunks sequentially on startup. Now available via checkHealth()
and repairIndex() methods.
- Short-circuit warmCache on empty workspace — skip 4-6 wasted storage reads.
- Parallelize MetadataIndex.init() and getGraphIndex() via Promise.all().
- Defer metadata writes during rebuild to batch boundaries (every 5000
entities) instead of flushing per-entity.
- Skip pre-reads for new entities in transactions — saves 2 storage
round-trips per add() on cloud storage.
