An internal report from cross-engine write-path instrumentation: with
individual writes slower than the idle window (a contended disk), every
inter-write gap looked idle and fired a background full flush — 15 extra
flushes during 100 contended adds, amplifying the very pressure that
slowed the writes. The law now: an idle fire landing within the spacing
floor of the last flush DEFERS to the floor boundary instead of flushing;
the floor is min(interval, 10× the CONFIGURED idle window) — scaled to
caller intent (a tiny idle window keeps fast idle-driven durability;
default 2s/30s config gets a 20s floor), derived from the configured
idle, never from a deferred re-arm delay (which would compound into
runaway deferral). Deferred is never dropped: a lone write on a
then-quiet store still persists at the floor without any further write
arriving.
Pins: the contended-shape pin (six slow-spaced writes fire ≤2 idle
flushes, not one per gap; then still persist) + the original quiet-store
idle pin unchanged. Unit 2055/2055.
The generic reprojection engine (pure TS; the twin of the native
implementation — same frozen contract, one shared conformance intent):
register any ProjectionAdapter; advance(family, {budgetMs}) folds facts
from the adapter's own watermark to the head in installments ≤50ms with
real macrotask yields; foreground door traffic bumps the DoorSignal and
an in-flight advance yields within one installment ('preempted');
advanceAll round-robins families fairly. swap(family, buildAdapter) is
the doors-open migration primitive: the OLD projection keeps serving
while the new one builds beside it, the flip is atomic at parity, and a
concurrent second swap refuses typed. A fact the fold cannot apply
(typed ProjectionApplyError) is QUARANTINED — skipped, ledgered,
narrated per-doubling, exposed for refuse-affected-reads — the service
class law's fourth answer: never a wedged rebuild, never a silent skip.
The engine never writes stamps: each adapter owns its durability and its
stamp-after-data discipline. Upgrade, heal, and rebuild are now the same
machinery behind open doors.
FactLogSource wires any host's fact scan in one line
(factSourceFromHost(brain)); window-contract violations are loud.
Pins: 23 unit (budget resume without refold · preemption within one
installment · round-robin fairness under a skewed backlog · build-beside
visibility mid-swap · atomic flip · single-flight refusal · quarantine
skip/ledger/doubling · non-typed throw aborts · losing adapter
discarded) + 3 integration on a real brain (fold matches ground truth ·
doors answer mid-fold with the preemption path exercised · crash
mid-fold resumes from the stamp, never refolds).
Gates: unit 2054/2054 (157 files) · integration 820 (93 files) ·
conformance 31/31.
The private recovery discipline, applied to its own machinery: pending-
embed markers stop being sidecar files and become first-class log records
riding the write's OWN commit fact — embed.pending lands in the same
atomic append as its after-image (a marker can never be orphaned from its
write, or vice versa; in durable-at-ack mode it shares the write's
covering fsync — zero extra syncs), and the worker's landing commit rides
embed.landed with the inline vector. Crash recovery is now a FOLD of the
log (pending without a matching landed = recovered), skipped wholesale on
brains with no v2 history; the one-time legacy bridge folds existing
sidecar files in, migrates them as one fact, and deletes them —
idempotent under a crash mid-bridge. No code path writes the sidecar
again.
Plus the ENTITY-TRUTH digest law, found by this train's own pins:
canonical vector wrappers denormalize HNSW residue (connections + the
randomly-assigned node level) that the log deliberately does not carry —
the verification oracle digested it and would have reported false
state-differs on ~any nonzero-level node (a ~15% flake in the cutover pin
was the symptom). Both sides of every oracle comparison now normalize to
entity truth (nounEntityTruth); index residue has its own rebuild path
and is not entity state.
Pins: embed-markers-in-log 5/5 (same-generation marker, landed+fold-to-
zero, crash recovery via the log with the sidecar prefix EMPTY on disk,
legacy bridge, VFS hung-embedder ack) · deferred-embedding 5/5 unchanged
(the contract outlived its mechanism) · kill-matrix 11/11 · cutover 5/5
×10 runs (flake dead) · unit 2031/2031.
One deterministic v2 log (nine facts covering every fold-relevant behavior:
genesis, after-images with minted ints, a deferred embed pending→landed,
a sameAsGeneration vector ref, a verb, a tombstone, and an all-deduped
empty commit) whose ENCODED BYTES and FOLDED STATE are both pinned by
sha256 literals. The fixture (tests/fixtures/golden-log-v2.bin, 4128 B,
byte-verified against the encoder on every run) is the shared artifact a
second reader implementation consumes — it must reproduce the identical
fold digest; the pair is normative on disagreement. The fold law is
stated in prose beside the code: generation-ordered latest-per-id,
tombstone masking, embed.landed vector application, single-hop ref
resolution, key-sorted digest.
Also: decodeGroupV2 discriminated pad filler by RECORD COUNT, silently
swallowing legitimate empty commits (an all-deduped batch at a real
generation). Pads carry generation 0 — which writers can never mint — so
the generation is the honest discriminator; empty commits stay visible.
Pins: 4/4 (encode-exact, fixture-identical, fold-exact, human-readable
spot checks beside the hashes).
- Watermark stamping fans out at flush: all three projections stamped
with the committed generation before their flushes persist.
- waitForIndexed(path?, {generation, timeoutMs}) — the one honest read
barrier for write-then-recall consumers; typed timeout error carries
the pending count and names the gauge; getIndexStatus() gains
per-projection gauges. awaitPendingEmbeds() unchanged underneath.
- adoptLogAuthority() self-backfills curable divergences (pre-log
records, witness drift) by identity re-commit before flipping — a
fresh brain flips clean; log-ahead divergences still refuse loudly.
- The verification oracle gains VERB legs (all four divergence classes;
unwired = honest verbsChecked: 0, never a scope claim).
- find({where: {}}) match-all serves (was silent-empty, warm AND cold;
same fix in count/streaming/subgraph seeding); removeMany({where:{}})
refuses typed — a match-all bulk delete must be explicit.
- Aggregation native envelope stamped via noteSourceGeneration before
serializeState; the native-blob restore gates through the same
adoption verdict as caller-side state (the unconditional adopt dies).
- LC8 pinned: a wholesale directory move opens and serves identically
across all three intelligences, with history traveling.
Gates: unit 2031/2031 (156 files) · integration 812 (91 files) ·
conformance 27/27.
Every persisted projection artifact (metadata field indexes + column
segments, HNSW node records, graph adjacency LSM trees) now carries a
stamp asserting 'this state reflects every committed generation ≤ W,
atomically' — written LAST in each owner's flush (stamp-after-data: a
crash between data and stamp = unstamped = rescan, never trust). At load,
each owner computes the three-way verdict: stamped==committed → adopt
(zero work) · behind → catchup (gap reported) · above/unstamped → RESCAN,
loudly. Legacy artifacts re-derive once, then are stamped forever. Shared
law in projectionWatermark.ts (the aggregation verdict machinery,
generalized); vector artifacts carry model dimensions. Verdicts are
computed and exposed (watermark()/watermarkVerdict()/watermarkGap());
rebuild triggers unchanged — acting on 'catchup' is the fold train.
Pins: 22 unit (7 metadata · 8 hnsw · 7 graph, incl. spy-order
stamp-after-data) + the end-to-end reopen-adopts pin.
The cutover: new tail segments write format v2 (per-record [type, version,
cipherFlag, keyId] envelope; noun/verb after-images carry dense ints
MINTED AT APPEND from the id mapper — a rebuilt mapper reproduces
assignments exactly; log.genesis opens every new log with the id-space
width + a minted brain id; sync() seals to the header-declared sector
boundary with reader-invisible pad frames). Existing v1 segments are
never rewritten — per-segment decoder dispatch reads both formats and v2
facts map to the exact CommitFact shape all consumers already read.
Cutover on a live v1 log: an empty v1 tail re-heads in place; a non-empty
one is sealed by rotation, byte-identical. Records reserve the encryption
fields (cipherFlag 0 / keyId nil are the only legal values; anything else
refuses typed naming the needed newer reader) — crypto-ready with no
future bump on the compat surface. Empty-records facts are legal (an
all-deduped batch is a real generation — v1 semantics preserved; the
refusal there tore a column-store flush mid-commit in the full suite, the
consistency guard caught it loudly, and the root is fixed).
Golden byte vectors pinned for the second (native) reader implementation.
Pins: cutover 5/5 · codec 54 · kill-matrix stays 11/11.
The time-travel recall row moves from envelope-note to contracted: vector
search at a pinned past generation serves the vectors AS THEY STOOD —
a later re-embed never leaks into an earlier pin (byte-exact), tombstones
mask, the deferred-embed pin serves the stub on the vector leg until the
landing generation (text/metadata legs unaffected — triple intelligence by
design), and beyond-head pins refuse typed. Brainy-alone leg = the
documented ephemeral at-generation materialization; the at-scale leg rides
the accelerated provider's as-of index. Registry row added (shared ID
pending the master table).
Two release-blocking findings from the durability kill-matrix, both fixed
in the owning layer:
1. LOG-AUTHORITY REPLAY AT OPEN: durable-at-ack fsynced the fact before
the ack, but open() truncated every fact above the manifest — after a
power loss that takes the un-fsynced tmp+rename canonical bytes, the
acked write's ONLY durable copy was discarded. Now: under 'log'
authority, open() REPLAYS intact facts above the manifest into
canonical (FactLog.peekFactsAbove — CRC-gated, order-sorted) and
advances the manifest to cover them; tree-authority brains keep the
truncate contract they were promised. Pinned end to end: the power-loss
row constructs the exact disk state (fsynced log, vanished canonical
rename) and the acked write lives.
2. NO SILENT COMMIT: commitSingleOp buffered the generation BEFORE the
fact append; an append failure (ENOSPC) rejected the caller but the
next flush durably committed the generation with NO fact — a permanent
silent log gap. Now the failure path un-buffers and returns the counter
reservation: nothing commits, the log stays gap-free, and the canonical
execute-residue orphan is the documented crash-equivalent.
Plus: the kill-matrix itself (11 rows — every commit-path fault point ×
reopen-as-crash recovery contract, at-ack variants, disk-full row; five
new zero-cost faultPoint sites), the log-authority pin suite (oracle
green/red/state-differs, flip refusal, switch survives reopen, 9/9), and
the group-commit covering pins (5/5).
Gates: unit 2002/2002 (152 files) · integration 785 · conformance 27/27.
The provider contract (metadata addToIndex/removeFromIndex, vector
addItem/removeItem, id-mapper getOrAssign/remove) gains an optional
trailing generation — evaluated lazily at operation execute time (the
graph surface's thunk pattern, generalized), threaded from all 17
construction sites: undefined during generation-0 bootstrap, the real
committed generation everywhere else. Optional = additive: no existing
provider or caller breaks; native delta logs that stamped literal zero
start hearing truth. JS twins accept the parameter with parity notes.
Pins: provider doubles capture and assert nonzero monotonic generations
across add/update/remove on both surfaces.
The two-implementation contract surface as one pure module (no I/O):
segment header v2 (formatVersion 2 + sealSize in the reserved bytes),
per-record [type u8, version u8] envelope killing the unknown-kind
misclassification trap, the 12-type registry (after-images with minted
ints, tombstones, batch.meta, embed.pending/landed, blob.manifest,
projection.note, bootstrap.baseline, log.genesis with id-space width and
TYPED width-mismatch refusal), vectorLeg inline|{sameAsGeneration} with
writer-enforced single-hop, sector-sealed groups with pad frames, torn-tail
discipline, and GOLDEN BYTE VECTORS pinned so a second (native) reader
implementation can conform byte-for-byte. 50 format pins + a
fault-injecting storage wrapper (tear/drop-sync/fail-append) with 13
self-tests. v1 segments remain readable; nothing writes v2 yet — the
live-format cutover is its own commit.
A3 of the service-class pair (BRAINY-PROD-LATENCY-TRIAD): a VFS file write
ran the embedder synchronously while the caller waited — 5.6s p50 / 21.4s
p95 per small file on a production deployment, the dominant stage of every
capture write.
- add()/update() gain deferEmbedding: the write acks at durability (data +
metadata persisted, a DURABLE pending marker under
_system/pending_embeds/<id> written BEFORE the commit — orphan-safe
direction); the single-flight background worker embeds the CURRENT data
and swaps the vector in ATOMICALLY (ReplaceInVectorIndex — the row is
never absent from search; a deferred UPDATE keeps serving the OLD vector,
stale-beats-absent per the flicker law). Typed refusals: defer+vector,
defer-without-data.
- CRASH-SAFE: markers are recovered at open by a BOUNDED prefix listing
(never a store walk) and the worker resumes in the background — a crash
can delay a vector, never lose one. A wedged embedder trips a LOUD 60s
hang guard and the worker moves on (marker retained for retry).
- The honest gauges: getIndexStatus().pendingEmbeds + pendingEmbedCount();
awaitPendingEmbeds() is the eventual-vector-index BARRIER for callers
and tests that need searchability before proceeding.
- VFS adopts it everywhere a write path could wait on the embedder:
writeFile (both branches) and directory creation. Pinned in the
strongest form: writeFile resolves while the embedder HANGS FOREVER.
Pins: deferred-embedding 5/5 (ack law · stale-beats-absent · crash
recovery across sessions · VFS hung-embedder ack · typed refusals).
Gates: unit 1928/1928 · integration 765 · conformance 27/27.
DP6/DP8 of the Path Registry (BRAINY-PROD-LATENCY-TRIAD, the proven flicker
mechanism): update paths staged RemoveFromVectorIndex then AddToVectorIndex
as two separately-awaited transaction ops — between them a live row was in
NEITHER index (dark to semantic recall, fine in metadata list). The native
pair widened that window to seconds in production before their side's
visibility-commit fix; the structural cure lands here:
- hnswIndex.updateItem: absent → add; SAME vector → pure no-op (the
production shape — a type-only update re-indexed an unchanged vector,
remove+add did pure damage); changed vector → the node NEVER leaves the
index: synchronous vector swap first (every query from that instant sees
correct distances), then unlink/relink at the node's existing level via
shared internals (linkNode/unlinkNodeEdges refactored out of add/remove;
entry point and maxLevel provably unchanged).
- ReplaceInVectorIndexOperation: ONE transaction leg; feature-detects
provider updateItem (native seam flagged — their side ships updateItem,
then the adjacent remove+add fallback is dead code). Both update staging
sites swapped; delete sites untouched.
- LAZY-OPEN GATE (fleet adoption find, SELF-ENGINE-PAIR-STANDARD): under
disableAutoRebuild, ensureIndexesLoaded assessed ONLY the vector index —
a not-ready native METADATA provider never blocked the completion latch
and every find() silently returned [] on a populated store. All three
providers now vote; any not-ready report falls through to the rebuild.
- docs/path-registry.md: brainy's twin table for the 32 shared path IDs —
service class, budgets, lifecycle, narration, and the cited pin per row;
owed rows named (LC4 doors-open migration, MT4 yielding heals, LC7
downgrade contract) per the lifecycle-sprint choreography.
Pins: update-item-atomic 9/9 (visibility-atomic swap, reverse-index parity
vs fresh rebuild, entry-point invariants) · lazy-notready-honor 2/2.
Gates: unit 1928/1928 (148 files) · integration 760 · conformance 27/27.
A4 of the service-class pair (SELF-ENGINE-LIFECYCLE-SPRINT, David-directed:
'why do we need manual flushes at all?'). The production disease: 829
caller-scheduled per-write flushes convoying into 45-66s write walls —
cadence hand-rolled a layer above the only layer that can see dirty-node
counts and IO pressure.
- BrainyConfig.persistence: policy 'auto' (DEFAULT) | 'manual', with
flushEveryWrites (512) / flushIntervalMs (30s) / flushOnIdleMs (2s)
triggers. Auto = the engine kicks ONE single-flight BACKGROUND flush at
a threshold or when the store goes quiet; write acks NEVER await it (a
hung flush cannot block a write — pinned); a failed background flush is
LOUD and re-arms the trigger. 'manual' restores caller-owned cadence.
- Triggers wired at both write chokepoints (single-op post-commit +
transact post-commit); idle timer unref'd; close() tears the timer down
and drains the flight before its own final flush.
- RECOVERY SEMANTICS documented on the config: canonical records are
durable per-write regardless of policy — a crash between background
flushes loses derived state only, which converges at next open (epoch
machinery + the new incremental aggregation catch-up), bounded by the
un-flushed window. Never data loss.
Pins: write-count trigger fires one background flush with zero caller
calls · idle trigger · manual never self-flushes · THE ACK LAW (writes
acknowledge under a never-resolving flush). Gates: unit 1917/1917 ·
integration 760 · conformance 27/27 — green WITH auto as the default.
SELF-ENGINE-LIFECYCLE-SPRINT + BRAINY-PROD-LATENCY-TRIAD, the four asks:
(a) brain.flush() persists aggregation state stamped at the committed
generation. The stamp used to advance only at close(), so a long-lived
writer that flushes but never closes — the primary production shape —
left every write window behind the stamp, and ANY unclean exit forced a
whole-store backfill walk (per-entity work, measured >60s and
door-starving on a 9k-row production brain) on the first stats call.
(b) BEHIND-stamp adoption becomes adopt + INCREMENTAL CATCH-UP: the exact
missing window (stamp, committed] resolves its affected-id set from the
fact log and reconciles each entity with time-travel before/after reads
(asOf at both window bounds) through the same delta algebra the live
hooks use — cost bounded by writes since the last flush, never store
size, and exact under interleaving because reconciliation targets the
FIXED window end while later writes chain through hooks. Oversized
windows (>5000 affected) and unreadable windows demote to the announced
rescan — never a silent partial serve.
(c) The native provider's parallel rebuildAggregate — on the contract since
8.x but never invoked anywhere — is now the backfill walk's preferred
door: one call per aggregate with source-matched entities, replacing
the per-entity FFI stream.
(d) A delete whose before-image is unavailable can no longer SKIP the
aggregation hook silently (counts drifted upward forever): both delete
paths (remove() and transact) flag an exact rescan, loudly.
Pins: integration (flush stamp; unclean-exit reopen → exact counts through
an add + group-move + delete window with the walk spy proving ZERO
whole-store walks) + unit (provider rebuild invoked once with filtered
entities; flagAllForRescan; reconcile delta algebra). Gates: unit 1913/1913
· integration 760 · conformance 27/27.
BRAINY-PROD-LATENCY-TRIAD Track A1 (David-approved plan): the sort path's
value resolution goes BATCHED — one chunked metadata-record batch pass
serves any N, replacing the serial per-row getNoun loop (62-98ms x 3,224
rows = the measured 199-317 second silent scan on self prod). The
metadata record carries every sortable value: system scalars EXACT
(bucketed-index precision loss can never force a per-row disk read
again) and the user bag via the shape-aware split, both record eras.
- resolveOrderValuesBatch: the one sanctioned value source for ordered
reads (batch door: getNounMetadataBatch -> getMetadataBatch -> chunked
parallel; never serial).
- Column top-K page re-sort and the no-column fallback both rewired.
- B2 down-payment: the no-column fallback ANNOUNCES itself once per
field past 500 rows - silent degradation is illegal.
- THE CALL-SHAPE PIN (tests/unit/utils/metadataIndex-sort-callshape):
zero vector-record reads, batch calls only, latency-blind so it holds
on any machine - the serial loop cannot quietly return. Ordering
contract re-pinned through the batch path (nulls last both directions,
ties by id, never drop).
(! = perf contract change only; no API change. Gates: unit 1904/1904,
integration 758, conformance 27/27.)
The write side of the law, ruled 2026-08-03: data is either in main space
where developers can use anything, or it is in system.*.
- The reserved-name write door DIES: add/update/relate/updateRelation
metadata bags accept EVERY name (confidence, type, id, data, level,
content, ...) as ordinary user fields — indexed, filterable, sortable,
aggregatable, identical to any other field. The remap/enforce/warn
machinery, the reservedFieldPolicy config (now a typed init refusal),
and the compile-time metadata key bans are all removed. The one write
refusal left: keys spelled 'system.*' (namespace forgery), now enforced
on all four write doors.
- STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag
nested verbatim under 'metadata', sealed by a format stamp — by-name
storage discrimination is unsound once colliders are admitted. Legacy
flat records stay readable forever through the shape-aware splitters
(sound for them: the old door refused colliders). Time travel rides the
same split (generation store snapshots whole records).
- Name-based index exclusions DIE: user frame indexes every name; the
excludeFields/indexedFields knobs and their silent-[] holes are gone;
bulk-payload protection is value-shape only, uniform across names.
- Consumer-sweep findings fixed in the same wave: per-type counts read
the frozen 'system.type' column (addToIndex sort, affinity tracking,
cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for
pre-rebuild reads); resolveHiddenIds addresses 'system.visibility'
(bare 'visibility' was a silent no-op under the law — VFS/system
entities leaked into default reads).
- Fidelity fallout fixed in the owning layers: readEntityFieldAddress
reads the bag first (colliders were absent-shadowed by its own guard)
and never serves system addresses from the bag; blob history refs read
the bag shape-aware; migration transforms now receive ONE normalized
view (engine fields + nested bag) regardless of stored era, and stray
flat-habit keys refuse with the fix in the message.
- THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as
gates-green): all ten collider names + plumbing names written as user
fields, verified verbatim + queryable across live reads, flush+reopen,
a forced epoch rebuild, and asOf time travel; relation mirror; forgery
refusals; legacy flat-record compat. 8/8 green.
Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance
27/27 (exit 0) · consumer test sweep migrated (10 files).
Also completes the v8.10.2 write-granularity law for the transact() plan
path: a metadata-only batch update never rewrites the vector-bearing noun
record (planUpdate staged the unconditional save the update() fix removed).
Seven pins in tests/integration/level-field-shadow.test.ts including the
reporting consumer's exact repro rows; orderBy JSDoc documents the ordering
contract and the announced field-addressing law.
Also: idle PathResolver stats tick no longer logs NaN% every minute (logs
only on new traffic, via prodLog); graph-lsm-* key family recognized as
system resources (kills the per-boot unknown-key warning on provider-backed
brains). Four regression pins in tests/integration/update-write-granularity.
A production deployment's warm report showed metadata: 'unavailable' under a
native metadata provider. brain.warm()'s metadata leg only duck-typed the
built-in JS manager's hydrateAll() method, which a native provider has no
reason to implement.
- MetadataIndexProvider (src/plugin.ts) gains an optional warm?(): Promise<void>
hook, mirroring the existing vector and graph provider hooks. brain.warm()
now checks the active provider's own warm() FIRST, falls back to the JS
manager's hydrateAll() when absent, and reports 'unavailable' only when
neither exists -- never init() as a stand-in, since a native provider's
init() may be a cheap verify rather than a real warm.
- Tests (tests/unit/brainy/warm.test.ts): a live provider instance shaped to
have warm() reports 'warmed' and the hook called with no hydrateAll
fallback; shaped to have neither hook reports 'unavailable' (pins the
honest branch); the unmodified built-in JS manager still reports 'warmed'
via hydrateAll(), unchanged.
Additive scope agreed mid-flight with the native-provider team: a
maintenance-debt observability seam so an operator sees a grind coming
instead of discovering it as a CPU storm.
- New optional maintenanceDebt?(): Promise<ProviderMaintenanceDebt> hook on
all three provider contracts (vector, metadata, graph -- the same three
warm?() lives on). ProviderMaintenanceDebt is fields-all-optional: a
provider reports only what it truly measures (pendingBytes, pendingItems,
lastPassCompletedAt, lastPassOutcome, converging), never an estimate
dressed as fact.
- New public brain.maintenanceDebt(): a pure passthrough -- for each surface
it calls only the active provider's own hook and reports the payload
verbatim, or 'unavailable' when absent. No thresholds, no polling, no
JS-side estimation; the provider owns the numbers, the operator owns the
policy.
- ProviderMaintenanceDebt, MaintenanceDebtReport, and MaintenanceDebtOutcome
are exported from the package root.
- Tests (tests/unit/brainy/maintenance-debt.test.ts): hook present reports
'reported' with the exact payload passed through; hook absent reports
'unavailable' on every surface; mixed surfaces resolve independently of
each other.
RELEASES.md gains the 8.10.1 entry covering both fixes above and this
feature, including the no-hot-retry contract from the prior commit.
A production incident: a native-provider op ground 38-40s inside a transaction,
blew the apply-phase budget, rolled back, and a downstream pipeline hot-retried
the identical operation into a 6-minute CPU storm. Brainy itself never
auto-retried the timeout; the gap was that TransactionTimeoutError only said
"retryable" in prose, with nothing machine-readable for a caller to branch on.
- TransactionTimeoutError gains two typed, always-true fields: retryable
(a later attempt may succeed once the slowness resolves or the budget is
raised) and hotRetryUnsafe (an immediate identical retry re-pays the full
cost that just timed out and can cascade into a CPU storm -- callers must
latch and back off, never loop). context's existing telemetry fields
(timeoutMs, operationIndex, elapsedMs, totalOperations, operationName) are
now documented as the caller's backoff inputs.
- Updated the "retryable" doc-prose sites (transact()'s timeoutMs option,
transactionBudgetFloorMs, Transaction.execute()'s contract) to point at
the new fields instead of bare prose.
- Regression pin (tests/unit/transaction/timeout-never-internally-retried.test.ts):
an execution counter proves the engine never re-drives a timed-out
operation, through both the single-op engine TransactionManager/Transaction
drives for every single-record write, and add()'s upsert-race retry loop
(which must exit on the first TransactionTimeoutError, never treat it like
the lost-insert-race signal it retries on).
- Removed TransactionManager.executeTransactionWithResult -- zero callers
anywhere in the codebase.
Reconciles the vector-index rename to the ruled three-layer naming: the
provider contract's identity field is now a REQUIRED readonly name (was
optional providerId), self-reported and truthful, rendered as
[vector-index:<name>] where the index identifies itself and stamped into
the op-name strings journals already parse (AddToVectorIndex(<name>)).
The built-in JS engine names itself hnsw-js. A runtime provider instance
compiled against the previous optional contract is tolerated - never
crashed on, never silently mislabeled: it stamps unknown-provider and
emits one loud warning naming the missing field. Graph index operations
keep their static names (they never interpolate provider identity), and
no public API exports a provider-routed hnsw-carrying name, so no
deprecation shim is required.
Three pieces addressing the cold-restart-write incident where a production
deployment's first writes after every restart (33-35s each on a cold page
cache) blew the op-count-scaled transact budget mid-batch: every write is
itself a multi-op transaction, so one cold operation consumed the whole
budget, the gate before the next operation tripped, and the write rolled
back atomically - refused, retried, and refused again until the page cache
warmed passively.
- The budget's start-gating contract is now explicit and pinned: it gates
STARTING the next operation, never rolling back completed work for
elapsed time (the shipped schedule since 8.7.0, now stated in contract
JSDoc, guarded by code for operation 0, and enforced by regression
tests). The 30s floor is configurable via transactionBudgetFloorMs for
stores whose cold operations legitimately run long.
- New brain.warm() eagerly loads the vector index, metadata index, and
graph adjacency so first operations after a cold restart run at
steady-state cost. Returns a WarmReport with an honest per-surface
outcome (warmed / probed / unavailable) - never reports a probe as a
warm. warmOnOpen: true runs it during init(). New optional provider hook
warm() on the vector and graph plugin contracts.
- Vector-index transaction op classes renamed from the backend-specific
AddToHNSWOperation / RemoveFromHNSWOperation to backend-neutral
AddToVectorIndexOperation / RemoveFromVectorIndexOperation, stamping the
active backend into the emitted op-name string (AddToVectorIndex(js-hnsw)
vs a native provider's own identity) so journals never misdirect an
operator toward an index that isn't running.
The packed tier goes live inside GenerationStore:
- Two-tier reads: getDelta / readBeforeImage / readGenerationRecords
fall through live-tier → sealed segments (live-tier-wins: a crash
mid-fold leaves a duplicate representation, never a gap). Cold-open
seeds committedRanges from the segment manifest via interval merge —
packed generations resolve without their directories existing.
- repackHistory({timeBudgetMs, batchGenerations}): folds cold
generations (older than the newest 1024) oldest-first into sealed
segments, deleting per-generation directories only after segment +
manifest are durable. Public API + automatic time-bounded pass at
close() (before compaction, so reclaim can drop whole segments);
re-representation only — the sole history transform under the
archival profile. Early stop = consistent prefix, next pass resumes.
- compact(): packed generations reclaim logically in the loop and
physically at whole-segment boundaries via dropSegmentsBelow (the
frozen partial-segments-wait rule).
- generationDigest(g) (D8): deterministic content digest through g —
sealed-segment checksum chain + live-tier delta hashes; O(segments +
live window); RangeError out of range, GenerationCompactedError
below the horizon (a gate can never silently pin reclaimed history).
Four end-to-end pins: asOf answers byte-identical across fold + cold
reopen with folded dirs physically gone; repack+reclaim composition;
digest reopen-stability/divergence/loud-horizon; budget no-op+resume.
First stage of the co-frozen D1+D3+repacking unit: the format core,
self-contained under _generations/segments/.
- seg-<firstGen,20pad>.bgs: append-once packs of consecutive
generations (magic BGS1; frame = u32 len + u32 crc32c + msgpack
[generation, timestamp, delta, records, flags]; flags reserves
compressed-payload evolution without a format break). Sealed
segments are immutable — fold refuses overlap with sealed ranges.
- seg-<firstGen>.idx: DERIVED sidecar (per-generation frame offsets +
per-id generation postings + checksums); lost/corrupt sidecars
rebuild from their segment loudly; a damaged segment (frame CRC
mismatch) fails loudly, never serves wrong bytes.
- manifest.json: the one discovery path — open() reads it and never
lists the packed backlog (the scan-wedge class's cure); refuses a
newer manifest version rather than serving partial history.
- D3 semantics: dropSegmentsBelow reclaims WHOLE segments at
boundaries only and bumps compactedBelow durably; archival-profile
enforcement stays with the caller per the co-freeze.
- D8 rider: digestThroughPacked(g) — deterministic crc32c chain over
sealed-segment checksums (+ frame-level prefix mid-segment),
O(segments), reopen-stable.
Also fixes a cross-adapter contract bug the suite caught: memory
storage's deleteObjectFromPath ignored the raw-bytes store, so
deleteRawObject on a raw-bytes path (fact-log or segment files)
silently no-op'd — deletes now match filesystem unlink semantics.
Six pins. Wiring into GenerationStore (two-tier reads, the repacker,
cold-open manifest path) lands with the rest of the unit before its
release; cortex's fact-record/stamp shapes reconcile the sidecar
keying when they post.
Stage-2 D1 contract item (co-frozen): a fact scan may be slow, never
silent. batches() now races its FIRST pull against
SCANFACTS_FIRST_BATCH_MS (10s, exported; test-overridable) — a wedged
or unreadably slow store produces a loud abort naming the contract
instead of a consumer hanging indistinguishably from progress (the
production shape: a heal against a generations-backlogged brain
wedged silently on the first segment read).
Only the first pull is raced: the bound is time-to-first-batch (proof
the producer is alive), not per-batch pacing, and it runs only while
a pull is pending — consumer think-time between pulls never counts
against the producer (pinned).
Three pins: wedged-store loud failure within the bound, healthy scan
untouched end-to-end, slow-consumer immunity.
clearInterval() stops future heartbeat ticks but not one already in
flight: a straggler tick past its ownership guards could land its
atomic lock rewrite AFTER releaseWriterLock()'s unlink, re-creating
the lock file as a phantom that blocks the next writer until the
stale TTL expires (~60s) — the pool-eviction reopen case. Found as an
ENOENT heartbeat warning during benchmark teardown; the quiet variant
is the harmful one.
releaseWriterLock() now awaits the in-flight tick (tracked per tick,
self-clearing) before reading/unlinking, so a straggler's write always
lands BEFORE the unlink and gets removed with everything else. The
heartbeat's ENOENT is also now benign-by-contract (lock or directory
removed under us — the next acquire recreates it); other errors stay
loud.
Pin: straggler-past-guards simulation — lock file absent after close,
directory immediately claimable (fails on the undrained code).
flush() is durability work: it must cost what the current window's
deltas cost, never what the history backlog costs. Under adaptive
retention the byte budget derives from free memory, so bulk-load
pressure shrank the budget exactly at peak write volume and flush paid
actual reclaim inline — a production deployment measured single writes
blocked 25-191s behind reclaim-on-flush.
- flush() no longer calls autoCompactHistory(); close() is THE
auto-compaction site (already ran there; now alone).
- Every auto pass is time-bounded (CLOSE_COMPACTION_BUDGET_MS = 5s):
reclamation is oldest-first, so an early stop is a consistent prefix
and the next pass resumes. Explicit compactHistory() gains an
optional timeBudgetMs for caller-chosen maintenance windows.
- Documented trade stated where operators read: a long-lived writer
that never closes accumulates history until its next explicit
compactHistory() — predictable writes, explicit maintenance.
Pins: flush-never-reclaims + close-reclaims-durably (db-mvcc), bounded
pass stops-then-resumes as a consistent prefix (generationStore unit).
Four fixes from a consumer conformance report, one root disease — two
field-resolution regimes where there must be one:
- The delete/update aggregation hooks fed the engine a partial entity
view (type/service/data/metadata only), so a reserved-field groupBy
(subtype, visibility, ...) resolved to a nonexistent group on the way
down: counts drifted upward forever after deletes, and updates moving
an entity between reserved-field groups double-counted. The hooks now
pass the full-fidelity view via entityForAggFromRawRecord (every
reserved field top-level, mirroring the add path); the update sites
pass the full get() view instead of a hand-rolled subset.
- Aggregation source.where resolved fields only against the custom
metadata bag, so where on a reserved field silently matched nothing.
The matcher now resolves each filtered field through
resolveEntityField — the same single source of truth groupBy uses.
- removeMany() with no usable selector (bare array passed positionally,
empty params, ids: []) resolved successfully having deleted nothing.
All three now throw; the two legacy tests that pinned the silent
no-op as 'graceful' now pin the refusal.
- find() where keys accept both spellings: a metadata.-prefixed key
falls back to its flattened spelling when the prefixed one is not
indexed (metadata is flattened at index time). A literal nested custom
key named metadata still wins when indexed as spelled.
Five regression pins in aggregate-reserved-fields.test.ts (4 of 5 vary
red on the unfixed code).
Under default adaptive retention, every flush() recomputed total history
bytes by walking EVERY committed generation's delta — O(all generations)
with disk re-reads past the 4096-entry delta-cache bound. On a production
brain with 70,000+ accumulated generations this turned every write into a
full-tail scan (60-100s writes, escalating with history growth), even
though the free-RAM budget never tripped and nothing was ever reclaimed
(SELF-GENERATIONS-GROWTH).
historyBytes() now maintains a running total: seeded by one walk on first
use, then updated incrementally at both commit paths (+bytes) and the
compaction reclaim loop (−bytes), dropped on reopenAfterRestore. The
adaptive retention check on every flush is O(1). Invariant regression-
pinned: running total ≡ fresh walk through transact commits, single-op
group commits, and compaction.
New brain.historyStats() (exported HistoryStats): read-only generation
count / bytes / generation+timestamp range / horizon / retention mode /
effective budget — the one-call per-brain fleet audit for retention
exposure.
The post-import background deduplication pass (a merge-DELETE writer,
debounced ~5 minutes after import) had four lifecycle defects:
- enableDeduplication: false did not gate the background schedule — an
import that explicitly opted out could still have entities auto-removed
minutes later. The flag now gates both the inline and background passes.
- Each import() constructed its own coordinator-owned deduplicator, so the
debounce never spanned imports (N imports = N delete timers). The brain
now owns a single lazy instance (getBackgroundDeduplicator).
- close() never cancelled pending dedup; a delete pass could fire against
a closed brain. close() now cancels it first.
- The 5-minute timer held the process open (exit-hang class); now unref'd.
Four regression tests pin the contract (background-dedup-lifecycle);
import guides document that false disables both passes.
- New src/utils/osLimits.ts: reads RLIMIT_NOFILE (soft/hard, from
/proc/self/limits) and vm.max_map_count at open — once per process,
Linux-only, measurement-only — and warns loudly when either sits below the
pool-scale floors (soft NOFILE < 65536, max_map_count < 262144), with the
exact raise commands. On stock defaults the failure otherwise arrives as
EMFILE or a failed mmap deep inside an index open, long after the cause
stopped being visible. An unreadable limit produces NO warning — no
measurement, no claim — so non-Linux platforms stay silent.
- Exported for ops doors: checkOsLimits() returns the full OsLimitsReport;
floors exported as constants. Wired fire-and-forget in performInit after
storage init; the check can never affect open.
- Unit tests pin the parser (incl. 'unlimited'), the floor thresholds, the
null-never-warns rule, and the off-Linux silent path.
- acquireWriterLock now CLAIMS with an atomic create-exclusive write (O_EXCL)
inside a bounded retry loop: two processes racing an absent lock can never
both succeed (the old read-then-tmp-rename flow let the loser keep running
unlocked, silently). An EEXIST loser re-evaluates and either throws loudly
with the winner's details or performs a verified stale-takeover (re-read
before unlink so a lock that changed hands mid-deliberation is never
clobbered). Exhausted contention fails loudly instead of degrading into a
lockless open.
- BRAINY_WRITER_LOCKED passes through init() unwrapped: the error documents a
machine-readable contract (err.code + err.lockInfo with the holder's
pid/host/heartbeat), but init's blanket error wrapping stripped both,
leaving consumers a message to regex against.
- Two contract tests added: stale-foreign takeover installs OUR lock via the
atomic claim; the conflict error carries code + lockInfo at the public
init() surface.
- The transact apply budget scales with the batch — max(30s, opCount x 2s) —
or is exactly the caller's new TransactOptions.timeoutMs. A flat 30s cap
silently limited honest bulk work to ~15 operations on network-attached
storage (~2s/op measured in a production import) while looking generous for
small batches. Internal batch paths (removeMany chunks) get the same
scaling via the shared transactTimeoutBudget helper.
- TransactionTimeoutError now reports the operation it stopped at as i/N with
the operation's name, elapsed vs budgeted time, and states the batch rolled
back atomically and is retryable; context carries the fields
programmatically.
- The transact operational envelope is documented (optimistic-concurrency
guide): budget math, chunking with ifAbsent idempotency, when to reach for
addMany vs transact, and the precompute pattern (embedBatch + per-op
vector) that keeps inference out of the commit path.
- Embedding claims made honest per an end-to-end probe of the built dist:
batch and single embedding paths produce bit-identical vectors and a
vector-supplied add is fully searchable, but batch throughput on the
default WASM engine measures comparable to sequential (~160ms/text) — the
5-10x speedup claim in the addMany JSDoc is replaced with the measured
reality and the actual win (inference outside the budgeted write path).
- New public API: walks every canonical relationship record, queries the same
read path applications use (related() with all visibility tiers included),
and classifies every discrepancy into its failure family: missingFromReads
(records the read path omits — stale adjacency), danglingEndpoints
(endpoint entity gone — the partial-delete scar class), readOnlyVerbIds
(read-path edges with no canonical record — ghosts). Design-hidden
internal/system edges are counted separately so intentional hiding is never
misclassified as loss. Counts exact; example lists capped with an explicit
truncatedExamples flag; loud narration on incoherence; mutates nothing.
- Classification core lives in src/graph/graphAudit.ts behind injected seams
(canonical walks + the end-to-end read) so the discrepancy logic is testable
in isolation; brainy wires the seams to storage walks and related().
- getNounIdsWithPagination now refuses an undecodable resume cursor loudly —
the third and final pagination walk brought under the 8.5.2 cursor contract.
- Types exported: GraphAuditReport, GraphAuditDiscrepancy. Docs: the
inspection guide gains the audit -> repair -> audit verification ritual.
- Aggregation backfill walks build into a STAGING map and swap in atomically
on completion. A mid-walk failure drops the staging map — the previous live
state keeps serving, the aggregate stays flagged pending, and the storage
error surfaces to the failing query. Previously the walk wiped live state
before a scan that could throw, never cleared the pending flag on failure,
and re-ran a full walk on every subsequent query: a silent wipe/walk/throw
loop at the caller's retry rate.
- Failed walks are latched: retries within a 30s cooldown rethrow the recorded
error instantly instead of re-walking, so a tight caller-side retry loop
costs one loud error per query, never a full store walk per query.
- Persisted aggregation state is stamped with the store's committed generation
at flush; reopen adoption requires stamp equality. Stale state (unclean
shutdown) or over-counting state (a log truncation on a copied store pulled
the watermark back) triggers exactly one loud rescan, never a silent adopt.
- The backfill/adoption path narrates: adoption decisions, walk start/finish
with counts and duration, and failures all log by default.
- getNouns/getVerbs refuse a supplied-but-undecodable pagination cursor with a
loud error instead of silently restarting the walk at offset 0 (which
re-served page 1 forever to any while(hasMore) caller).
- The graph cold-load verb walk aborts loudly on a missing or non-advancing
cursor with hasMore=true.
- A versioned index provider whose generation is AHEAD of the committed
watermark (torn copy / crash-recovery truncation) is now named loudly at
open, alongside the existing behind-direction message.
- AggregationIndex: defineAggregate before init no longer forces a backfill.
init reconciles instead of clobbering: the app definition wins, persisted
state is adopted on hash match, a write landing pre-adoption forces an exact
rescan, and a successful state load clears the backfill flag. New ready()
settles every adoption decision before query paths consult backfill state.
- brainy: backfills are single-flight and batched. Concurrent queries share one
store walk and every pending aggregate fills from that same walk; the old
behavior let each concurrent query wipe the others' partial state and start
its own full walk, so a store under steady aggregate traffic never converged.
queryAggregate also waits for persisted definitions before deciding an
aggregate does not exist.
- paramValidation: recordQuery is telemetry-only. The duration-based cap
ratchet (x0.8 per recorded query while lifetime-average exceeded 1s, floored
at 1000 - below the documented 10000 auto floor, reported under a stale
basis label) is removed; the cap comes from its construction-time basis or
explicit overrides alone.
- storage: __aggregation_* and singleton system keys (brainy:entityIdMapper)
are recognized before the unknown-key warning fires; routing is unchanged.
- docs: find-limits cap-immutability note, aggregation reopen/adoption
semantics, RELEASES.md 8.5.1 entry.
setTimeout can fire a few ms early under load (timer coalescing) — a
10ms sleep measured 9ms and failed the >=10 assertion, tripping a
release gate. Sleep 25ms, assert >=20: the test verifies time is
MEASURED, not the OS timer's precision.
