A prerelease the other engine devDeps from our own registry while the two
are being proven together must never reach npmjs: --source-only tags, lets
CI publish to The Source, creates the release page, and SKIPS the storefront
publish, the cross-registry verification and the docs push — loudly, at
each step. Refused for a non-prerelease version: a public floor always ships
byte-identical on both registries.
The pin wrote a post-flip row, abandoned the brain as crashed, and then
deleted the row's canonical bytes to prove the first post-flip boot folds
BOUNDED above the flip's stamp. Between the write's ack and the abandon sat
the store's 50ms pending-flush timer: on a loaded box (the plant lane) the
flush won, barrier-synced the row and advanced the checkpoint over it — and
the fold, correctly bounded, did not restore bytes the test had destroyed
after they were stamped durable. Green locally, red on the plant: the
engine was right, the pin was timing-dependent.
The crash is now armed at exactly singleop-after-fact-append: the fact is
appended and at-ack synced, no flush is ever scheduled, the stamp provably
still reads the flip's value when the pre-flip bytes are dropped, and the
post-flip row's bytes — which lived only in the pending tier's RAM — are
lost for real, not synthetically. The reopen must re-materialize it from
its fact and must not restore the pre-flip row.
A validateInvariants() that threw was re-synthesized by the host's catch as
heal:'rebuild' with serving:false — a rebuild lever one transient exception
away, while the native provider's own composer reports the same event as
heal:'none'. Two components disagreeing on what a thrown check means is how
a flaky probe becomes an outage. Both now agree: the report is named
('validate-invariants-threw'), loud (healthy:false, the error in detail),
unverified — and it never buys a rebuild and never withholds serving; the
provider's serving verdict is the provider's to compose, not inferred from a
probe that failed to run. The synthesized report also carries the provider's
name instead of 'unknown'.
RepairFamilyReport gains `missing: {count, sample}` (an exact count plus a
capped id sample — a verdict, not a dump), `rebuilt` (a full generational
rebuild ran, as opposed to an incremental heal) and `reason`, aligning the
receipt's shape with the provider-side health report.
Pinned in tests/unit/validate-invariants-delegation.test.ts.
The storage-level unfiltered getNouns()/getVerbs() walks enumerate every
tier, but their totalCount reported the user-facing scalar, which skips
system/internal records on the write path — so a derived-index coverage
ledger comparing its posted count against that total would read
"over-posted by N" on every store with a VFS. This adds the ledger's real
denominators:
- totalNounCountAll / totalVerbCountAll: +1 for every new canonical record
regardless of tier, −1 for every PROVEN delete (record read, or the
caller's prior image), persisted in counts.json beside the counted
scalars, recomputed by the sanctioned recount (rebuildTypeCounts).
- The unfiltered storage-level totalCount is now the ALL scalar and is
never clamped: Math.max(scalar, scanned) could only move a scalar up, so
an inflated counter hid forever; a divergence is now visible and healed
by repairIndex().
- A delete that cannot prove the record existed never decrements on faith:
it marks the ledger SUSPECT (persisted, narrated once per session) and
the recount clears the flag with proof.
- getCanonicalCounts() on StorageAdapter (optional) exposes {counted, all}
per family plus the suspect flag — O(1), no I/O.
- A counts.json written before the ledger existed derives both scalars
once from the canonical id tree at open and persists them; absent keys
are a legacy file, never a zero.
User-facing getNounCount()/getVerbCount() are unchanged.
Pinned in tests/integration/canonical-count-ledger.test.ts (5 laws).
remove() guarded its index legs with 'if (metadata)': a row whose canonical
metadata was unreadable at delete time kept its postings forever, silently
(the leak class a partner audit confirmed at this exact site). Closed at
all three provider shapes: a provider exposing the id-keyed removal
contract (removeEntityById, arriving with the accelerator's next minor)
gets exact per-entity retraction; the JS index gets id-keyed cleanup
(deleted bitmap + id mapper — field stats reconcile at rebuild), narrated;
a native provider without the contract is NEVER called metadata-omitted
(that path walks the store's value space) — its skip is narrated and
tracked in the degraded set for repairIndex, never silent. The pre-reads
are torn-tolerant: a torn record is deletable (the delete is the cure).
Pinned: a metadata-less row with live postings deletes cleanly and leaves
the query universe; the delete-family suites stay green alongside.
repairIndex() now returns a RepairReport: one row per repair family
(orphaned containers, count rollups, VFS containment, metadata corruption,
write quarantine, each provider's invariant pass, degraded-read state) with
checked / healed counts and an explicit skip reason for anything not run —
no silent rows. A summary line narrates families checked and heals applied.
This is the receipts half of the graph-trust program's ask: a repair that
cannot show its work per store is a repair nobody can audit. Pinned: a
healthy store yields a complete zero-heal receipt with every family
accounted; a manufactured pre-8.3.1 ghost container appears in the receipt
as a counted heal. Additive: void-callers are unaffected.
A production store acked writes while readback served empty for fifteen
minutes. The brainy half: the lazy readiness gate had exactly one caller —
find() — while related() and every VFS path served straight from providers
that init had deferred (disableAutoRebuild on a large store). A false
health verdict from the accelerator pulled the trigger; the unguarded read
surfaces were the gun.
Every index read funnels through three helpers; the gate now lives at those
choke points, so any first read on a cold instance waits for the build and
then serves truth — the fast path after the latch is one boolean. The lazy
build's start is narrated through the production logger, never the
silent-suppressible console: fifteen silent minutes taught that line.
Pinned with the production shape: related() as the first-ever read on a
fresh lazy instance serves the relation; a filtered find serves the row.
Two guards for every gate lane, born from the 2026-08-13 lost-day ledger.
gate-preflight.sh refuses a lane on a machine that cannot be trusted to
produce honest numbers — co-tenant processes named by pid and command, load
average, CPU governor, disk floors — one FATAL line per violation so the
operator can act from the message alone. vitest-verdict-check.sh refuses a
suite log that cannot be trusted as a verdict — missing or mismatched
summary counts, files that never executed (a truncated run once read as
green from three files of ninety-nine), and worker-pool death signatures.
Both verified live: the preflight correctly refuses this workstation naming
its actual offenders; the verdict guard passes/fails five fixture shapes
(clean, wrong-count, truncated, worker-death, no-summary) and both CLI
modes. Wire-up into the CI lanes rides the runner program.
A production brain's first process boot after a live authority flip looked
hung and was restarted three times mid-recovery — three defects with one
scene. (1) THE FOLD MATERIALIZED THE LOG: peekFactsAbove(0) decoded every
fact into one array (GBs of after-images on a ~7k-fact log, a GC storm, a
starved write lane). The fold now STREAMS one segment-batch at a time —
memory is one segment at any log size — with structural ordering asserted
loudly. (2) THE FOLD WAS SILENT UNTIL DONE: minutes of boot work with zero
narration is what invited the restarts. It now announces itself BEFORE the
work ('do not restart, the fold is finite') and prints progress every
thousand facts. (3) THE CHAIN COULD ONLY ARM AT A CRASH: a live mid-session
flip left the fold checkpoint unfounded, so the brain's first unclean boot
paid a whole-log fold. Adoption now founds the checkpoint AT THE FLIP — one
paged full canonical barrier (bounded memory), then the stamp — so bounded
recovery holds from minute zero for every store that flips, at any size.
Pinned: a non-fresh flip stamps immediately; the first post-flip unclean
boot folds bounded (an unflushed at-ack fact above the checkpoint is
restored; a barrier-covered row below it is outside the fold). Kill matrix
and both adoption suites green alongside.
The plant's integration lane caught it twice: the fence's startedAt-strict
comparison turned the documented same-process warn-and-take-over path (two
instances in one Node process — the server-restart test pattern, and the
shared-default-store pattern across test files) into a flush-killer: the
first instance's background flushes latched dead while its own process held
the lock ('PID N no longer holds the lock — it is now held by PID N').
Ownership is per-process: pid + hostname. startedAt stays in the lock for
observability but not in the fence — it protects nothing (a pid-recycled
successor's victim is a dead process that runs no fence checks) and it
convicted the innocent. Pinned: a same-process re-open leaves both
instances' flushes working; the cross-process eviction pins unchanged.
Verified under the lane's exact command: 102/102 files, 850 passed, exit 0.
Seven micro-budget tests were calibrated on one fast desktop and failed on
other honest iron with zero functional failures (bisect-proven pre-existing;
David-waived for 10.1/10.2 with this recalibration filed as the cure). Every
budget is now at least 3x the worst measurement observed across three
machines, each with a comment naming its calibration basis; the find-unified
micro-comparison of two sub-millisecond timings becomes a ratio assertion
(absolute equality of microsecond pairs can never be stable). The
inference-bound trim-history correctness test gets a timeout covering its
slowest observed run (174s) — its assertions are exact and untouched.
These remain order-of-magnitude guards; real perf enforcement lives in the
dedicated perf lanes with iron-specific budgets, per the gate-speed standard.
Known non-test artifact, documented not hidden: on slow-inference machines a
minutes-long awaited-embed loop can trip vitest's worker-RPC 60s tolerance
('Timeout calling onTaskUpdate') — all tests pass, vitest exits 1 on the
unhandled orchestration error. The CI lanes on faster iron exit clean; if a
lane ever trips it, the test moves to deterministic embeddings (its
assertions are size-bookkeeping, not embedding quality).
The production dev-store split-brain (two live writers alternating a store's
id-mapper between two internally-consistent truths), cured at all three of
its roots. (1) STALENESS REQUIRES PID-DEATH: the old rule evicted on
heartbeat age alone, so a >60s event-loop stall (debugger pause, GC, heavy
sync work) handed the lock to a second opener while the first kept writing;
a live process is now never auto-evicted — a wedged-but-alive holder is the
operator's call via {force:true}, and the heartbeat stays for observability.
(2) THE CLAIM IS ATOMIC: writeFile(wx)'s open→write→close left an empty-file
window a concurrent opener could read as torn, unlink a LIVE claim, and take
the lock; the claim is now tmp-write + hard-link — the lock appears with its
full contents in one step. (3) THE FENCE: every flush commit and transact
barrier verifies lock ownership first (one small read per window) — a
forced-out or lock-deleted writer fails typed (BRAINY_WRITER_FENCED) before
a single staged byte or manifest advance, instead of writing on unaware.
Pinned: live-with-ancient-heartbeat refuses typed; dead-PID self-clears
narrated; a forced-out writer's flush and transact both fence, advancing
nothing. Requested by a downstream team as single-writer guard or loud
lockout — this is both.
Two consumer-driven cures sharing one stamp. (1) TX-LOG ORIGIN: engine-
originated commits stamp an optional origin on their tx-log entry AND the
commit fact's meta — 'system:embed-landing' (the deferred vector landing),
'system:adoption-backfill' (baseline re-commits), 'system:reconcile'. A
downstream activity feed showed a double tick because the landing commit was
indistinguishable from a user save, and the consumer rightly refused a
time-window collapse as a quiet loss; feeds now filter on fact. User writes
stay unstamped — absent origin is the user shape, every existing consumer
unchanged. (2) reconcileLogDivergence(id, {attest}): the human's door for
log-live-canonical-absent, the one class adoption refuses by design because
a lost-tombstone deletion is indistinguishable from canonical loss.
'deleted' mints the missing tombstone (history keeps the earlier live
record); 'restore' folds the log's only copy back into canonical; wrong-
class calls refuse typed with nothing written. Loud, narrated, single-row,
origin-stamped. From a production adoption's one surviving divergence.
A production brain with a 12.7k-row pre-log baseline advanced exactly 800
rows per adoptLogAuthority() call (a five-pass ceiling × the oracle's
200-row listing cap), refused the flip, and sat tree-authoritative for
hours across restarts. The bound was sized for drift, never for a baseline.
Now: the adoption path runs the oracle uncapped so ONE scan yields the
ENTIRE curable set, every pass cures all of it, and the loop runs to
completion with the no-progress guard as its only stop. Pace rides the
write path (one full-brain scan amortizes over thousands of cures, not two
hundred): 1,000 drifted rows adopt green in one call in ~10s. Progress is
narrated for a live operator. The wire report keeps its 200-row cap.
Pinned: a baseline above the old ceiling adopts green in a single call.
Two defects with one root, found by the fold-checkpoint work's first
integration gate. (1) THE RACE: restore() never quiesced the generation
store, so a background flush could write into _system/ while the swap was
removing it — observed as ENOTEMPTY mid-swap when a checkpoint stamp landed
between readdir and rmdir. The swap now runs inside the store's exclusive
section (runStateReplacement): flush timer disarmed, pending tier and
checkpoint accumulator discarded BEFORE any directory moves. (2) THE
INHERITED ASSERTION: a snapshot carries its source brain's clean-shutdown
marker and fold checkpoint, but the restored files were bulk-copied without
per-file fsync — the inherited stamps would suppress exactly the recovery
fold that cures a post-restore power cut. reopenAfterRestore now deletes
both stamps before reopening: the open treats the store as uncleanly shut,
folds the restored log into canonical, barrier-syncs what it re-applied,
and stamps fresh — the restored state is durably founded at restore time
instead of borrowing assertions about bytes this disk never synced.
Pinned: restore under in-flight traffic completes; the pre-restore stamp
does not survive; the post-restore stamp is the reopen fold's own, at the
restored watermark.
The fold checkpoint (_system/fold-checkpoint.json) is stamped strictly after
a canonical-sync barrier over every live entity touched since the last stamp
(syncEntityCanonical: ids → canonical paths → fsync; an absent file fsyncs
its parent directory so deletes are as durable as writes). An unclean open
under log authority now folds only (checkpoint, head]; the chain bootstraps
at an empty brain's adoption (three-phase hooks around adoptLogAuthority) or
at a brain's first whole-log fold — existing brains converge at their first
crash with zero regression. Rollback restores sync immediately; abort paths
feed the barrier; a failed barrier retains the old bound (bigger fold later,
never a lost write). Five structural pins including boundedness itself.
Also: the production-shaped write-flow gate leg (mixed traffic racing
flushes, crash mid-traffic, every ack survives — from a consumer-reported
gate miss), and two release-ceremony cures (tag-first push so the publish
never queues behind the release commit's CI run; raw-curl npmjs shasum
probe with propagation grace instead of a one-shot false divergence).
An adopter's full suite found two v2 write-path defects on fresh brains,
reproduced with stacks; both cured and both pinned with their exact
production shapes:
1. PAD-FRAME CONSTRUCTIBILITY: a single msgpack bin filler steps its
header by one byte at each size class (bin8→bin16→bin32), leaving one
unreachable payload size per boundary — the sealer requested a
291-byte pad, the encoder threw 'not constructible', and sync() died
whole. Construction is now TOTAL: the class-boundary holes bridge with
a trailing fixint beside the bin ({bin(n)} ∪ {bin(n)+fixint} covers
every size ≥ minimum). Pinned exhaustively: every size from the
minimum through a full sector plus boundary spill constructs
byte-exact and decodes as reader-invisible filler.
2. THE NON-MONOTONIC REFUSAL LOOP: the append-failure compensation
rewound the generation counter on ANY throw — including a covering
SYNC failure after a SUCCESSFUL append. The log carried generation N
while the counter re-minted N, and every later append refused
'non-monotonic (N ≤ head N)' — the write path wedged in a refusal
loop through deferred-embed retries and flush backoff. The
compensation now splits by phase: an append failure (log never took
the fact) fully compensates — un-buffer and rewind; a sync failure
after append earns the rewind ONLY if the appended fact is provably
dropped, otherwise the generation stays consumed and buffered — the
counter never re-mints a number the log may carry. Pinned: an
injected one-shot sync failure fails its write loudly and the very
next write mints fresh and succeeds, with the log scanning strictly
ascending end to end.
Also probed against the adopter's carried report: the 9.0 vfs.rename
stale-ghost shape does NOT reproduce on this head (old path cleanly
unresolvable on exists/stat/readdir after rename).
Gates: unit 2067/2067 (160 files) · integration 833 (97 files) ·
conformance 36/36.
A first adopter's namespace migration went 341 red on one class: the
never-carried-field refusal firing on CORRECT filters against fresh and
sparse stores — a freshly provisioned tenant refused its own first
filtered read, with the did-you-mean built for typos firing hardest on
day-one stores where nothing is wrong.
The ruled cut: a WHERE filter naming a field no row carries is SERVED
OPERATOR-TRUTHFULLY — eq/in/range/contains answer [] (nothing carries
it, nothing matches); ne and exists:false answer ALL rows (the equally
true complement — a blanket empty here would be silently wrong, which is
why the simpler cut was rejected); exists:true answers []. Served from
the field registry, with the did-you-mean demoted to a once-per-field
WARN. orderBy and genuinely ambiguous addresses KEEP their hard typed
refusals: no truthful order exists over an uncarried field, and
ambiguity is a contract error while absence is data.
Mechanics: the negative operator absorbs the FIELD_NOT_INDEXED throw as
its empty exclude set (the clause-level catch correctly zeroes positive
operators only); the egress matcher already agreed. Plus the
provider-seam belt: a field refusal thrown by a replacement metadata
manager is normalized to THIS package's UnresolvableFieldError at every
filter call site — one class identity for consumers, instanceof works
(a first adopter's cross-package finding).
Conformance: tests/conformance/sparse-store-cut.test.ts — the shared
operator rows both engines run (positive-empty, negative-all,
fresh-tenant day-one, orderBy refusal kept, compound composition).
Gates: unit 2065/2065 · integration 832 · conformance 36/36.
The last rung of the default-flip ruling: with the sentinel exemption in,
real production-shaped brains still refused adoption over state-differs
mismatches the backfill could not cure — rows written before the
hydration law carry denormalized wrapper fields that disagree with their
own metadata leg, and the previous as-is identity re-commit PRESERVED
that drift, so the oracle re-flagged it every pass and the flip never
happened. In practice the crash-safe default reached zero existing
brains: the exact outcome the hold ruling forbade.
The cure: the backfill now rewrites canonical in the LAW SHAPE — exactly
the wrapper the log's reconstruction produces (denormalized enumeration
fields derived from the metadata leg, which is their authority under the
field-addressing law; the embedding floats ride through byte-identical;
adjacency residue keeps its own rebuild path). The oracle then verifies
the rewrite before the flip — the same safety, no operator chore.
Log-ahead divergence classes (a log the witness denies) still refuse
loudly, exactly as before.
Classification note for the record: the flagged uuid-v7 rows postdate the
fact log's introduction, so they classify as state-differs (in-log,
drift-shaped) rather than pre-log — both classes ride the same backfill.
Pins: a manufactured depot-shape drifted wrapper adopts green with floats
preserved and metadata intact; log-ahead still refuses typed.
Gates: unit 2065/2065 · integration 832 · conformance 31/31.
The release-holding finding from the joint gate's six real depot brains:
the adoption path's positive-int mint check false-flagged the reserved
VFS-root sentinel (the all-zeros UUID, minted int 0 BY CONSTRUCTION at
genesis on existing brains) as a corrupt mint — so every existing brain
refused log-authority adoption and stayed on the old lossy-under-power-cut
durability, defeating the release's headline crash-safety exactly where
it matters most.
The exemption, at both mint seams (the host's minter thunk and the fact
log's encoder guard): int 0 is legal iff the id is the reserved root;
zero for ANY other id remains a corrupt-mint refusal naming the reserved
exception. The codec's u64 layer already tolerated 0 — only the guards
over-refused.
Pins: adoption goes green on a brain whose VFS root carries int 0 (the
depot-brain shape, previously refused) · a non-root zero still refuses
typed at the mint seam — held at the seam itself because a full write
SELF-HEALS a poisoned zero (the index cycle re-mints before the fact is
written, which is the correct outcome and was verified in the pinning).
Gates: unit 2065/2065 · integration 830 · conformance 31/31.
The quiet-loss cure regressed recovery: the new typed torn-record error
was correct at identity-read time but threw inside init-time recovery
walks, killing opens that previously survived. The boundary, redrawn:
- IDENTITY READS (get-by-id of a specific record, CAS blob point-get):
typed TornRecordError, unchanged — a caller who asked for THAT record
can act on the answer.
- SET-SHAPED READS AND WALKS (enumeration, pagination, batch hydration —
the paths recovery rebuilds and finds page over): HEAL PAST the torn
victim. The adapter's loud floor (error log + counted gauge) fires at
the encounter; the walk serves the remaining rows. One crash casualty
can no longer kill every query on its shard — or the open itself.
- WRITES OVER TORN RECORDS ARE THE CURE: the save path's read-merge, the
commit path's before-image capture, and the operations' rollback
captures all treat a torn prior as the create sentinel, narrated — the
incoming bytes replace the unreadable ones, and history for the id
honestly restarts at that generation. Corruption can never block its
own heal.
- THE NaN SOURCE: torn mapper state (nextId/entries carrying garbage)
discards with narration and re-derives via the existing rebuild path;
the mint gains a source guard healing a non-integer counter from the
live map. The reopen and first-write RangeError shapes are dead at the
source, both authority branches.
Pinned with the exact fault-injection scenarios: a torn entity record
(including the VFS root) no longer kills the open — walks heal past it,
the keeper rows serve, and the identity read of the victim itself is
typed-or-healed; a torn mapper reopens and mints sanely on the first
post-recovery write.
Gates: tsc 0 · unit 2065/2065 · integration 828 · conformance 31/31.
THE DEFAULT FLIP (ruled on proven evidence — at-ack survived 301/301
acked-writes-through-power-cut in block-layer fault injection; deferred
tree authority demonstrably loses flush-covered acks): a brain with NO
stored authority artifact now ADOPTS LOG AUTHORITY AT OPEN. The oracle
gates the flip exactly as the guarded adoption path always did — curable
divergences baseline-backfilled, the flip lands ONLY on a green verdict —
and a brain that cannot verify STAYS tree-authoritative loudly, with the
refusal recorded on the switch artifact so subsequent opens are cheap.
config logAuthority: 'defer' is the explicit documented opt-out (no
automatic adoption; declared flush-window loss; adoptLogAuthority() flips
later). A stored artifact always wins. RELEASES.md carries the posture.
Two standing .fails debt pins FLIP TO HOLDING under the default: the
at-ack crash-survival gap and the ack-at-log durability target — both now
permanent asserted truths, not aspirations.
POWER-CUT THROW SITES (fault-injection findings, brainy-alone config):
- A manifest-listed-but-unloadable column segment QUARANTINES at
discovery (loud once, counted always, quarantinedSegments() exposed for
the heal) and the field serves its remaining segments DEGRADED — never
a raw throw killing every query on the field. Real storage faults still
propagate untouched.
- Torn generation artifacts (NaN/garbage in manifest or counter) DISCARD
with narration at the store's open and recovery re-derives — plus a
defensive finite-integer guard at the init consumer. Never a RangeError
killing an open.
THE LOUD TORN-RECORD CONTRACT: an existing-but-unparseable stored record
now surfaces as a typed, counted TornRecordError on every entity-read
surface (including fifteen previously-blind per-item batch catches);
ENOENT stays clean-absent; artifact readers with designed absent-recovery
keep null-tolerance behind the loud floor. Disk corruption can no longer
read as silent data invisibility.
Suite migration: the default's pins inverted deliberately, generation
baselines made relative, quarantine-contract pins rewritten to the ruled
behavior.
Gates: tsc 0 · unit 2065/2065 (159 files) · integration 826 (93 files) ·
conformance 31/31 · kill-matrix 15/15 · torn-open guards 2/2.
An internal cross-engine fault-injection run (frozen-platter power-loss
capture) surfaced three release-gating findings; each cured in its owning
layer, each pinned:
1. WHOLE-LOG REPLAY ON UNCLEAN OPEN (the big one): log-authority replay
only covered facts ABOVE the manifest — but live canonical entity
writes are tmp+rename without per-file fsync, and the group-commit
flush syncs staging + manifest, never the live tree. Power loss could
therefore vaporize acked canonical bytes BELOW the manifest while the
log held every fact scan-clean (measured: 299 of 301 acks lost).
Now: a clean close stamps a clean-shutdown marker (fsynced, written
last); every open consumes it; an UNCLEAN open under log authority
folds the ENTIRE log into canonical — whole-entity after-images make
the re-apply idempotent and byte-safe. Zero cost on the happy path;
crash recovery pays one narrated fold. Recovery is replay: a crash is
just bigger lag.
2. TORN WRITER LOCK: power loss legally leaves the lock file present but
empty; the parse failure read as 'no holder' while the O_EXCL claim
EEXISTed forever — a PERMANENT lockout no staleness check could clear.
An unparseable lock is stale by definition (no live holder has one):
unlink loudly and re-loop; a racer rewriting a valid lock first wins.
3. PAIR GUARD: flush() called metadataIndex.stampWatermark unguarded;
a replacement metadata provider without the method killed the pair at
first flush. All three stamp calls are optional-chained — a missing
stamp is a verdict-side rescan, never a flush crash.
Pins: whole-log fold restores rows vanished below the manifest ·
clean-shutdown marker lifecycle (stamp/consume/re-stamp) · torn-lock
recovery with a fresh write after · stampless-provider flush.
Gates: unit 2055/2055 · integration 824 · kill-matrix 15/15.
An internal report from cross-engine write-path instrumentation: with
individual writes slower than the idle window (a contended disk), every
inter-write gap looked idle and fired a background full flush — 15 extra
flushes during 100 contended adds, amplifying the very pressure that
slowed the writes. The law now: an idle fire landing within the spacing
floor of the last flush DEFERS to the floor boundary instead of flushing;
the floor is min(interval, 10× the CONFIGURED idle window) — scaled to
caller intent (a tiny idle window keeps fast idle-driven durability;
default 2s/30s config gets a 20s floor), derived from the configured
idle, never from a deferred re-arm delay (which would compound into
runaway deferral). Deferred is never dropped: a lone write on a
then-quiet store still persists at the floor without any further write
arriving.
Pins: the contended-shape pin (six slow-spaced writes fire ≤2 idle
flushes, not one per gap; then still persist) + the original quiet-store
idle pin unchanged. Unit 2055/2055.
The generic reprojection engine (pure TS; the twin of the native
implementation — same frozen contract, one shared conformance intent):
register any ProjectionAdapter; advance(family, {budgetMs}) folds facts
from the adapter's own watermark to the head in installments ≤50ms with
real macrotask yields; foreground door traffic bumps the DoorSignal and
an in-flight advance yields within one installment ('preempted');
advanceAll round-robins families fairly. swap(family, buildAdapter) is
the doors-open migration primitive: the OLD projection keeps serving
while the new one builds beside it, the flip is atomic at parity, and a
concurrent second swap refuses typed. A fact the fold cannot apply
(typed ProjectionApplyError) is QUARANTINED — skipped, ledgered,
narrated per-doubling, exposed for refuse-affected-reads — the service
class law's fourth answer: never a wedged rebuild, never a silent skip.
The engine never writes stamps: each adapter owns its durability and its
stamp-after-data discipline. Upgrade, heal, and rebuild are now the same
machinery behind open doors.
FactLogSource wires any host's fact scan in one line
(factSourceFromHost(brain)); window-contract violations are loud.
Pins: 23 unit (budget resume without refold · preemption within one
installment · round-robin fairness under a skewed backlog · build-beside
visibility mid-swap · atomic flip · single-flight refusal · quarantine
skip/ledger/doubling · non-typed throw aborts · losing adapter
discarded) + 3 integration on a real brain (fold matches ground truth ·
doors answer mid-fold with the preemption path exercised · crash
mid-fold resumes from the stamp, never refolds).
Gates: unit 2054/2054 (157 files) · integration 820 (93 files) ·
conformance 31/31.
The private recovery discipline, applied to its own machinery: pending-
embed markers stop being sidecar files and become first-class log records
riding the write's OWN commit fact — embed.pending lands in the same
atomic append as its after-image (a marker can never be orphaned from its
write, or vice versa; in durable-at-ack mode it shares the write's
covering fsync — zero extra syncs), and the worker's landing commit rides
embed.landed with the inline vector. Crash recovery is now a FOLD of the
log (pending without a matching landed = recovered), skipped wholesale on
brains with no v2 history; the one-time legacy bridge folds existing
sidecar files in, migrates them as one fact, and deletes them —
idempotent under a crash mid-bridge. No code path writes the sidecar
again.
Plus the ENTITY-TRUTH digest law, found by this train's own pins:
canonical vector wrappers denormalize HNSW residue (connections + the
randomly-assigned node level) that the log deliberately does not carry —
the verification oracle digested it and would have reported false
state-differs on ~any nonzero-level node (a ~15% flake in the cutover pin
was the symptom). Both sides of every oracle comparison now normalize to
entity truth (nounEntityTruth); index residue has its own rebuild path
and is not entity state.
Pins: embed-markers-in-log 5/5 (same-generation marker, landed+fold-to-
zero, crash recovery via the log with the sidecar prefix EMPTY on disk,
legacy bridge, VFS hung-embedder ack) · deferred-embedding 5/5 unchanged
(the contract outlived its mechanism) · kill-matrix 11/11 · cutover 5/5
×10 runs (flake dead) · unit 2031/2031.
One deterministic v2 log (nine facts covering every fold-relevant behavior:
genesis, after-images with minted ints, a deferred embed pending→landed,
a sameAsGeneration vector ref, a verb, a tombstone, and an all-deduped
empty commit) whose ENCODED BYTES and FOLDED STATE are both pinned by
sha256 literals. The fixture (tests/fixtures/golden-log-v2.bin, 4128 B,
byte-verified against the encoder on every run) is the shared artifact a
second reader implementation consumes — it must reproduce the identical
fold digest; the pair is normative on disagreement. The fold law is
stated in prose beside the code: generation-ordered latest-per-id,
tombstone masking, embed.landed vector application, single-hop ref
resolution, key-sorted digest.
Also: decodeGroupV2 discriminated pad filler by RECORD COUNT, silently
swallowing legitimate empty commits (an all-deduped batch at a real
generation). Pads carry generation 0 — which writers can never mint — so
the generation is the honest discriminator; empty commits stay visible.
Pins: 4/4 (encode-exact, fixture-identical, fold-exact, human-readable
spot checks beside the hashes).
- Watermark stamping fans out at flush: all three projections stamped
with the committed generation before their flushes persist.
- waitForIndexed(path?, {generation, timeoutMs}) — the one honest read
barrier for write-then-recall consumers; typed timeout error carries
the pending count and names the gauge; getIndexStatus() gains
per-projection gauges. awaitPendingEmbeds() unchanged underneath.
- adoptLogAuthority() self-backfills curable divergences (pre-log
records, witness drift) by identity re-commit before flipping — a
fresh brain flips clean; log-ahead divergences still refuse loudly.
- The verification oracle gains VERB legs (all four divergence classes;
unwired = honest verbsChecked: 0, never a scope claim).
- find({where: {}}) match-all serves (was silent-empty, warm AND cold;
same fix in count/streaming/subgraph seeding); removeMany({where:{}})
refuses typed — a match-all bulk delete must be explicit.
- Aggregation native envelope stamped via noteSourceGeneration before
serializeState; the native-blob restore gates through the same
adoption verdict as caller-side state (the unconditional adopt dies).
- LC8 pinned: a wholesale directory move opens and serves identically
across all three intelligences, with history traveling.
Gates: unit 2031/2031 (156 files) · integration 812 (91 files) ·
conformance 27/27.
Every persisted projection artifact (metadata field indexes + column
segments, HNSW node records, graph adjacency LSM trees) now carries a
stamp asserting 'this state reflects every committed generation ≤ W,
atomically' — written LAST in each owner's flush (stamp-after-data: a
crash between data and stamp = unstamped = rescan, never trust). At load,
each owner computes the three-way verdict: stamped==committed → adopt
(zero work) · behind → catchup (gap reported) · above/unstamped → RESCAN,
loudly. Legacy artifacts re-derive once, then are stamped forever. Shared
law in projectionWatermark.ts (the aggregation verdict machinery,
generalized); vector artifacts carry model dimensions. Verdicts are
computed and exposed (watermark()/watermarkVerdict()/watermarkGap());
rebuild triggers unchanged — acting on 'catchup' is the fold train.
Pins: 22 unit (7 metadata · 8 hnsw · 7 graph, incl. spy-order
stamp-after-data) + the end-to-end reopen-adopts pin.
The cutover: new tail segments write format v2 (per-record [type, version,
cipherFlag, keyId] envelope; noun/verb after-images carry dense ints
MINTED AT APPEND from the id mapper — a rebuilt mapper reproduces
assignments exactly; log.genesis opens every new log with the id-space
width + a minted brain id; sync() seals to the header-declared sector
boundary with reader-invisible pad frames). Existing v1 segments are
never rewritten — per-segment decoder dispatch reads both formats and v2
facts map to the exact CommitFact shape all consumers already read.
Cutover on a live v1 log: an empty v1 tail re-heads in place; a non-empty
one is sealed by rotation, byte-identical. Records reserve the encryption
fields (cipherFlag 0 / keyId nil are the only legal values; anything else
refuses typed naming the needed newer reader) — crypto-ready with no
future bump on the compat surface. Empty-records facts are legal (an
all-deduped batch is a real generation — v1 semantics preserved; the
refusal there tore a column-store flush mid-commit in the full suite, the
consistency guard caught it loudly, and the root is fixed).
Golden byte vectors pinned for the second (native) reader implementation.
Pins: cutover 5/5 · codec 54 · kill-matrix stays 11/11.
The time-travel recall row moves from envelope-note to contracted: vector
search at a pinned past generation serves the vectors AS THEY STOOD —
a later re-embed never leaks into an earlier pin (byte-exact), tombstones
mask, the deferred-embed pin serves the stub on the vector leg until the
landing generation (text/metadata legs unaffected — triple intelligence by
design), and beyond-head pins refuse typed. Brainy-alone leg = the
documented ephemeral at-generation materialization; the at-scale leg rides
the accelerated provider's as-of index. Registry row added (shared ID
pending the master table).
Two release-blocking findings from the durability kill-matrix, both fixed
in the owning layer:
1. LOG-AUTHORITY REPLAY AT OPEN: durable-at-ack fsynced the fact before
the ack, but open() truncated every fact above the manifest — after a
power loss that takes the un-fsynced tmp+rename canonical bytes, the
acked write's ONLY durable copy was discarded. Now: under 'log'
authority, open() REPLAYS intact facts above the manifest into
canonical (FactLog.peekFactsAbove — CRC-gated, order-sorted) and
advances the manifest to cover them; tree-authority brains keep the
truncate contract they were promised. Pinned end to end: the power-loss
row constructs the exact disk state (fsynced log, vanished canonical
rename) and the acked write lives.
2. NO SILENT COMMIT: commitSingleOp buffered the generation BEFORE the
fact append; an append failure (ENOSPC) rejected the caller but the
next flush durably committed the generation with NO fact — a permanent
silent log gap. Now the failure path un-buffers and returns the counter
reservation: nothing commits, the log stays gap-free, and the canonical
execute-residue orphan is the documented crash-equivalent.
Plus: the kill-matrix itself (11 rows — every commit-path fault point ×
reopen-as-crash recovery contract, at-ack variants, disk-full row; five
new zero-cost faultPoint sites), the log-authority pin suite (oracle
green/red/state-differs, flip refusal, switch survives reopen, 9/9), and
the group-commit covering pins (5/5).
Gates: unit 2002/2002 (152 files) · integration 785 · conformance 27/27.
The provider contract (metadata addToIndex/removeFromIndex, vector
addItem/removeItem, id-mapper getOrAssign/remove) gains an optional
trailing generation — evaluated lazily at operation execute time (the
graph surface's thunk pattern, generalized), threaded from all 17
construction sites: undefined during generation-0 bootstrap, the real
committed generation everywhere else. Optional = additive: no existing
provider or caller breaks; native delta logs that stamped literal zero
start hearing truth. JS twins accept the parameter with parity notes.
Pins: provider doubles capture and assert nonzero monotonic generations
across add/update/remove on both surfaces.
The two-implementation contract surface as one pure module (no I/O):
segment header v2 (formatVersion 2 + sealSize in the reserved bytes),
per-record [type u8, version u8] envelope killing the unknown-kind
misclassification trap, the 12-type registry (after-images with minted
ints, tombstones, batch.meta, embed.pending/landed, blob.manifest,
projection.note, bootstrap.baseline, log.genesis with id-space width and
TYPED width-mismatch refusal), vectorLeg inline|{sameAsGeneration} with
writer-enforced single-hop, sector-sealed groups with pad frames, torn-tail
discipline, and GOLDEN BYTE VECTORS pinned so a second (native) reader
implementation can conform byte-for-byte. 50 format pins + a
fault-injecting storage wrapper (tear/drop-sync/fail-append) with 13
self-tests. v1 segments remain readable; nothing writes v2 yet — the
live-format cutover is its own commit.
The storage-authority adoption path, guarded shape: the canonical tree
stays authoritative by default ('tree'); a brain flips to 'log' only
through the verification oracle, and the flip is stored, per-brain,
checked at open only.
- FactLog.ensureSynced(): classic group commit — concurrent writers
append, then join ONE covering fsync (running + queued slots give the
covering guarantee: the sync a caller awaits always starts after its
append landed). Solo writer = immediate sync.
- GenerationStore.logDurability 'deferred' (default, byte-identical to
today: fact durability rides the group-commit flush, ack latency
unchanged) | 'at-ack' (log-authority mode: every single-op ack awaits a
covering log fsync — an acked write's fact survives power loss, by
contract). transact() was already durable-at-return in both modes.
- src/db/logAuthority.ts: the stored switch artifact
(_system/log-authority.json, absent = tree), readLogAuthority, and the
VERIFICATION ORACLE — replay the fact log, fold latest state per id
(digests, never bodies — memory-bounded), diff against the canonical
tree paged; verdict green iff every canonical row is exactly reproduced
AND the log claims nothing canonical denies. Divergences are NAMED by
class (pre-log-record → needs baseline backfill; state-differs;
log-live-canonical-absent; log-tombstone-canonical-present). The flip
REFUSES on red with the first divergence and the cure in the message.
- Brainy: authority read at open (log → durable-at-ack enabled);
logAuthority() / verifyLogAuthority() / adoptLogAuthority() public API.
Nothing flips by itself; nothing changes for existing brains.
A3 of the service-class pair (BRAINY-PROD-LATENCY-TRIAD): a VFS file write
ran the embedder synchronously while the caller waited — 5.6s p50 / 21.4s
p95 per small file on a production deployment, the dominant stage of every
capture write.
- add()/update() gain deferEmbedding: the write acks at durability (data +
metadata persisted, a DURABLE pending marker under
_system/pending_embeds/<id> written BEFORE the commit — orphan-safe
direction); the single-flight background worker embeds the CURRENT data
and swaps the vector in ATOMICALLY (ReplaceInVectorIndex — the row is
never absent from search; a deferred UPDATE keeps serving the OLD vector,
stale-beats-absent per the flicker law). Typed refusals: defer+vector,
defer-without-data.
- CRASH-SAFE: markers are recovered at open by a BOUNDED prefix listing
(never a store walk) and the worker resumes in the background — a crash
can delay a vector, never lose one. A wedged embedder trips a LOUD 60s
hang guard and the worker moves on (marker retained for retry).
- The honest gauges: getIndexStatus().pendingEmbeds + pendingEmbedCount();
awaitPendingEmbeds() is the eventual-vector-index BARRIER for callers
and tests that need searchability before proceeding.
- VFS adopts it everywhere a write path could wait on the embedder:
writeFile (both branches) and directory creation. Pinned in the
strongest form: writeFile resolves while the embedder HANGS FOREVER.
Pins: deferred-embedding 5/5 (ack law · stale-beats-absent · crash
recovery across sessions · VFS hung-embedder ack · typed refusals).
Gates: unit 1928/1928 · integration 765 · conformance 27/27.
