Commit graph

2 commits

Author SHA1 Message Date
b47787bbf7 feat(embedding): deferred-embed markers become log records — the sidecar recovery path is deleted
Some checks failed
CI / Node 22 (push) Has been cancelled
CI / Node 24 (push) Has been cancelled
CI / Bun (latest) (push) Has been cancelled
The private recovery discipline, applied to its own machinery: pending-
embed markers stop being sidecar files and become first-class log records
riding the write's OWN commit fact — embed.pending lands in the same
atomic append as its after-image (a marker can never be orphaned from its
write, or vice versa; in durable-at-ack mode it shares the write's
covering fsync — zero extra syncs), and the worker's landing commit rides
embed.landed with the inline vector. Crash recovery is now a FOLD of the
log (pending without a matching landed = recovered), skipped wholesale on
brains with no v2 history; the one-time legacy bridge folds existing
sidecar files in, migrates them as one fact, and deletes them —
idempotent under a crash mid-bridge. No code path writes the sidecar
again.

Plus the ENTITY-TRUTH digest law, found by this train's own pins:
canonical vector wrappers denormalize HNSW residue (connections + the
randomly-assigned node level) that the log deliberately does not carry —
the verification oracle digested it and would have reported false
state-differs on ~any nonzero-level node (a ~15% flake in the cutover pin
was the symptom). Both sides of every oracle comparison now normalize to
entity truth (nounEntityTruth); index residue has its own rebuild path
and is not entity state.

Pins: embed-markers-in-log 5/5 (same-generation marker, landed+fold-to-
zero, crash recovery via the log with the sidecar prefix EMPTY on disk,
legacy bridge, VFS hung-embedder ack) · deferred-embedding 5/5 unchanged
(the contract outlived its mechanism) · kill-matrix 11/11 · cutover 5/5
×10 runs (flake dead) · unit 2031/2031.
2026-08-10 11:27:07 -07:00
26c6025158 feat(log): v2 is the LIVE write format — envelope records with minted ints, genesis, sector seals; v1 readable forever
The cutover: new tail segments write format v2 (per-record [type, version,
cipherFlag, keyId] envelope; noun/verb after-images carry dense ints
MINTED AT APPEND from the id mapper — a rebuilt mapper reproduces
assignments exactly; log.genesis opens every new log with the id-space
width + a minted brain id; sync() seals to the header-declared sector
boundary with reader-invisible pad frames). Existing v1 segments are
never rewritten — per-segment decoder dispatch reads both formats and v2
facts map to the exact CommitFact shape all consumers already read.
Cutover on a live v1 log: an empty v1 tail re-heads in place; a non-empty
one is sealed by rotation, byte-identical. Records reserve the encryption
fields (cipherFlag 0 / keyId nil are the only legal values; anything else
refuses typed naming the needed newer reader) — crypto-ready with no
future bump on the compat surface. Empty-records facts are legal (an
all-deduped batch is a real generation — v1 semantics preserved; the
refusal there tore a column-store flush mid-commit in the full suite, the
consistency guard caught it loudly, and the root is fixed).

Golden byte vectors pinned for the second (native) reader implementation.
Pins: cutover 5/5 · codec 54 · kill-matrix stays 11/11.
2026-08-10 10:55:11 -07:00