feat(embedding): deferred-embed markers become log records — the sidecar recovery path is deleted
Some checks failed
CI / Node 22 (push) Has been cancelled
CI / Node 24 (push) Has been cancelled
CI / Bun (latest) (push) Has been cancelled

The private recovery discipline, applied to its own machinery: pending-
embed markers stop being sidecar files and become first-class log records
riding the write's OWN commit fact — embed.pending lands in the same
atomic append as its after-image (a marker can never be orphaned from its
write, or vice versa; in durable-at-ack mode it shares the write's
covering fsync — zero extra syncs), and the worker's landing commit rides
embed.landed with the inline vector. Crash recovery is now a FOLD of the
log (pending without a matching landed = recovered), skipped wholesale on
brains with no v2 history; the one-time legacy bridge folds existing
sidecar files in, migrates them as one fact, and deletes them —
idempotent under a crash mid-bridge. No code path writes the sidecar
again.

Plus the ENTITY-TRUTH digest law, found by this train's own pins:
canonical vector wrappers denormalize HNSW residue (connections + the
randomly-assigned node level) that the log deliberately does not carry —
the verification oracle digested it and would have reported false
state-differs on ~any nonzero-level node (a ~15% flake in the cutover pin
was the symptom). Both sides of every oracle comparison now normalize to
entity truth (nounEntityTruth); index residue has its own rebuild path
and is not entity state.

Pins: embed-markers-in-log 5/5 (same-generation marker, landed+fold-to-
zero, crash recovery via the log with the sidecar prefix EMPTY on disk,
legacy bridge, VFS hung-embedder ack) · deferred-embedding 5/5 unchanged
(the contract outlived its mechanism) · kill-matrix 11/11 · cutover 5/5
×10 runs (flake dead) · unit 2031/2031.
This commit is contained in:
David Snelling 2026-08-10 11:27:07 -07:00
parent c95bea8887
commit b47787bbf7
7 changed files with 637 additions and 76 deletions

View file

@ -51,7 +51,8 @@ import {
storageSupportsFactLog,
type CommitFact,
type FactOp,
type FactIntMinter
type FactIntMinter,
type FactMarkerRecord
} from './factLog.js'
import { GenerationSegmentStore, type FoldGeneration } from './generationSegments.js'
import { crc32c } from '../utils/crc32c.js'
@ -903,6 +904,8 @@ export class GenerationStore {
nouns: string[]
verbs: string[]
meta?: Record<string, unknown>
/** V2 marker records riding this fact (same generation, same append). */
records?: FactMarkerRecord[]
}): Promise<CommitFact> {
const ops: FactOp[] = []
const afterRecords: GenerationRecord[] = []
@ -926,7 +929,8 @@ export class GenerationStore {
timestamp: args.timestamp,
ops,
...(args.meta ? { meta: args.meta } : {}),
...(blobHashes.length > 0 ? { blobHashes } : {})
...(blobHashes.length > 0 ? { blobHashes } : {}),
...(args.records && args.records.length > 0 ? { records: args.records } : {})
}
}
@ -939,6 +943,12 @@ export class GenerationStore {
* per-record analogue of `ifAtGeneration`. A throw aborts the whole batch:
* the generation reservation is returned and no staging I/O has happened. */
precommit?: (before: CommitBeforeImages) => void
/** Optional v2 marker records riding this batch's ONE commit fact (e.g.
* the deferred-embedding lifecycle markers) same generation, same
* atomic append, same durability barrier as the batch itself, so a
* marker can never be orphaned from its write nor the write from its
* marker. Additive: omitted on every markerless path. */
records?: FactMarkerRecord[]
execute: () => Promise<void>
}): Promise<{ generation: number; timestamp: number }> {
return this.withMutex(async () => {
@ -1075,7 +1085,8 @@ export class GenerationStore {
timestamp,
nouns,
verbs,
...(args.meta ? { meta: args.meta } : {})
...(args.meta ? { meta: args.meta } : {}),
...(args.records && args.records.length > 0 ? { records: args.records } : {})
})
await this.factLog.append(fact)
await this.factLog.sync()
@ -1288,6 +1299,18 @@ export class GenerationStore {
touched: { nouns?: string[]; verbs?: string[] }
execute: () => Promise<void>
precommit?: (before: CommitBeforeImages) => void
/**
* Optional v2 marker records riding this write's commit fact (e.g. the
* deferred-embedding lifecycle markers) same generation, same atomic
* append, and in 'at-ack' log durability the SAME covering fsync as the
* write itself (zero extra sync). A marker can never be orphaned from
* its write nor the write from its marker. Additive: omitted on every
* markerless path. When the storage hosts no fact log the markers have
* no durable home matching that storage's overall durability posture
* (it cannot host the log's crash guarantees either); callers own
* surfacing that honestly.
*/
records?: FactMarkerRecord[]
}): Promise<{ generation: number; timestamp: number; degraded?: string[] }> {
return this.withMutex(async () => {
// Refuse to accept a write whose history we cannot make durable: if the
@ -1357,7 +1380,13 @@ export class GenerationStore {
// buffered history).
if (this.factLog) {
await this.factLog.append(
await this.buildCommitFact({ generation: gen, timestamp, nouns, verbs })
await this.buildCommitFact({
generation: gen,
timestamp,
nouns,
verbs,
...(args.records && args.records.length > 0 ? { records: args.records } : {})
})
)
}
prodLog.warn(
@ -1411,7 +1440,13 @@ export class GenerationStore {
if (this.factLog) {
try {
await this.factLog.append(
await this.buildCommitFact({ generation: gen, timestamp, nouns, verbs })
await this.buildCommitFact({
generation: gen,
timestamp,
nouns,
verbs,
...(args.records && args.records.length > 0 ? { records: args.records } : {})
})
)
if (this.logDurability === 'at-ack') {
await this.factLog.ensureSynced()