feat(embedding): deferred-embed markers become log records — the sidecar recovery path is deleted
The private recovery discipline, applied to its own machinery: pending- embed markers stop being sidecar files and become first-class log records riding the write's OWN commit fact — embed.pending lands in the same atomic append as its after-image (a marker can never be orphaned from its write, or vice versa; in durable-at-ack mode it shares the write's covering fsync — zero extra syncs), and the worker's landing commit rides embed.landed with the inline vector. Crash recovery is now a FOLD of the log (pending without a matching landed = recovered), skipped wholesale on brains with no v2 history; the one-time legacy bridge folds existing sidecar files in, migrates them as one fact, and deletes them — idempotent under a crash mid-bridge. No code path writes the sidecar again. Plus the ENTITY-TRUTH digest law, found by this train's own pins: canonical vector wrappers denormalize HNSW residue (connections + the randomly-assigned node level) that the log deliberately does not carry — the verification oracle digested it and would have reported false state-differs on ~any nonzero-level node (a ~15% flake in the cutover pin was the symptom). Both sides of every oracle comparison now normalize to entity truth (nounEntityTruth); index residue has its own rebuild path and is not entity state. Pins: embed-markers-in-log 5/5 (same-generation marker, landed+fold-to- zero, crash recovery via the log with the sidecar prefix EMPTY on disk, legacy bridge, VFS hung-embedder ack) · deferred-embedding 5/5 unchanged (the contract outlived its mechanism) · kill-matrix 11/11 · cutover 5/5 ×10 runs (flake dead) · unit 2031/2031.
This commit is contained in:
parent
c95bea8887
commit
b47787bbf7
7 changed files with 637 additions and 76 deletions
|
|
@ -92,6 +92,27 @@ export async function readLogAuthority(
|
|||
return { authority: 'tree' }
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalize a canonical noun record to its ENTITY TRUTH before diffing:
|
||||
* the canonical vector-file wrapper denormalizes derived index residue
|
||||
* (`connections` — HNSW graph edges; `level` — the node's random skip-list
|
||||
* level) that the generation log deliberately does NOT carry (projections
|
||||
* own their own rebuild paths). Digesting the residue would report false
|
||||
* `state-differs` on ~any brain whose HNSW assigned a nonzero level. Both
|
||||
* sides of every oracle comparison pass through this normalizer.
|
||||
*/
|
||||
export function nounEntityTruth(record: {
|
||||
metadata: unknown
|
||||
vector: unknown
|
||||
}): { metadata: unknown; vector: unknown } {
|
||||
const v = record.vector
|
||||
if (v && typeof v === 'object' && !Array.isArray(v)) {
|
||||
const { connections: _c, level: _l, ...entity } = v as Record<string, unknown>
|
||||
return { metadata: record.metadata, vector: entity }
|
||||
}
|
||||
return { metadata: record.metadata, vector: v }
|
||||
}
|
||||
|
||||
/**
|
||||
* Stable content hash of a stored record for diffing — key-sorted JSON so
|
||||
* property order can never fake a divergence.
|
||||
|
|
|
|||
Reference in a new issue