David Snelling
287384cf1e
A3 of the service-class pair (BRAINY-PROD-LATENCY-TRIAD): a VFS file write ran the embedder synchronously while the caller waited — 5.6s p50 / 21.4s p95 per small file on a production deployment, the dominant stage of every capture write. - add()/update() gain deferEmbedding: the write acks at durability (data + metadata persisted, a DURABLE pending marker under _system/pending_embeds/<id> written BEFORE the commit — orphan-safe direction); the single-flight background worker embeds the CURRENT data and swaps the vector in ATOMICALLY (ReplaceInVectorIndex — the row is never absent from search; a deferred UPDATE keeps serving the OLD vector, stale-beats-absent per the flicker law). Typed refusals: defer+vector, defer-without-data. - CRASH-SAFE: markers are recovered at open by a BOUNDED prefix listing (never a store walk) and the worker resumes in the background — a crash can delay a vector, never lose one. A wedged embedder trips a LOUD 60s hang guard and the worker moves on (marker retained for retry). - The honest gauges: getIndexStatus().pendingEmbeds + pendingEmbedCount(); awaitPendingEmbeds() is the eventual-vector-index BARRIER for callers and tests that need searchability before proceeding. - VFS adopts it everywhere a write path could wait on the embedder: writeFile (both branches) and directory creation. Pinned in the strongest form: writeFile resolves while the embedder HANGS FOREVER. Pins: deferred-embedding 5/5 (ack law · stale-beats-absent · crash recovery across sessions · VFS hung-embedder ack · typed refusals). Gates: unit 1928/1928 · integration 765 · conformance 27/27. |
||
|---|---|---|
| .. | ||
| api | ||
| benchmarks | ||
| comprehensive | ||
| configs | ||
| conformance | ||
| fixtures/import | ||
| helpers | ||
| integration | ||
| integrations | ||
| performance | ||
| regression | ||
| scripts | ||
| transaction | ||
| unit | ||
| vfs | ||
| critical-neural-validation.test.ts | ||
| critical-performance-benchmark.test.ts | ||
| model-loading.test.ts | ||
| package-size-breakdown.test.ts | ||
| package-size-limit.test.ts | ||
| setup-integration.ts | ||
| setup-semantic.ts | ||
| setup-unit.ts | ||
| setup.ts | ||
| type-utils.unit.test.ts | ||