feat(vector): the vectored-noun scalar joins the count ledger; the open gate closes the vector leg
The coverage denominator the health-by-accounting ratification named for
the vector family — never built until now, and its absence was measured as
the exact outage class it existed to prevent: a migrated store with
canonical vectors and no derived index opened with the vector leg EMPTY,
served [] from vector search with no error, and the report-driven read gate
had nothing to refuse on (the provider's coverage invariant was honestly
unledgered — the denominator was ours to supply).
- getCanonicalCounts() gains vectors: { all } — the count of canonical
nouns holding a REAL vector. Incremented where a vector lands (the
isNew-gated metadata seam for explicit vectors — the same discipline that
keeps HNSW neighbor-link re-saves from inflating counts; a narrow
noteVectorLanded hook for the deferred-embed landing, gated on the
worker's own pre-embed read). Decremented on a proven delete of a
vectored noun; a vector-uncertain delete marks the ledger suspect rather
than guessing (no new reads on the delete path). Recounted by the
sanctioned recount; legacy counts.json derives it once (a deferred noun's
vector file exists with an empty vector, so presence requires one
content read at derivation — never on the hot path).
- The open gate's vector leg: when a health-reporting provider claims
serving while the index holds zero nodes and the ledger proves vectored
canonical rows exist, open BUILDS (narrated) — routed through the
provider's idempotent fillFromCanonical() when exposed (the joint door;
a partial shortfall stays repair()'s operator business), the JS rebuild
otherwise — or fails typed pre-serve. Scoped exactly: bare isReady()
providers, migrating providers, and white-box size stubs open as before.
Pinned end-to-end from the partner gate's probe shape (store with vectored
canonical rows, no derived index, reopen → search serves N, never []),
red-proved against the pre-fix path; the inverse (zero vectored rows) opens
without building and serves [] honestly.
This commit is contained in:
parent
bce2593e24
commit
9730835bdf
10 changed files with 851 additions and 32 deletions
|
|
@ -1041,12 +1041,27 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
|
|||
*/
|
||||
protected totalNounCountAll = 0
|
||||
protected totalVerbCountAll = 0
|
||||
/**
|
||||
* The count of canonical nouns holding a REAL (non-empty) vector — the
|
||||
* vector-side mirror of `totalNounCountAll` and the coverage denominator a
|
||||
* vector index's node-count ledger is measured against. A deferred-embed
|
||||
* noun (`add({ deferEmbedding: true })`) counts only once its vector
|
||||
* LANDS (the `system:embed-landing` commit) — its canonical record exists
|
||||
* (already counted in `totalNounCountAll`) with an empty vector until
|
||||
* then. Maintained on the write path (a fresh insert whose vector is
|
||||
* non-empty +1, a deferred embed's landing +1, a PROVEN delete of a
|
||||
* vectored noun −1), persisted beside the other ALL scalars, recomputed by
|
||||
* the sanctioned recount. Shares `allCountsSuspect` — no separate flag.
|
||||
*/
|
||||
protected totalVectoredNounCount = 0
|
||||
/**
|
||||
* `true` when a delete could not prove whether the record existed (no
|
||||
* canonical read, no caller-provided prior) — the ALL scalar may be off by
|
||||
* the unprovable deletes since. Loud, persisted, and cleared only by the
|
||||
* sanctioned recount; a consumer reading the scalar as a ledger denominator
|
||||
* must treat a suspect scalar as unverified, never as exact.
|
||||
* must treat a suspect scalar as unverified, never as exact. Also covers
|
||||
* `totalVectoredNounCount` — a delete whose vector-presence fact was
|
||||
* unknowable marks this SAME flag rather than minting a second one.
|
||||
*/
|
||||
protected allCountsSuspect = false
|
||||
/** One narration per session for the suspect transition (never per delete). */
|
||||
|
|
@ -1083,15 +1098,19 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
|
|||
* The canonical count ledger — O(1), no I/O. `counted` is the user-facing
|
||||
* scalar (public/internal tiers, what `getNounCount()` returns); `all` is
|
||||
* the ALL-visibility scalar every unfiltered storage walk is measured
|
||||
* against (the coverage-ledger denominator for derived-index providers).
|
||||
* `suspect` is `true` when an unprovable delete has made `all` unverified
|
||||
* since the last sanctioned recount (`rebuildTypeCounts`).
|
||||
* @returns Both scalars per family plus the suspect flag.
|
||||
* against (the coverage-ledger denominator for derived-index providers);
|
||||
* `vectors.all` is the vectored-noun scalar — the coverage denominator for
|
||||
* a vector index's node-count ledger specifically.
|
||||
* `suspect` is `true` when an unprovable delete has made `all` (any
|
||||
* family, including `vectors`) unverified since the last sanctioned
|
||||
* recount (`rebuildTypeCounts`).
|
||||
* @returns All scalars per family plus the suspect flag.
|
||||
*/
|
||||
async getCanonicalCounts(): Promise<CanonicalCounts> {
|
||||
return {
|
||||
nouns: { counted: this.totalNounCount, all: this.totalNounCountAll },
|
||||
verbs: { counted: this.totalVerbCount, all: this.totalVerbCountAll },
|
||||
vectors: { all: this.totalVectoredNounCount },
|
||||
suspect: this.allCountsSuspect
|
||||
}
|
||||
}
|
||||
|
|
@ -1103,7 +1122,7 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
|
|||
* @param family - Which family's delete was unprovable.
|
||||
* @param id - The id whose existence could not be established.
|
||||
*/
|
||||
protected markAllCountsSuspect(family: 'noun' | 'verb', id: string): void {
|
||||
protected markAllCountsSuspect(family: 'noun' | 'verb' | 'noun-vector', id: string): void {
|
||||
this.allCountsSuspect = true
|
||||
if (!this.allCountsSuspectNarrated) {
|
||||
this.allCountsSuspectNarrated = true
|
||||
|
|
@ -1116,6 +1135,23 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* OPTIONAL narrow ledger hook (see {@link StorageAdapter.noteVectorLanded}):
|
||||
* record a deferred-embed noun's FIRST real vector landing. The caller
|
||||
* (the deferred-embed worker) proves this is a genuine landing — not a
|
||||
* re-embed of an already-vectored row — by observing its own pre-embed
|
||||
* read's vector was empty, at no added storage cost.
|
||||
* @param id - The noun whose vector just landed (retained for a future
|
||||
* narration seam; the count itself needs no id-keyed state).
|
||||
*/
|
||||
async noteVectorLanded(id: string): Promise<void> {
|
||||
void id
|
||||
this.totalVectoredNounCount++
|
||||
this.scheduleCountPersist().catch(() => {
|
||||
// Ignore persist errors — the in-memory count is authoritative; a later op retries.
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Increment count for entity type - O(1) operation.
|
||||
* Concurrency is handled by the process-global mutex
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue