feat(vector): the vectored-noun scalar joins the count ledger; the open gate closes the vector leg
All checks were successful
CI / Node 24 (push) Successful in 12m22s
CI / Node 22 (push) Successful in 12m32s
CI / Integration + conformance (Node 22) (push) Successful in 19m42s
CI / Bun (latest) (push) Successful in 12m15s

The coverage denominator the health-by-accounting ratification named for
the vector family — never built until now, and its absence was measured as
the exact outage class it existed to prevent: a migrated store with
canonical vectors and no derived index opened with the vector leg EMPTY,
served [] from vector search with no error, and the report-driven read gate
had nothing to refuse on (the provider's coverage invariant was honestly
unledgered — the denominator was ours to supply).

- getCanonicalCounts() gains vectors: { all } — the count of canonical
  nouns holding a REAL vector. Incremented where a vector lands (the
  isNew-gated metadata seam for explicit vectors — the same discipline that
  keeps HNSW neighbor-link re-saves from inflating counts; a narrow
  noteVectorLanded hook for the deferred-embed landing, gated on the
  worker's own pre-embed read). Decremented on a proven delete of a
  vectored noun; a vector-uncertain delete marks the ledger suspect rather
  than guessing (no new reads on the delete path). Recounted by the
  sanctioned recount; legacy counts.json derives it once (a deferred noun's
  vector file exists with an empty vector, so presence requires one
  content read at derivation — never on the hot path).
- The open gate's vector leg: when a health-reporting provider claims
  serving while the index holds zero nodes and the ledger proves vectored
  canonical rows exist, open BUILDS (narrated) — routed through the
  provider's idempotent fillFromCanonical() when exposed (the joint door;
  a partial shortfall stays repair()'s operator business), the JS rebuild
  otherwise — or fails typed pre-serve. Scoped exactly: bare isReady()
  providers, migrating providers, and white-box size stubs open as before.

Pinned end-to-end from the partner gate's probe shape (store with vectored
canonical rows, no derived index, reopen → search serves N, never []),
red-proved against the pre-fix path; the inverse (zero vectored rows) opens
without building and serves [] honestly.
This commit is contained in:
David Snelling 2026-08-25 15:31:19 -07:00
parent bce2593e24
commit 9730835bdf
10 changed files with 851 additions and 32 deletions

View file

@ -1041,12 +1041,27 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
*/
protected totalNounCountAll = 0
protected totalVerbCountAll = 0
/**
* The count of canonical nouns holding a REAL (non-empty) vector the
* vector-side mirror of `totalNounCountAll` and the coverage denominator a
* vector index's node-count ledger is measured against. A deferred-embed
* noun (`add({ deferEmbedding: true })`) counts only once its vector
* LANDS (the `system:embed-landing` commit) its canonical record exists
* (already counted in `totalNounCountAll`) with an empty vector until
* then. Maintained on the write path (a fresh insert whose vector is
* non-empty +1, a deferred embed's landing +1, a PROVEN delete of a
* vectored noun 1), persisted beside the other ALL scalars, recomputed by
* the sanctioned recount. Shares `allCountsSuspect` no separate flag.
*/
protected totalVectoredNounCount = 0
/**
* `true` when a delete could not prove whether the record existed (no
* canonical read, no caller-provided prior) the ALL scalar may be off by
* the unprovable deletes since. Loud, persisted, and cleared only by the
* sanctioned recount; a consumer reading the scalar as a ledger denominator
* must treat a suspect scalar as unverified, never as exact.
* must treat a suspect scalar as unverified, never as exact. Also covers
* `totalVectoredNounCount` a delete whose vector-presence fact was
* unknowable marks this SAME flag rather than minting a second one.
*/
protected allCountsSuspect = false
/** One narration per session for the suspect transition (never per delete). */
@ -1083,15 +1098,19 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
* The canonical count ledger O(1), no I/O. `counted` is the user-facing
* scalar (public/internal tiers, what `getNounCount()` returns); `all` is
* the ALL-visibility scalar every unfiltered storage walk is measured
* against (the coverage-ledger denominator for derived-index providers).
* `suspect` is `true` when an unprovable delete has made `all` unverified
* since the last sanctioned recount (`rebuildTypeCounts`).
* @returns Both scalars per family plus the suspect flag.
* against (the coverage-ledger denominator for derived-index providers);
* `vectors.all` is the vectored-noun scalar the coverage denominator for
* a vector index's node-count ledger specifically.
* `suspect` is `true` when an unprovable delete has made `all` (any
* family, including `vectors`) unverified since the last sanctioned
* recount (`rebuildTypeCounts`).
* @returns All scalars per family plus the suspect flag.
*/
async getCanonicalCounts(): Promise<CanonicalCounts> {
return {
nouns: { counted: this.totalNounCount, all: this.totalNounCountAll },
verbs: { counted: this.totalVerbCount, all: this.totalVerbCountAll },
vectors: { all: this.totalVectoredNounCount },
suspect: this.allCountsSuspect
}
}
@ -1103,7 +1122,7 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
* @param family - Which family's delete was unprovable.
* @param id - The id whose existence could not be established.
*/
protected markAllCountsSuspect(family: 'noun' | 'verb', id: string): void {
protected markAllCountsSuspect(family: 'noun' | 'verb' | 'noun-vector', id: string): void {
this.allCountsSuspect = true
if (!this.allCountsSuspectNarrated) {
this.allCountsSuspectNarrated = true
@ -1116,6 +1135,23 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
}
}
/**
* OPTIONAL narrow ledger hook (see {@link StorageAdapter.noteVectorLanded}):
* record a deferred-embed noun's FIRST real vector landing. The caller
* (the deferred-embed worker) proves this is a genuine landing not a
* re-embed of an already-vectored row by observing its own pre-embed
* read's vector was empty, at no added storage cost.
* @param id - The noun whose vector just landed (retained for a future
* narration seam; the count itself needs no id-keyed state).
*/
async noteVectorLanded(id: string): Promise<void> {
void id
this.totalVectoredNounCount++
this.scheduleCountPersist().catch(() => {
// Ignore persist errors — the in-memory count is authoritative; a later op retries.
})
}
/**
* Increment count for entity type - O(1) operation.
* Concurrency is handled by the process-global mutex