feat(vector): the vectored-noun scalar joins the count ledger; the open gate closes the vector leg

The coverage denominator the health-by-accounting ratification named for
the vector family — never built until now, and its absence was measured as
the exact outage class it existed to prevent: a migrated store with
canonical vectors and no derived index opened with the vector leg EMPTY,
served [] from vector search with no error, and the report-driven read gate
had nothing to refuse on (the provider's coverage invariant was honestly
unledgered — the denominator was ours to supply).

- getCanonicalCounts() gains vectors: { all } — the count of canonical
  nouns holding a REAL vector. Incremented where a vector lands (the
  isNew-gated metadata seam for explicit vectors — the same discipline that
  keeps HNSW neighbor-link re-saves from inflating counts; a narrow
  noteVectorLanded hook for the deferred-embed landing, gated on the
  worker's own pre-embed read). Decremented on a proven delete of a
  vectored noun; a vector-uncertain delete marks the ledger suspect rather
  than guessing (no new reads on the delete path). Recounted by the
  sanctioned recount; legacy counts.json derives it once (a deferred noun's
  vector file exists with an empty vector, so presence requires one
  content read at derivation — never on the hot path).
- The open gate's vector leg: when a health-reporting provider claims
  serving while the index holds zero nodes and the ledger proves vectored
  canonical rows exist, open BUILDS (narrated) — routed through the
  provider's idempotent fillFromCanonical() when exposed (the joint door;
  a partial shortfall stays repair()'s operator business), the JS rebuild
  otherwise — or fails typed pre-serve. Scoped exactly: bare isReady()
  providers, migrating providers, and white-box size stubs open as before.

Pinned end-to-end from the partner gate's probe shape (store with vectored
canonical rows, no derived index, reopen → search serves N, never []),
red-proved against the pre-fix path; the inverse (zero vectored rows) opens
without building and serves [] honestly.

src/storage/adapters/baseStorageAdapter.ts

@@ -1041,12 +1041,27 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
    */
   protected totalNounCountAll = 0
   protected totalVerbCountAll = 0
+  /**
+   * The count of canonical nouns holding a REAL (non-empty) vector — the
+   * vector-side mirror of `totalNounCountAll` and the coverage denominator a
+   * vector index's node-count ledger is measured against. A deferred-embed
+   * noun (`add({ deferEmbedding: true })`) counts only once its vector
+   * LANDS (the `system:embed-landing` commit) — its canonical record exists
+   * (already counted in `totalNounCountAll`) with an empty vector until
+   * then. Maintained on the write path (a fresh insert whose vector is
+   * non-empty +1, a deferred embed's landing +1, a PROVEN delete of a
+   * vectored noun −1), persisted beside the other ALL scalars, recomputed by
+   * the sanctioned recount. Shares `allCountsSuspect` — no separate flag.
+   */
+  protected totalVectoredNounCount = 0
   /**
    * `true` when a delete could not prove whether the record existed (no
    * canonical read, no caller-provided prior) — the ALL scalar may be off by
    * the unprovable deletes since. Loud, persisted, and cleared only by the
    * sanctioned recount; a consumer reading the scalar as a ledger denominator
-   * must treat a suspect scalar as unverified, never as exact.
+   * must treat a suspect scalar as unverified, never as exact. Also covers
+   * `totalVectoredNounCount` — a delete whose vector-presence fact was
+   * unknowable marks this SAME flag rather than minting a second one.
    */
   protected allCountsSuspect = false
   /** One narration per session for the suspect transition (never per delete). */
@@ -1083,15 +1098,19 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
    * The canonical count ledger — O(1), no I/O. `counted` is the user-facing
    * scalar (public/internal tiers, what `getNounCount()` returns); `all` is
    * the ALL-visibility scalar every unfiltered storage walk is measured
-   * against (the coverage-ledger denominator for derived-index providers).
-   * `suspect` is `true` when an unprovable delete has made `all` unverified
-   * since the last sanctioned recount (`rebuildTypeCounts`).
-   * @returns Both scalars per family plus the suspect flag.
+   * against (the coverage-ledger denominator for derived-index providers);
+   * `vectors.all` is the vectored-noun scalar — the coverage denominator for
+   * a vector index's node-count ledger specifically.
+   * `suspect` is `true` when an unprovable delete has made `all` (any
+   * family, including `vectors`) unverified since the last sanctioned
+   * recount (`rebuildTypeCounts`).
+   * @returns All scalars per family plus the suspect flag.
    */
   async getCanonicalCounts(): Promise<CanonicalCounts> {
     return {
       nouns: { counted: this.totalNounCount, all: this.totalNounCountAll },
       verbs: { counted: this.totalVerbCount, all: this.totalVerbCountAll },
+      vectors: { all: this.totalVectoredNounCount },
       suspect: this.allCountsSuspect
     }
   }
@@ -1103,7 +1122,7 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
    * @param family - Which family's delete was unprovable.
    * @param id - The id whose existence could not be established.
    */
-  protected markAllCountsSuspect(family: 'noun' | 'verb', id: string): void {
+  protected markAllCountsSuspect(family: 'noun' | 'verb' | 'noun-vector', id: string): void {
     this.allCountsSuspect = true
     if (!this.allCountsSuspectNarrated) {
       this.allCountsSuspectNarrated = true
@@ -1116,6 +1135,23 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
     }
   }
 
+  /**
+   * OPTIONAL narrow ledger hook (see {@link StorageAdapter.noteVectorLanded}):
+   * record a deferred-embed noun's FIRST real vector landing. The caller
+   * (the deferred-embed worker) proves this is a genuine landing — not a
+   * re-embed of an already-vectored row — by observing its own pre-embed
+   * read's vector was empty, at no added storage cost.
+   * @param id - The noun whose vector just landed (retained for a future
+   *   narration seam; the count itself needs no id-keyed state).
+   */
+  async noteVectorLanded(id: string): Promise<void> {
+    void id
+    this.totalVectoredNounCount++
+    this.scheduleCountPersist().catch(() => {
+      // Ignore persist errors — the in-memory count is authoritative; a later op retries.
+    })
+  }
+
   /**
    * Increment count for entity type - O(1) operation.
    * Concurrency is handled by the process-global mutex

src/storage/adapters/fileSystemStorage.ts

@@ -2583,6 +2583,7 @@ export class FileSystemStorage extends BaseStorage {
         // record reads), persist, and never scan again. Absent keys are a
         // legacy file, not a zero — a zero here would make every provider's
         // coverage ledger read "over-posted" on a populated store.
+        let needsPersist = false
         if (
           typeof counts.totalNounCountAll === 'number' &&
           typeof counts.totalVerbCountAll === 'number'
@@ -2601,6 +2602,29 @@ export class FileSystemStorage extends BaseStorage {
               `derived once from the canonical id tree (${nouns.count} nouns, ${verbs.count} verbs, ` +
               `every tier) and persisted; no further scan.`
           )
+          needsPersist = true
+        }
+
+        // The vectored-noun scalar (shipped after the ALL scalars above — a
+        // counts.json can carry `totalNounCountAll`/`totalVerbCountAll` but
+        // still predate THIS key). Unlike the ALL scalars, presence cannot be
+        // decided from the id-directory listing alone: a deferred-embed
+        // noun's `vectors.json` EXISTS with an empty `vector: []` until its
+        // embed lands, so this derivation reads every noun's `vectors.json`
+        // ONCE (O(nouns) reads, not O(ids) listing) — honest, one-time cost.
+        if (typeof counts.totalVectoredNounCount === 'number') {
+          this.totalVectoredNounCount = counts.totalVectoredNounCount
+        } else {
+          const vectored = await this.scanVectoredNounCount()
+          this.totalVectoredNounCount = vectored
+          console.warn(
+            `[FileSystemStorage] counts.json predates the vectored-noun count ledger — ` +
+              `derived once by reading every noun's vectors.json (${vectored} vectored) and ` +
+              `persisted; no further scan.`
+          )
+          needsPersist = true
+        }
+        if (needsPersist) {
           await this.persistCounts()
         }
 
@@ -2643,6 +2667,12 @@ export class FileSystemStorage extends BaseStorage {
       this.totalNounCountAll = nouns.count
       this.totalVerbCountAll = verbs.count
       this.allCountsSuspect = false
+      // Vectored-noun scalar: presence needs each noun's vectors.json CONTENT
+      // (a deferred-embed noun's file exists but holds an empty vector until
+      // its embed lands), so this is a full O(nouns) content scan — see
+      // scanVectoredNounCount()'s JSDoc for the cost note. Paid once, here,
+      // alongside the rest of this from-disk recovery.
+      this.totalVectoredNounCount = await this.scanVectoredNounCount()
 
       // Sample some entities for the type distribution (don't read all).
       // Read the metadata files DIRECTLY with fs — this runs inside init(),
@@ -2728,6 +2758,61 @@ export class FileSystemStorage extends BaseStorage {
     }
   }
 
+  /**
+   * Read one canonical noun's `vectors.json` (or `.json.gz`) directly with fs
+   * — the vector-side mirror of {@link readEntityMetadataRaw}, same
+   * reentrancy reason (bypasses `getNoun()`'s `ensureInitialized()`).
+   * @param entityDir - Absolute `entities/nouns/<shard>/<id>` directory.
+   * @returns The parsed vector record, or null when absent/unreadable.
+   */
+  private async readEntityVectorRaw(entityDir: string): Promise<any | null> {
+    const base = path.join(entityDir, 'vectors.json')
+    try {
+      return JSON.parse(await fs.promises.readFile(base, 'utf-8'))
+    } catch {
+      // fall through to the compressed variant
+    }
+    try {
+      const gz = await fs.promises.readFile(`${base}.gz`)
+      return JSON.parse(zlib.gunzipSync(gz).toString('utf-8'))
+    } catch {
+      return null
+    }
+  }
+
+  /**
+   * Count canonical nouns holding a REAL (non-empty) vector — the vectored-
+   * noun ledger scalar. UNLIKE {@link scanCanonicalEntities}, presence
+   * cannot be decided from the id-directory listing alone: a deferred-embed
+   * noun's `vectors.json` EXISTS (written at `add()` time with `vector: []`)
+   * until its embed LANDS, so this walk reads every noun's `vectors.json`
+   * CONTENT — O(nouns) reads, not O(ids) listing. Used ONLY for a one-time
+   * legacy-counts.json derivation or a lost/corrupted counts.json recovery;
+   * the result is persisted so this scan never repeats.
+   */
+  private async scanVectoredNounCount(): Promise<number> {
+    const base = path.join(this.rootDir, 'entities', 'nouns')
+    let vectored = 0
+    try {
+      const shards = await fs.promises.readdir(base, { withFileTypes: true })
+      for (const shard of shards) {
+        if (!shard.isDirectory() || !/^[0-9a-f]{2}$/i.test(shard.name)) continue
+        const shardPath = path.join(base, shard.name)
+        const ids = await fs.promises.readdir(shardPath, { withFileTypes: true })
+        for (const entry of ids) {
+          if (!entry.isDirectory()) continue
+          const record = await this.readEntityVectorRaw(path.join(shardPath, entry.name))
+          if (record && Array.isArray(record.vector) && record.vector.length > 0) {
+            vectored++
+          }
+        }
+      }
+    } catch (error: any) {
+      if (error?.code !== 'ENOENT') throw error
+    }
+    return vectored
+  }
+
   /**
    * Persist counts to filesystem storage
    */
@@ -2744,6 +2829,10 @@ export class FileSystemStorage extends BaseStorage {
         // written before the ledger existed; initializeCounts() derives them once.
         totalNounCountAll: this.totalNounCountAll,
         totalVerbCountAll: this.totalVerbCountAll,
+        // Vectored-noun ledger scalar — absent in files written before it
+        // existed; initializeCounts() derives it once (a content scan, see
+        // scanVectoredNounCount()'s JSDoc).
+        totalVectoredNounCount: this.totalVectoredNounCount,
         allCountsSuspect: this.allCountsSuspect,
         lastUpdated: new Date().toISOString()
       }

src/storage/adapters/memoryStorage.ts

@@ -520,6 +520,12 @@ export class MemoryStorage extends BaseStorage {
 
     let totalNouns = 0
     let totalVerbs = 0
+    // Vectored-noun scalar: unlike the bare presence check above, this needs
+    // the vectors.json RECORD'S content — a deferred-embed noun's record
+    // exists with an empty `vector: []` until its embed lands. In-memory this
+    // is a free field access (no I/O), unlike the filesystem adapter's
+    // per-noun disk read.
+    let totalVectoredNouns = 0
 
     // Scan all paths in objectStore
     for (const path of this.objectStore.keys()) {
@@ -528,6 +534,10 @@ export class MemoryStorage extends BaseStorage {
       if (nounMatch) {
         // Type is in metadata, not path - just count total
         totalNouns++
+        const record = this.objectStore.get(path) as { vector?: unknown } | undefined
+        if (Array.isArray(record?.vector) && record.vector.length > 0) {
+          totalVectoredNouns++
+        }
       }
 
       // Count verbs (entities/verbs/{shard}/{id}/vectors.json)
@@ -543,6 +553,7 @@ export class MemoryStorage extends BaseStorage {
     // A scan of every canonical record IS the ALL-visibility count.
     this.totalNounCountAll = totalNouns
     this.totalVerbCountAll = totalVerbs
+    this.totalVectoredNounCount = totalVectoredNouns
     this.allCountsSuspect = false
   }
 

src/storage/baseStorage.ts

@@ -1762,8 +1762,10 @@ export abstract class BaseStorage extends BaseStorageAdapter {
 
   /**
    * Delete a noun from storage
+   * @param hadVector - OPTIONAL vectored-noun ledger hint, forwarded to
+   *   {@link deleteNounMetadata} unchanged — see its JSDoc.
    */
-  public async deleteNoun(id: string, priorMetadata?: NounMetadata | null): Promise<void> {
+  public async deleteNoun(id: string, priorMetadata?: NounMetadata | null, hadVector?: boolean): Promise<void> {
     await this.ensureInitialized()
 
     // FULL removal (live-HEAD hygiene): remove BOTH canonical legs AND the
@@ -1780,7 +1782,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
     // LONGER wrapped in a blind catch that masked faults as "file didn't exist".
     // `priorMetadata` (the caller's pre-delete read) keeps the decrement honest
     // even when the canonical read inside returns null (replace race / ghost).
-    await this.deleteNounMetadata(id, priorMetadata)
+    await this.deleteNounMetadata(id, priorMetadata, hadVector)
 
     // Remove the now-empty entity container (a no-op for key/prefix stores).
     await this.removeCanonicalContainer(getNounVectorPath(id))
@@ -3392,11 +3394,12 @@ export abstract class BaseStorage extends BaseStorageAdapter {
   /**
    * Save noun metadata to storage (now typed)
    * Routes to correct sharded location based on UUID
+   * @param hasVector - See {@link StorageAdapter.saveNounMetadata}'s JSDoc.
    */
-  public async saveNounMetadata(id: string, metadata: NounMetadata): Promise<void> {
+  public async saveNounMetadata(id: string, metadata: NounMetadata, hasVector?: boolean): Promise<void> {
     // Validate noun type in metadata - storage boundary protection
     validateNounType(metadata.noun)
-    return this.saveNounMetadata_internal(id, metadata)
+    return this.saveNounMetadata_internal(id, metadata, hasVector)
   }
 
   /**
@@ -3407,9 +3410,10 @@ export abstract class BaseStorage extends BaseStorageAdapter {
    * This ensures counts are updated AFTER metadata exists, fixing the race condition
    * where storage adapters tried to read metadata before it was saved.
    *
+   * @param hasVector - See {@link StorageAdapter.saveNounMetadata}'s JSDoc.
    * @protected
    */
-  protected async saveNounMetadata_internal(id: string, metadata: NounMetadata): Promise<void> {
+  protected async saveNounMetadata_internal(id: string, metadata: NounMetadata, hasVector?: boolean): Promise<void> {
     await this.ensureInitialized()
 
     // ID-first path - no type needed!
@@ -3465,6 +3469,14 @@ export abstract class BaseStorage extends BaseStorageAdapter {
     // record persists here so the ALL scalar never lags the tree.
     if (isNew) {
       this.totalNounCountAll++
+      // Vectored-noun ledger: rides the SAME isNew gate (once per id, at
+      // creation) — this seam is metadata-write-driven and never re-runs on
+      // the HNSW neighbor-link re-saves that hit saveNoun_internal, so it
+      // cannot double-count. A deferred-embed insert passes hasVector=false
+      // (or omits it); its vector lands later via noteVectorLanded().
+      if (hasVector) {
+        this.totalVectoredNounCount++
+      }
       if (!(metadata.noun && isCounted)) {
         this.scheduleCountPersist().catch(() => {
           // Ignore persist errors — the in-memory count is authoritative; a later op retries.
@@ -3860,8 +3872,18 @@ export abstract class BaseStorage extends BaseStorageAdapter {
    *   the skip permanently inflated the persisted totals (adds counted, paired
    *   removals not decremented), and `Math.max(totalNounCount, scanned)` made
    *   the inflation unfixable by any disk cleanup.
+   * @param hadVector - OPTIONAL vectored-noun ledger hint — see
+   *   {@link StorageAdapter.deleteNounMetadata}'s JSDoc. This method never
+   *   reads `vectors.json` to answer the question itself (a canonical read
+   *   the delete path must never add); a caller that cannot supply the fact
+   *   for free leaves it `undefined`, and the ledger goes SUSPECT rather
+   *   than guessing.
    */
-  public async deleteNounMetadata(id: string, priorRecord?: NounMetadata | null): Promise<void> {
+  public async deleteNounMetadata(
+    id: string,
+    priorRecord?: NounMetadata | null,
+    hadVector?: boolean
+  ): Promise<void> {
     await this.ensureInitialized()
 
     // Direct O(1) delete with ID-first path. Read the canonical record BEFORE
@@ -3885,6 +3907,15 @@ export abstract class BaseStorage extends BaseStorageAdapter {
     } else {
       this.markAllCountsSuspect('noun', id)
     }
+
+    // Vectored-noun ledger: a KNOWN vector fact decrements (or no-ops);
+    // an UNKNOWN one goes suspect rather than guessing — see @param hadVector.
+    if (hadVector === true) {
+      if (this.totalVectoredNounCount > 0) this.totalVectoredNounCount--
+      else this.markAllCountsSuspect('noun-vector', id)
+    } else if (hadVector === undefined) {
+      this.markAllCountsSuspect('noun-vector', id)
+    }
     this.scheduleCountPersist().catch(() => {
       // Ignore persist errors — the in-memory count is authoritative; a later op retries.
     })
@@ -4549,6 +4580,16 @@ export abstract class BaseStorage extends BaseStorageAdapter {
     // construction.
     let allNouns = 0
     let allVerbs = 0
+    // Vectored-noun scalar: unlike `allNouns` (decided from the metadata.json
+    // LISTING alone), presence cannot be decided from the vectors.json
+    // listing alone — a deferred-embed noun's vectors.json EXISTS with an
+    // empty `vector: []` until its embed lands, so the file's CONTENT must be
+    // read. This walk already lists every path per shard (including
+    // vectors.json entries — `listCanonicalObjects` yields both legs), so
+    // reading them here costs one EXTRA read per noun beyond the metadata.json
+    // read above (doubling this walk's per-noun I/O) — honest cost, paid only
+    // by this diagnostic/repair recount, never on the hot path.
+    let allVectoredNouns = 0
 
     // Scan noun shards
     for (let shard = 0; shard < 256; shard++) {
@@ -4559,6 +4600,18 @@ export abstract class BaseStorage extends BaseStorageAdapter {
         const paths = await this.listCanonicalObjects(shardDir)
 
         for (const path of paths) {
+          if (path.includes('/vectors.json')) {
+            try {
+              const vectorRecord = await this.readCanonicalObject(path)
+              if (vectorRecord && Array.isArray(vectorRecord.vector) && vectorRecord.vector.length > 0) {
+                allVectoredNouns++
+              }
+            } catch (error) {
+              // Skip vector records that fail to load — best-effort ground truth,
+              // same as the metadata read below.
+            }
+            continue
+          }
           if (!path.includes('/metadata.json')) continue
           allNouns++
 
@@ -4634,17 +4687,19 @@ export abstract class BaseStorage extends BaseStorageAdapter {
     // IS the proof an unprovable delete could not give.
     const nounsAllBefore = this.totalNounCountAll
     const verbsAllBefore = this.totalVerbCountAll
+    const vectoredBefore = this.totalVectoredNounCount
     this.totalNounCountAll = allNouns
     this.totalVerbCountAll = allVerbs
+    this.totalVectoredNounCount = allVectoredNouns
     this.allCountsSuspect = false
     this.countCache.clear()
     await this.persistCounts()
 
     prodLog.info(
       `[BaseStorage] Rebuilt counts: ${totalNouns} nouns, ${totalVerbs} verbs (user-facing); ` +
-        `ALL-visibility ledger ${allNouns} nouns / ${allVerbs} verbs` +
-        (nounsAllBefore !== allNouns || verbsAllBefore !== allVerbs
-          ? ` (corrected from ${nounsAllBefore} / ${verbsAllBefore})`
+        `ALL-visibility ledger ${allNouns} nouns / ${allVerbs} verbs / ${allVectoredNouns} vectored nouns` +
+        (nounsAllBefore !== allNouns || verbsAllBefore !== allVerbs || vectoredBefore !== allVectoredNouns
+          ? ` (corrected from ${nounsAllBefore} / ${verbsAllBefore} / ${vectoredBefore})`
           : ' (unchanged)') +
         ` — scalar + per-type persisted`
     )