feat(storage): the canonical count ledger — ALL-visibility scalars, unclamped totals, suspect-on-unprovable-delete

The storage-level unfiltered getNouns()/getVerbs() walks enumerate every
tier, but their totalCount reported the user-facing scalar, which skips
system/internal records on the write path — so a derived-index coverage
ledger comparing its posted count against that total would read
"over-posted by N" on every store with a VFS. This adds the ledger's real
denominators:

- totalNounCountAll / totalVerbCountAll: +1 for every new canonical record
  regardless of tier, −1 for every PROVEN delete (record read, or the
  caller's prior image), persisted in counts.json beside the counted
  scalars, recomputed by the sanctioned recount (rebuildTypeCounts).
- The unfiltered storage-level totalCount is now the ALL scalar and is
  never clamped: Math.max(scalar, scanned) could only move a scalar up, so
  an inflated counter hid forever; a divergence is now visible and healed
  by repairIndex().
- A delete that cannot prove the record existed never decrements on faith:
  it marks the ledger SUSPECT (persisted, narrated once per session) and
  the recount clears the flag with proof.
- getCanonicalCounts() on StorageAdapter (optional) exposes {counted, all}
  per family plus the suspect flag — O(1), no I/O.
- A counts.json written before the ledger existed derives both scalars
  once from the canonical id tree at open and persists them; absent keys
  are a legacy file, never a zero.

User-facing getNounCount()/getVerbCount() are unchanged.

Pinned in tests/integration/canonical-count-ledger.test.ts (5 laws).

src/storage/adapters/baseStorageAdapter.ts

@@ -12,7 +12,8 @@ import {
   HNSWNounWithMetadata,
   HNSWVerbWithMetadata,
   NounMetadata,
-  VerbMetadata
+  VerbMetadata,
+  CanonicalCounts,
 } from '../../coreTypes.js'
 import { StorageBatchConfig } from '../baseStorage.js'
 import { extractFieldNamesFromJson, mapToStandardField } from '../../utils/fieldNameTracking.js'
@@ -1028,6 +1029,28 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
   // Universal count tracking - O(1) operations
   protected totalNounCount = 0
   protected totalVerbCount = 0
+  /**
+   * The ALL-visibility canonical scalars — every noun / verb the unfiltered
+   * storage walk yields, system and internal tiers included. These are the
+   * denominators a derived-index provider's coverage ledger subtracts from
+   * (`posted === all` is the whole-store coverage verdict); the user-facing
+   * `totalNounCount` / `totalVerbCount` skip hidden tiers by design and can
+   * never serve as a ledger denominator. Maintained on the write path
+   * (every new record +1, every proven delete −1), persisted beside the
+   * counted scalars, recomputed by the sanctioned recount. Never clamped.
+   */
+  protected totalNounCountAll = 0
+  protected totalVerbCountAll = 0
+  /**
+   * `true` when a delete could not prove whether the record existed (no
+   * canonical read, no caller-provided prior) — the ALL scalar may be off by
+   * the unprovable deletes since. Loud, persisted, and cleared only by the
+   * sanctioned recount; a consumer reading the scalar as a ledger denominator
+   * must treat a suspect scalar as unverified, never as exact.
+   */
+  protected allCountsSuspect = false
+  /** One narration per session for the suspect transition (never per delete). */
+  private allCountsSuspectNarrated = false
   protected entityCounts: Map<string, number> = new Map() // type -> count
   protected verbCounts: Map<string, number> = new Map() // verb type -> count
   protected countCache: Map<string, { count: number; timestamp: number }> = new Map()
@@ -1056,6 +1079,43 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
     return this.totalVerbCount
   }
 
+  /**
+   * The canonical count ledger — O(1), no I/O. `counted` is the user-facing
+   * scalar (public/internal tiers, what `getNounCount()` returns); `all` is
+   * the ALL-visibility scalar every unfiltered storage walk is measured
+   * against (the coverage-ledger denominator for derived-index providers).
+   * `suspect` is `true` when an unprovable delete has made `all` unverified
+   * since the last sanctioned recount (`rebuildTypeCounts`).
+   * @returns Both scalars per family plus the suspect flag.
+   */
+  async getCanonicalCounts(): Promise<CanonicalCounts> {
+    return {
+      nouns: { counted: this.totalNounCount, all: this.totalNounCountAll },
+      verbs: { counted: this.totalVerbCount, all: this.totalVerbCountAll },
+      suspect: this.allCountsSuspect
+    }
+  }
+
+  /**
+   * Mark the ALL scalars unverified after a delete that could not prove the
+   * record existed. Narrates ONCE per session (the flag is what persists);
+   * the sanctioned recount clears it.
+   * @param family - Which family's delete was unprovable.
+   * @param id - The id whose existence could not be established.
+   */
+  protected markAllCountsSuspect(family: 'noun' | 'verb', id: string): void {
+    this.allCountsSuspect = true
+    if (!this.allCountsSuspectNarrated) {
+      this.allCountsSuspectNarrated = true
+      console.warn(
+        `[Storage] ${family} delete of ${id} could not prove the record existed ` +
+          `(no canonical read, no prior record) — the ALL-visibility count ledger is ` +
+          `SUSPECT until brain.repairIndex() recounts. Further unprovable deletes ` +
+          `this session are counted silently under the same flag.`
+      )
+    }
+  }
+
   /**
    * Increment count for entity type - O(1) operation.
    * Concurrency is handled by the process-global mutex