perf(open): a sealed segment the manifest proves is below the bound is never read

Every log-authority open asks the fact log one question — is there a fact
above the committed pointer? — and answered it by reading and CRC-decoding
EVERY segment file the manifest names. MEASURED in production on a 16k-row
brain at generation ~478,819: 34-37 seconds inside `generation-store-open-fold`
on every open, including the clean one where the answer is always "nothing".

The manifest already knows. A sealed segment's `lastGeneration` is written at
seal time, and the seal order has been the same since the log was introduced:
`rotate()` fsyncs the tail's bytes FIRST ("sealed segments are always fully
durable"), builds the entry from the content that fsync covered, and only then
flips the manifest — atomically, fsynced, and in the same write re-pointing
`tailSegment`, so a sealed file is never appended to again. A crash in that
order is safe in the pruning direction: before the manifest write the segment
is still the TAIL and is read whole; after it, the entry describes bytes that
were already durable. The only later mutation of a sealed segment is open()'s
straddle truncation, which removes facts and re-derives the entry from the
actual bytes — a recorded bound can drift DOWN with its file, never up.

So `lastGeneration = L` proves the file holds no fact above L, and both
manifest-direct passes (`peekFactsAbove` and its streaming twin, the recovery
fold) now read only the unsealed tail, entries with no numeric
`lastGeneration` — legacy or hand-repaired manifests, never prune what you
cannot prove — and entries whose recorded maximum is actually above the bound.
The open narrates what it read and what it pruned when the log holds more than
one segment.

Pinned in tests/integration/factlog-open-prune.test.ts, from the log's own
counters rather than a clock: a clean reopen over five sealed segments reads
exactly the tail (1 of 6) and finds nothing; a real SIGKILLed writer that
sealed segments holding facts above the committed pointer has those segments
READ, and its peek, its fold stream and its rollback all match the unpruned
full scan fact for fact; a manifest entry missing `lastGeneration` is read.

src/db/factLog.ts

@@ -40,7 +40,10 @@
  * The manifest (`_generations/facts/manifest.json`, JSON — forensics stay
  * terminal-readable) is the single source of truth for the segment SET;
  * rotation flips it atomically (write-new → fsync → rename) BEFORE the new
- * tail's first byte exists, so no segment file is ever unaccounted for.
+ * tail's first byte exists, so no segment file is ever unaccounted for. Its
+ * per-segment `firstGeneration`/`lastGeneration` are LOAD-BEARING at open: a
+ * recovery pass looking for facts above a bound reads only the segments those
+ * bounds cannot rule out (the prune law — see `segmentsHoldingFactsAbove`).
  *
  * ## Mixed-version logs (the v2 live-write cutover)
  *
@@ -689,6 +692,74 @@ function parseSegment(
   return { facts, validBytes: offset, formatVersion: FACT_LOG_FORMAT_V1 }
 }
 
+/**
+ * THE PRUNE LAW — which segment files a pass looking for facts ABOVE
+ * `committedGeneration` actually has to read, and how many the manifest's own
+ * recorded bounds took off the table.
+ *
+ * A sealed segment's `lastGeneration` is written at SEAL time and never
+ * mutated upward afterwards ({@link FactLog.rotate}, unchanged since the log
+ * was introduced): the tail's bytes are fsynced FIRST (`await this.sync()` —
+ * "sealed segments are always fully durable"), the entry is then built from
+ * the content that fsync covered, and only then does the manifest flip —
+ * atomically (tmp+rename) and fsynced — which in the SAME write re-points
+ * `tailSegment` at a new file, so the sealed file is never appended to again.
+ * A crash anywhere in that order is safe in the pruning direction: crash
+ * before the manifest write and the segment is still the TAIL (read whole);
+ * crash after it and the entry describes bytes that were already durable. The
+ * only later mutation of a sealed segment is `open()`'s straddle truncation,
+ * which REMOVES facts and re-derives the entry from the actual bytes — so a
+ * recorded bound can drift DOWN with its file, never up.
+ *
+ * Therefore: `lastGeneration = L` proves the file holds no fact above L, and
+ * a pass above `committedGeneration >= L` can skip it whole — no read, no
+ * CRC decode, no msgpack. What the manifest cannot PROVE is never pruned: an
+ * entry with no numeric `lastGeneration` (a legacy or hand-repaired manifest)
+ * is read, and the unsealed tail is always read.
+ *
+ * This is the difference between an open that costs O(whole fact log) and one
+ * that costs O(the facts that could matter). MEASURED in production: a 16k-row
+ * brain at generation ~478,819 paid 34-37s of segment reads and CRC decoding
+ * in `generation-store-open-fold` on EVERY open — to answer a question whose
+ * answer, after a clean close, is always "nothing".
+ */
+function segmentsHoldingFactsAbove(
+  stored: FactsManifest,
+  committedGeneration: number
+): { files: string[]; pruned: number } {
+  const files: string[] = []
+  let pruned = 0
+  for (const entry of stored.segments) {
+    const last = (entry as Partial<SegmentEntry>).lastGeneration
+    if (typeof last === 'number' && Number.isFinite(last) && last <= committedGeneration) {
+      pruned++
+      continue
+    }
+    files.push(entry.file)
+  }
+  if (stored.tailSegment) files.push(stored.tailSegment)
+  return { files, pruned }
+}
+
+/**
+ * Say what the open actually read. One line, and only when the log holds more
+ * than one segment (a single-segment log has nothing to prune and nothing to
+ * report) — the operator's receipt that the open is paying for the tail, not
+ * for the whole history.
+ */
+function narrateAboveScan(
+  pass: string,
+  committedGeneration: number,
+  read: number,
+  pruned: number
+): void {
+  if (read + pruned <= 1) return
+  prodLog.narrate(
+    `[FactLog] ${pass} above generation ${committedGeneration}: ${read} segment(s) read, ` +
+      `${pruned} pruned of ${read + pruned} (sealed at or below the bound)`
+  )
+}
+
 /**
  * The generation fact log. One instance per open store; every method assumes
  * the single-writer discipline the generation store already enforces (calls
@@ -754,22 +825,6 @@ export class FactLog {
     return this.manifest.brainId !== undefined || this.tailVersion === FACT_LOG_FORMAT_V2
   }
 
-  /**
-   * Open the log and reconcile it to committed truth: read the manifest,
-   * establish the tail's intact content (torn-tail scan), then TRUNCATE any
-   * fact with `generation > committedGeneration` — those never committed (a
-   * crash between fact-append and the commit point). After open, the log is
-   * exactly the committed prefix.
-   */
-  /**
-   * Read (without truncating) every intact fact ABOVE a generation — the
-   * log-authority recovery surface: after a crash, facts beyond the
-   * manifest watermark that survived with valid CRCs are ACKED writes in
-   * durable-at-ack mode, and the owner REPLAYS them instead of letting
-   * open() truncate them. Must be called BEFORE open() (it reads the raw
-   * segments directly; the torn tail's invalid suffix is ignored exactly
-   * like open() would).
-   */
   /**
    * STREAMING twin of {@link FactLog.peekFactsAbove} for the recovery fold:
    * yields facts above the bound one SEGMENT at a time, ascending, without
@@ -779,13 +834,18 @@ export class FactLog {
    * Works manifest-direct (safe before {@link FactLog.open}). Ordering is
    * structural (segments rotate in order; appends are ordered within one) and
    * ASSERTED — a violation aborts loudly, never a silent misordered replay.
+   *
+   * Reads only the segments that CAN hold a fact above the bound — see
+   * {@link segmentsHoldingFactsAbove}. A bounded fold above a high checkpoint
+   * therefore reads its own tail, not the whole history it already proved
+   * durable.
    */
   async *streamFactsAbove(committedGeneration: number): AsyncGenerator<CommitFact[], void> {
     const stored = (await this.storage.readRawObject(FACTS_MANIFEST_PATH)) as FactsManifest | null
     if (!stored || typeof stored !== 'object' || !Array.isArray(stored.segments)) return
     if (stored.formatVersion !== FACTS_FORMAT_VERSION) return
-    const files = [...stored.segments.map((s) => s.file)]
-    if (stored.tailSegment) files.push(stored.tailSegment)
+    const { files, pruned } = segmentsHoldingFactsAbove(stored, committedGeneration)
+    narrateAboveScan('recovery fold', committedGeneration, files.length, pruned)
     let lastGen = committedGeneration
     for (const file of files) {
       const bytes = await this.storage.readRawBytes(`${FACTS_PREFIX}/${file}`)
@@ -807,13 +867,27 @@ export class FactLog {
     }
   }
 
+  /**
+   * Read (without truncating) every intact fact ABOVE a generation — the
+   * log-authority recovery surface: after a crash, facts beyond the
+   * manifest watermark that survived with valid CRCs are ACKED writes in
+   * durable-at-ack mode, and the owner REPLAYS them instead of letting
+   * open() truncate them. Must be called BEFORE open() (it reads the raw
+   * segments directly; the torn tail's invalid suffix is ignored exactly
+   * like open() would).
+   *
+   * Reads only the segments that CAN hold such a fact — see
+   * {@link segmentsHoldingFactsAbove}. This runs on EVERY log-authority open,
+   * including the clean one where the answer is always empty, so the segments
+   * the manifest already proves irrelevant are never opened at all.
+   */
   async peekFactsAbove(committedGeneration: number): Promise<CommitFact[]> {
     const stored = (await this.storage.readRawObject(FACTS_MANIFEST_PATH)) as FactsManifest | null
     if (!stored || typeof stored !== 'object' || !Array.isArray(stored.segments)) return []
     if (stored.formatVersion !== FACTS_FORMAT_VERSION) return []
     const out: CommitFact[] = []
-    const files = [...stored.segments.map((s) => s.file)]
-    if (stored.tailSegment) files.push(stored.tailSegment)
+    const { files, pruned } = segmentsHoldingFactsAbove(stored, committedGeneration)
+    narrateAboveScan('above-manifest peek', committedGeneration, files.length, pruned)
     for (const file of files) {
       const bytes = await this.storage.readRawBytes(`${FACTS_PREFIX}/${file}`)
       if (bytes === null) continue
@@ -826,6 +900,13 @@ export class FactLog {
     return out
   }
 
+  /**
+   * Open the log and reconcile it to committed truth: read the manifest,
+   * establish the tail's intact content (torn-tail scan), then TRUNCATE any
+   * fact with `generation > committedGeneration` — those never committed (a
+   * crash between fact-append and the commit point). After open, the log is
+   * exactly the committed prefix.
+   */
   async open(committedGeneration: number): Promise<void> {
     const stored = (await this.storage.readRawObject(FACTS_MANIFEST_PATH)) as FactsManifest | null
     if (stored && typeof stored === 'object' && Array.isArray(stored.segments)) {