feat: flush() never compacts — history maintenance moves to close() with bounded passes

flush() is durability work: it must cost what the current window's
deltas cost, never what the history backlog costs. Under adaptive
retention the byte budget derives from free memory, so bulk-load
pressure shrank the budget exactly at peak write volume and flush paid
actual reclaim inline — a production deployment measured single writes
blocked 25-191s behind reclaim-on-flush.

- flush() no longer calls autoCompactHistory(); close() is THE
  auto-compaction site (already ran there; now alone).
- Every auto pass is time-bounded (CLOSE_COMPACTION_BUDGET_MS = 5s):
  reclamation is oldest-first, so an early stop is a consistent prefix
  and the next pass resumes. Explicit compactHistory() gains an
  optional timeBudgetMs for caller-chosen maintenance windows.
- Documented trade stated where operators read: a long-lived writer
  that never closes accumulates history until its next explicit
  compactHistory() — predictable writes, explicit maintenance.

Pins: flush-never-reclaims + close-reclaims-durably (db-mvcc), bounded
pass stops-then-resumes as a consistent prefix (generationStore unit).

src/brainy.ts

@@ -396,6 +396,15 @@ export type IndexFamily = 'vector' | 'metadata' | 'graph'
  */
 const AGGREGATION_BACKFILL_RETRY_COOLDOWN_MS = 30_000
 
+/**
+ * Time budget for the auto-compaction pass at close() (8.9.0). Bounds how long
+ * a clean shutdown spends reclaiming history backlog — an early stop is a
+ * consistent prefix and the next close/explicit pass resumes. Explicit
+ * `compactHistory()` calls are unbounded unless the caller passes their own
+ * `timeBudgetMs` (maintenance windows choose their own budgets).
+ */
+const CLOSE_COMPACTION_BUDGET_MS = 5_000
+
 /**
  * The main Brainy class - Clean, Beautiful, Powerful
  * REAL IMPLEMENTATION - No stubs, no mocks
@@ -8362,19 +8371,24 @@ export class Brainy<T = any> implements BrainyInterface<T> {
 
   /**
    * @description Run history compaction under the resolved `retention` policy
-   * when `autoCompact` is on (the default). Invoked from `flush()` and
-   * `close()` so generational record-sets cannot accumulate unbounded across a
-   * long-lived writer's lifetime.
+   * when `autoCompact` is on (the default). Invoked from `close()` ONLY
+   * (8.9.0) — flush() is durability work and never pays maintenance costs; a
+   * production deployment measured reclaim-on-flush blocking single writes
+   * for 25-191s under memory pressure. Long-lived writers that never close
+   * accumulate history until their next explicit `compactHistory()` — the
+   * documented trade: predictable writes, explicit maintenance.
    *
    * - `'all'` → returns without reclaiming (index compaction for speed still
    *   runs elsewhere; history is decoupled and kept).
    * - `'adaptive'` → reclaim oldest-unpinned history down to the byte budget.
    * - `'explicit'` → apply the supplied `maxGenerations`/`maxAge`/`maxBytes` caps.
    *
+   * Every auto pass is TIME-BOUNDED ({@link CLOSE_COMPACTION_BUDGET_MS}) so a
+   * large backlog can never stall a clean shutdown; the next pass resumes.
    * Read-only instances and an explicit `autoCompact: false` skip silently.
    * Pinned generations are never reclaimed ({@link GenerationStore.compact}).
    * Failures are logged and swallowed — compaction is housekeeping and must
-   * never fail a flush or a clean shutdown.
+   * never fail a clean shutdown.
    */
   private async autoCompactHistory(): Promise<void> {
     // Nothing to compact on a read-only instance or before init wired up the
@@ -8390,12 +8404,16 @@ export class Brainy<T = any> implements BrainyInterface<T> {
       if (policy.mode === 'adaptive') {
         const budget = this.adaptiveHistoryBudgetBytes(policy.budgetBytes)
         if (budget === Infinity) return // no pressure signal → keep everything this pass
-        await this.generationStore.compact({ maxBytes: budget })
+        await this.generationStore.compact({
+          maxBytes: budget,
+          timeBudgetMs: CLOSE_COMPACTION_BUDGET_MS
+        })
       } else {
         await this.generationStore.compact({
           maxGenerations: policy.maxGenerations,
           maxAge: policy.maxAge,
-          maxBytes: policy.maxBytes
+          maxBytes: policy.maxBytes,
+          timeBudgetMs: CLOSE_COMPACTION_BUDGET_MS
         })
       }
     } catch (error) {
@@ -10540,12 +10558,14 @@ export class Brainy<T = any> implements BrainyInterface<T> {
       this.generationStore.persistCounterNow()
     ])
 
-    // 6. Auto-compact generational history per config.retention (default on).
-    //    Runs after the flush so durable state is in place; respects live
-    //    Db pins and an explicit autoCompact: false.
-    await this.autoCompactHistory()
+    // NOTE (8.9.0): flush() no longer compacts history. Flush is DURABILITY
+    // work — it must cost what this window's deltas cost, never what the
+    // history backlog costs. Auto-compaction (a MAINTENANCE concern) runs at
+    // close() and via explicit compactHistory(); a production deployment
+    // measured reclaim-on-flush stalling single writes for 25-191s under
+    // memory pressure, which is exactly the class this separation ends.
 
-    // 7. Stamp the entity tree: which source generation the canonical tree
+    // 6. Stamp the entity tree: which source generation the canonical tree
     //    reflects + the rollup invariants that verify it whole (the counters
     //    persisted in step 1). Written at flush boundaries — the tree tracks
     //    every commit by construction, so the stamp is a durable checkpoint,
@@ -16173,8 +16193,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
     }
 
     // Phase 0b: Auto-compact generational history per config.retention (default
-    // on) BEFORE the generation store closes below. Respects live Db pins and
-    // an explicit autoCompact: false; no-op on read-only instances.
+    // on) BEFORE the generation store closes below. This is THE auto-compaction
+    // site (8.9.0 — flush() never compacts): time-bounded per pass, respects
+    // live Db pins and an explicit autoCompact: false; no-op on read-only
+    // instances.
     await this.autoCompactHistory()
 
     // Phase 1: Flush ALL components in parallel to persist buffered data

src/db/generationStore.ts

@@ -2220,6 +2220,11 @@ export class GenerationStore {
       const maxAge = options?.maxAge
       const maxBytes = options?.maxBytes
       const ageCutoff = maxAge !== undefined ? Date.now() - maxAge : undefined
+      // Bounded maintenance pass (8.9.0): stop reclaiming once the budget is
+      // spent. Safe mid-loop — reclamation is oldest-first, so an early stop
+      // leaves a consistent contiguous prefix and the next pass resumes.
+      const deadline =
+        options?.timeBudgetMs !== undefined ? Date.now() + options.timeBudgetMs : undefined
       const noCaps =
         maxGenerations === undefined && maxAge === undefined && maxBytes === undefined
 
@@ -2233,6 +2238,7 @@ export class GenerationStore {
       for (const gen of [...this.committedGensAsc()]) {
         // Pins are always exempt: never reclaim a generation a live pin needs.
         if (gen > minPinned) break // committedGensAsc ascending → nothing newer is eligible either
+        if (deadline !== undefined && Date.now() >= deadline) break // budget spent — resume next pass
         const delta = await this.getDelta(gen)
         if (!noCaps) {
           const violatesCount = maxGenerations !== undefined && remainingCount > maxGenerations

src/db/types.ts

@@ -176,6 +176,15 @@ export interface CompactHistoryOptions {
    * of each surviving generation's serialized record set (`GenerationDelta.bytes`).
    */
   maxBytes?: number
+  /**
+   * Stop reclaiming after this many milliseconds even if caps are still
+   * exceeded (8.9.0). Compaction is maintenance — a bounded pass keeps
+   * `close()` (and any explicit maintenance window) from stalling on a large
+   * backlog; the next pass resumes where this one stopped (reclamation is
+   * oldest-first, so an early stop is always a consistent prefix). Unset =
+   * run to completion.
+   */
+  timeBudgetMs?: number
 }
 
 /**

src/types/brainy.types.ts

@@ -1737,7 +1737,10 @@ export interface BrainyConfig {
    * Under Model-B EVERY write (`transact()` AND single-op `add`/`update`/
    * `remove`/`relate`) produces an immutable generation record-set serving
    * historical reads (`asOf()`, pinned `Db` values). Without compaction those
-   * accumulate, so Brainy **auto-compacts on every `flush()` and `close()`**.
+   * accumulate, so Brainy **auto-compacts at `close()`** (time-bounded per
+   * pass; 8.9.0 removed compaction from `flush()` — flush is durability work
+   * and never pays maintenance costs). A long-lived writer that never closes
+   * accumulates history until its next explicit `compactHistory()` call.
    * Live `Db` pins are ALWAYS exempt from reclamation, in every mode.
    *
    * Modes:
@@ -1754,7 +1757,7 @@ export interface BrainyConfig {
    *   the oldest unpinned generations while ANY supplied cap is exceeded
    *   (predictable ops). `maxAge` in ms; `maxBytes` total history bytes.
    *
-   * `autoCompact: false` disables the automatic flush/close compaction (manage
+   * `autoCompact: false` disables the automatic close() compaction (manage
    * manually via `brain.compactHistory()`). `budgetBytes` is the settable
    * adaptive byte budget a coordinator drives (also via `brain.setRetentionBudget()`).
    * Long-term archives belong in `db.persist(path)` snapshots, which compaction
@@ -1772,7 +1775,7 @@ export interface BrainyConfig {
         maxBytes?: number
         /** Adaptive byte budget for this brain, driven by a coordinator (e.g. cor). */
         budgetBytes?: number
-        /** Run compaction automatically on flush()/close() (default: true). */
+        /** Run compaction automatically at close() (default: true; 8.9.0 — flush() never compacts). */
         autoCompact?: boolean
       }