fix: O(1) adaptive retention accounting + historyStats fleet audit

Under default adaptive retention, every flush() recomputed total history
bytes by walking EVERY committed generation's delta — O(all generations)
with disk re-reads past the 4096-entry delta-cache bound. On a production
brain with 70,000+ accumulated generations this turned every write into a
full-tail scan (60-100s writes, escalating with history growth), even
though the free-RAM budget never tripped and nothing was ever reclaimed
(SELF-GENERATIONS-GROWTH).

historyBytes() now maintains a running total: seeded by one walk on first
use, then updated incrementally at both commit paths (+bytes) and the
compaction reclaim loop (−bytes), dropped on reopenAfterRestore. The
adaptive retention check on every flush is O(1). Invariant regression-
pinned: running total ≡ fresh walk through transact commits, single-op
group commits, and compaction.

New brain.historyStats() (exported HistoryStats): read-only generation
count / bytes / generation+timestamp range / horizon / retention mode /
effective budget — the one-call per-brain fleet audit for retention
exposure.
This commit is contained in:
David Snelling 2026-07-18 10:51:37 -07:00
parent 4fcef7b8ed
commit 6207e48b51
6 changed files with 228 additions and 5 deletions

View file

@ -192,6 +192,7 @@ import { MemoryStorage } from './storage/adapters/memoryStorage.js'
import type {
CompactHistoryOptions,
CompactHistoryResult,
HistoryStats,
TransactOptions,
TransactReceipt,
TxLogEntry,
@ -8211,6 +8212,33 @@ export class Brainy<T = any> implements BrainyInterface<T> {
return this.generationStore.compact(options)
}
/**
* @description Read-only generational-history footprint for fleet audits:
* generation count, total on-disk bytes, generation/timestamp range, the
* compaction horizon, and the retention policy in force. Touches no data and
* changes nothing. First call pays one walk over committed deltas to seed
* the running byte total (subsequent calls and every adaptive retention
* check are then O(1)).
*
* A pool operator's exposure check is one call per brain:
* @example
* const stats = await brain.historyStats()
* console.log(`${stats.generations} generations, ${stats.bytes} bytes, mode=${stats.retentionMode}`)
*/
async historyStats(): Promise<HistoryStats> {
await this.ensureInitialized()
const stats = await this.generationStore.historyStats()
const policy = this.resolveRetentionPolicy()
return {
...stats,
retentionMode: policy.mode,
effectiveBudgetBytes:
policy.mode === 'adaptive'
? this.adaptiveHistoryBudgetBytes(policy.budgetBytes)
: null
}
}
/**
* @description Drive the adaptive retention byte budget at runtime the
* settable input a machine-level coordinator (e.g. cor's `ResourceManager`,