open-brainy/src/aggregation
David Snelling 1dc861d299
Some checks failed
CI / Node 22 (push) Has been cancelled
CI / Node 24 (push) Has been cancelled
CI / Bun (latest) (push) Has been cancelled
fix(aggregation): the lifecycle cluster — flush stamps, behind-stamp catches up incrementally, the native rebuild finally gets invoked, deletes are never silently skipped
SELF-ENGINE-LIFECYCLE-SPRINT + BRAINY-PROD-LATENCY-TRIAD, the four asks:

(a) brain.flush() persists aggregation state stamped at the committed
    generation. The stamp used to advance only at close(), so a long-lived
    writer that flushes but never closes — the primary production shape —
    left every write window behind the stamp, and ANY unclean exit forced a
    whole-store backfill walk (per-entity work, measured >60s and
    door-starving on a 9k-row production brain) on the first stats call.

(b) BEHIND-stamp adoption becomes adopt + INCREMENTAL CATCH-UP: the exact
    missing window (stamp, committed] resolves its affected-id set from the
    fact log and reconciles each entity with time-travel before/after reads
    (asOf at both window bounds) through the same delta algebra the live
    hooks use — cost bounded by writes since the last flush, never store
    size, and exact under interleaving because reconciliation targets the
    FIXED window end while later writes chain through hooks. Oversized
    windows (>5000 affected) and unreadable windows demote to the announced
    rescan — never a silent partial serve.

(c) The native provider's parallel rebuildAggregate — on the contract since
    8.x but never invoked anywhere — is now the backfill walk's preferred
    door: one call per aggregate with source-matched entities, replacing
    the per-entity FFI stream.

(d) A delete whose before-image is unavailable can no longer SKIP the
    aggregation hook silently (counts drifted upward forever): both delete
    paths (remove() and transact) flag an exact rescan, loudly.

Pins: integration (flush stamp; unclean-exit reopen → exact counts through
an add + group-move + delete window with the walk spy proving ZERO
whole-store walks) + unit (provider rebuild invoked once with filtered
entities; flagAllForRescan; reconcile delta algebra). Gates: unit 1913/1913
· integration 760 · conformance 27/27.
2026-08-05 15:49:12 -07:00
..
AggregationIndex.ts fix(aggregation): the lifecycle cluster — flush stamps, behind-stamp catches up incrementally, the native rebuild finally gets invoked, deletes are never silently skipped 2026-08-05 15:49:12 -07:00
index.ts feat: add aggregation engine with incremental SUM/COUNT/AVG/MIN/MAX, GROUP BY, and time windows 2026-02-16 16:57:53 -08:00
materializer.ts fix: aggregation surfaces materialize/state-load failures loudly 2026-07-13 09:49:38 -07:00
timeWindows.ts feat: add aggregation engine with incremental SUM/COUNT/AVG/MIN/MAX, GROUP BY, and time windows 2026-02-16 16:57:53 -08:00