fix: exception-safe aggregation backfill + generation-verified adoption + loud open-path guards

- Aggregation backfill walks build into a STAGING map and swap in atomically
  on completion. A mid-walk failure drops the staging map — the previous live
  state keeps serving, the aggregate stays flagged pending, and the storage
  error surfaces to the failing query. Previously the walk wiped live state
  before a scan that could throw, never cleared the pending flag on failure,
  and re-ran a full walk on every subsequent query: a silent wipe/walk/throw
  loop at the caller's retry rate.
- Failed walks are latched: retries within a 30s cooldown rethrow the recorded
  error instantly instead of re-walking, so a tight caller-side retry loop
  costs one loud error per query, never a full store walk per query.
- Persisted aggregation state is stamped with the store's committed generation
  at flush; reopen adoption requires stamp equality. Stale state (unclean
  shutdown) or over-counting state (a log truncation on a copied store pulled
  the watermark back) triggers exactly one loud rescan, never a silent adopt.
- The backfill/adoption path narrates: adoption decisions, walk start/finish
  with counts and duration, and failures all log by default.
- getNouns/getVerbs refuse a supplied-but-undecodable pagination cursor with a
  loud error instead of silently restarting the walk at offset 0 (which
  re-served page 1 forever to any while(hasMore) caller).
- The graph cold-load verb walk aborts loudly on a missing or non-advancing
  cursor with hasMore=true.
- A versioned index provider whose generation is AHEAD of the committed
  watermark (torn copy / crash-recovery truncation) is now named loudly at
  open, alongside the existing behind-direction message.
This commit is contained in:
David Snelling 2026-07-17 16:00:11 -07:00
parent 01a7f3dd01
commit a77b064bd7
7 changed files with 335 additions and 36 deletions

View file

@ -95,9 +95,15 @@ describe('verb cursor pagination (graph-perf #2)', () => {
expect(new Set(cursorSeen)).toEqual(new Set(offsetSeen))
})
it('a foreign/malformed cursor falls back gracefully (no throw, starts from the beginning)', async () => {
const page = await storage.getVerbs({ pagination: { limit: 5, cursor: 'not-a-cv1-token' } })
expect(page.items.length).toBe(5)
expect(page.hasMore).toBe(true)
it('a foreign/malformed cursor FAILS LOUDLY — never a silent restart from page 1', async () => {
// The old behavior (decode-null → silent offset-0 fallback) re-served page 1
// forever to any while(hasMore) walker: an unbounded CPU loop with no log
// line. An undecodable resume token now refuses the walk instead.
await expect(
storage.getVerbs({ pagination: { limit: 5, cursor: 'not-a-cv1-token' } })
).rejects.toThrow('invalid pagination cursor')
await expect(
storage.getNouns({ pagination: { limit: 5, cursor: 'not-a-cv1-token' } })
).rejects.toThrow('invalid pagination cursor')
})
})