soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

fix: getNouns cursor pagination re-scanned the first page forever (permanent CPU loop)

Browse source 
The shard-scan pagination adapter is offset-based and ignored the cursor, so
getNouns({ pagination: { cursor } }) re-returned the first page on every cursor
call. Harmless until 7.32.1 made totalCount the true dataset total — after which
hasMore correctly stays true until a caller has paged through everything. A
caller that paginates by cursor (cursor = page.nextCursor) — notably aggregate
backfill over an already-populated store — then looped forever, re-walking the
entire entity shard tree each iteration and pegging 1-2 CPU cores permanently on
the JS main thread, with zero queries or traffic. (Pre-7.32.1 the same loop
ended after one page, silently backfilling only the first 500 entities — an
incomplete aggregate.)

getNouns() now treats the cursor as an opaque, advancing offset token, so cursor
pagination advances and terminates exactly like offset pagination — and an
aggregate backfill streams the whole corpus exactly once (no longer truncated,
no longer looping). Hardened the backfill loop to pure offset pagination as
defense in depth.

Regression (reproduces the infinite loop, fails fast on any re-scan):
tests/unit/storage/getNouns-cursor-pagination.test.ts
This commit is contained in:
David Snelling 2026-06-22 18:00:01 -07:00
parent 526aaad18f
commit 6721c52ad7
4 changed files with 209 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

29
RELEASES.md
View file

@ -10,6 +10,35 @@ Full auto-generated changelog: `CHANGELOG.md` · Releases: https://github.com/so
---
## v7.33.1 — 2026-06-22
**Affected products:** any consumer on `filesystem` / `mmap-filesystem` storage with **more than one
page (> 500) of entities** that defines an **aggregate** over an already-populated brain (the common
reopen-then-`defineAggregate()`/`queryAggregate()` case). **Critical fix** — a permanent post-init
CPU loop. Drop-in; no API or data changes. **Upgrade from 7.32.1 / 7.33.0 immediately.**
### Fix — `getNouns()` cursor pagination no longer re-scans the first page forever (permanent CPU loop)
`storage.getNouns({ pagination: { cursor } })` paginated by **cursor** never advanced: the shard-scan
adapter is offset-based and **ignored** the cursor, so every cursor call re-returned the first page.
On its own that was harmless — until v7.32.1 correctly made `totalCount` the true dataset total, which
makes `hasMore` stay `true` until a caller has paged through everything. A caller that paginates by
cursor (`cursor = page.nextCursor`) — notably **aggregate backfill** (`defineAggregate()` /
`queryAggregate()` over a pre-populated store) — therefore looped **forever**, re-walking the entire
entity shard tree on every iteration. With more than one page of entities this pegged 1–2 CPU cores
permanently on the JS main thread — with zero queries or traffic — and never settled. (Before v7.32.1
the very same loop terminated after one page, silently backfilling only the first 500 entities — an
incomplete aggregate.)
`getNouns()` now treats the cursor as an **opaque, advancing offset token**, so cursor pagination
behaves exactly like offset pagination — it advances and terminates — and an aggregate backfill now
streams the **whole** corpus exactly once (no longer truncated at 500, no longer looping). The backfill
loop was additionally hardened to pure offset pagination as defense in depth. Regression that
reproduces the infinite loop and fails fast on any re-scan:
`tests/unit/storage/getNouns-cursor-pagination.test.ts`.
---
## v7.32.1 — 2026-06-17
**Affected products:** consumers on `filesystem` / `mmap-filesystem` storage whose logs showed