Stage-2 D1 contract item (co-frozen): a fact scan may be slow, never
silent. batches() now races its FIRST pull against
SCANFACTS_FIRST_BATCH_MS (10s, exported; test-overridable) — a wedged
or unreadably slow store produces a loud abort naming the contract
instead of a consumer hanging indistinguishably from progress (the
production shape: a heal against a generations-backlogged brain
wedged silently on the first segment read).
Only the first pull is raced: the bound is time-to-first-batch (proof
the producer is alive), not per-batch pacing, and it runs only while
a pull is pending — consumer think-time between pulls never counts
against the producer (pinned).
Three pins: wedged-store loud failure within the bound, healthy scan
untouched end-to-end, slow-consumer immunity.
Every committed generation now also appends a FACT — an after-image
commit record (what each touched entity/relationship became, or a
body-less tombstone for a removal) — to an append-only, crc32c-framed
segment log under _generations/facts/. The before-image history and the
canonical tree remain authoritative; the fact log gives consumers ONE
sequential, self-verifying stream (index heals, incremental replays)
in place of a per-entity directory walk.
- Wire format: positional msgpack facts [generation, timestamp, ops,
meta, blobHashes]; op = [kind u8, id bin16, record | nil tombstone];
32-byte segment header (magic, formatVersion, firstGeneration,
zeroed+verified reserved); length+crc32c frame per fact; zero-padded
segment names so lexicographic order == generation order; JSON
manifest with an atomic rename flip, manifest-first rotation.
- Commit protocol: facts append+fsync BEFORE the commit point inside
the existing durability window, so a crash can only leave the log
AHEAD of committed truth — open() truncates back (torn tails detected
by CRC). Absent generation = never committed; a scan can never see an
uncommitted fact. transact() facts are durable-on-return; single-op
facts ride the group-commit flush exactly like buffered history. A
fact-append failure fails the write, loudly — a silent gap would be a
lie a later replay discovers.
- New public surface: brain.scanFacts() (sequential batches with heal
telemetry: head/segments/approx up front, per-batch generation range
+ bytes + segment id, loud abort on gaps, summary cross-check) and
brain.factSegmentPaths() (immutable sealed segments for zero-copy
consumers; the mutable tail excluded). Exported types CommitFact,
FactOp, FactScanBatch, FactScanHandle.
- Storage: optional binary raw-byte primitives (appendRawBytes,
readRawBytes, writeRawBytes, rawByteSize) on StorageAdapter —
feature-detected; filesystem + memory adapters implement them; an
adapter without them hosts no fact log. Fact segments are byte-copied
(never hard-linked) into snapshots. The _generations/facts/ namespace
is registered as a protected family (rebuildable: false): no sweeper
or GC may delete under it.
- New crc32c (Castagnoli) utility with RFC known-answer tests.
