feat: temporal VFS — file content joins the Model-B immutability model

The temporal model had a hole exactly where files were concerned: every
entity write is an immutable generation with before-images, but VFS content
BYTES lived under an eager refCount GC left over from the pre-8.0 design —
unlink could physically destroy bytes that in-window history still
referenced, and overwrite never released the old hash at all (an unbounded
silent leak whose accidental byproduct was the only thing "preserving"
history). Reading the past could therefore return a stale field, a dangling
hash, or nothing, depending on luck.

Fix: blob reclamation becomes a HISTORY decision instead of a LIVENESS
decision. Each blob's metadata now carries historyRefCount alongside the
live refCount:

- The commit seam counts one history reference per persisted before-image
  record carrying a content hash (commitTransaction staging and the
  group-commit flush), recorded BEFORE the record-set persists and carried
  in the generation delta (blobHashes — always present on new deltas, so
  compaction only falls back to reading records for pre-contract
  generations). An aborted transaction compensates best-effort.
- unlink/rmdir/overwrite drop ONLY the live reference (BlobStorage.delete →
  release; overwrite finally releases the superseded hash — cancelling the
  dedup increment on same-content rewrites and closing the leak), and only
  AFTER the canonical mutation commits, so a failed delete can never leave a
  live file whose bytes compaction might reclaim.
- History compaction is the ONE reclamation point: after deleting a
  generation's record-set it releases that set's references and physically
  reclaims any hash at zero live AND zero history references. Pins are
  exempt automatically. Crash ordering is over-count-only in every path
  (record before persist, release after delete), so a crash can leak until
  the scrub recounts but can never reclaim bytes a retained generation
  needs. scrubBlobHistoryRefCounts() restores exactness; existing stores get
  a one-time marker-gated backfill on open, failing into leak-safe mode
  (reclamation disabled) rather than guessing.

On top of the protected history, the temporal API the generational model
always implied:

- vfs.readFile(path, { asOf }) — the exact bytes as of a generation or Date,
  materialized from the history (pinned view released so compaction is
  never blocked by a read).
- vfs.history(path) — FileVersion[] ascending ({ generation, timestamp,
  hash, size, mimeType? }), the newest entry being the live state.
- Overwrites now refresh the file entity's data/embedding text — semantic
  search and the data field previously served the FIRST version's text
  forever (the stale-field defect a consumer's incident recovery depended
  on by luck).

Integration suite (temporal-vfs.test.ts): per-version exact reads +
history listing, leak-fix + history protection on overwrite, rm keeps bytes
readable, compaction reclaims past-window bytes and preserves in-window
(including the cross-file dedup case where an old file's history and a
newer file's removal share one hash), data freshness, and scrub exactness.
This commit is contained in:
David Snelling 2026-07-10 16:43:48 -07:00
parent 4af8fb31e2
commit a3467e1f9b
13 changed files with 890 additions and 57 deletions

View file

@ -366,6 +366,61 @@ are done with (including the ones `transact()` returns), and `persist()` any
generation you want to keep beyond the retention window: snapshots are
self-contained and unaffected by compaction.
## Time travel for files (the VFS)
Since 8.2.0, time travel covers Virtual Filesystem **content**, not just
entity records. File bytes are retention-protected: a content blob referenced
by any generation inside the retention window is never reclaimed, so reading
the past always returns the exact bytes — never a stale field or a
dangling hash.
**`vfs.readFile(path, { asOf })`** takes a generation number or a `Date` and
returns the file's exact bytes as they stood then. It resolves the path's
current entity, then materializes its state at the target generation — so it
answers *"what did the file at this path hold at that point?"* It bypasses
the content cache; the `encoding` option still applies. Asking about a
generation before the file existed throws the usual not-found error, and
asking past the retention window's compaction horizon throws a
compacted-generation error.
**`vfs.history(path)`** returns the file's versions inside the retention
window, oldest first — one `FileVersion` per generation that wrote the file,
the newest entry being the current state:
```typescript
// A CMS page evolves…
await brain.vfs.writeFile('/pages/home.json', '{"title":"Launch"}')
await brain.vfs.writeFile('/pages/home.json', '{"title":"Launch v2"}')
await brain.vfs.writeFile('/pages/home.json', '{"title":""}') // bad deploy!
// Every version is listed and readable:
const versions = await brain.vfs.history('/pages/home.json')
// → [{ generation, timestamp, hash, size, mimeType? }, …] ascending
const good = versions[versions.length - 2]
const bytes = await brain.vfs.readFile('/pages/home.json', {
asOf: good.generation
})
// Restore = write the old bytes back. This is a NEW write (a new
// generation) — history is never rewritten, so the bad version stays
// visible in the audit trail.
await brain.vfs.writeFile('/pages/home.json', bytes)
```
Two lifecycle consequences worth stating plainly:
- **Deleting or overwriting a file no longer frees its bytes immediately.**
Old content lives until history compaction reclaims the generations that
reference it — the same `retention` budget that bounds all Model-B history
(and pinned views are exempt, exactly as above). Size your `retention` for
the file-version depth you want; `retention: 'all'` keeps every version of
every file forever.
- **After `compactHistory()` reclaims a generation, its file versions are
gone** and their bytes are physically reclaimed. (This also fixed a
pre-8.2.0 defect where overwritten content was never reclaimed at all — an
unbounded silent leak.)
## From branches to values
If you used the pre-8.0 `fork`/`checkout`/`commit`/`versions` surface, every