fix: recover VFS content blobs stranded by a 7→8 upgrade, in place on open

The 7.x branch system stored Virtual Filesystem content as blobs in its
copy-on-write area (`_cow/`). 8.0 removed that system and stores content blobs
in the content-addressed store (`_cas/`), but the one-time layout migration only
moves entities — it never adopted the `_cow/` content blobs. A store that used
the VFS was left with every VFS-backed read throwing "Blob metadata not found"
and its pages 500ing, with no first-party recovery and the pre-upgrade backup
already removed on entity-migration "success".

Add an on-open recovery pass (`autoAdoptLegacyVfsBlobsIfNeeded`) that runs right
after the layout migration and adopts every orphaned `_cow/` blob into `_cas/` —
copying both the bytes and the metadata via the raw object primitives,
idempotently and non-destructively (the `_cow/` originals are never deleted). It
is a separate phase, not folded into the migration, so it also heals a store
already upgraded by an earlier 8.0.x that stranded the blobs (whose layout-
migration marker is stamped): it is gated on the presence of `_cow/` and its own
`_system/vfs-blob-adoption.json` marker. Filesystem-only; native-8.0 and non-VFS
stores no-op on a cheap existence check.

- `BaseStorage.adoptLegacyCowBlobs()` — the scan/copy primitive, returning
  `{ cowBlobs, adopted, alreadyPresent, incomplete }`; skips (does not half-adopt)
  a blob missing its bytes or metadata.
- `brain.vfs.adoptOrphanedBlobs()` — the explicit force path; self-initializes.
- Backup retention: the automatic pre-upgrade backup is now kept if any blob
  can't be fully adopted (`incomplete > 0`), instead of being removed on
  entity-migration success alone — a blob-parity gate the migration signal
  cannot provide.

Adds an end-to-end integration test (storage-level adopt with idempotency and
incomplete handling; self-heal on reopen; the explicit API) and an upgrade guide.
This commit is contained in:
David Snelling 2026-07-07 12:23:36 -07:00
parent 6821e1980b
commit c0f6ccd958
5 changed files with 584 additions and 1 deletions

View file

@ -680,6 +680,91 @@ export abstract class BaseStorage extends BaseStorageAdapter {
this.blobStorage = new BlobStorage(casAdapter)
}
/**
* @description Adopt orphaned 7.x copy-on-write VFS content blobs into the 8.0
* content-addressed store. 7.x kept VFS blobs (`blob:<hash>` bytes +
* `blob-meta:<hash>` JSON) under the copy-on-write area (`_cow/`) of the
* branch/versioning system that 8.0 removed. The 78 layout migration
* collapses entity files but historically did NOT bridge these blobs, so a VFS
* read of a still-in-`_cow/` blob throws "Blob metadata not found" and every
* page backed by it 500s.
*
* This scans `_cow/` and, for every blob whose `blob:`/`blob-meta:` pair is not
* already present in the 8.0 store (`_cas/`), copies BOTH objects across
* verbatim. Copying preserves the exact content bytes (so the content hash and
* every VFS reference still resolve) and the exact metadata (so the 7.x
* refCount is retained) the only first-party, index-consistent recovery.
* Hand-moving files at the OS level bypasses the paired-metadata contract and
* is unsafe; this goes through the adapter's object primitives.
*
* Idempotent and non-destructive: a pair already in `_cas/` is left untouched
* (counted `alreadyPresent`); the `_cow/` originals are never deleted, so a
* re-run or a rollback is always possible. A no-op on memory storage and
* on any brain without a `_cow/` area (returns zeros). Bytes are written before
* metadata so an interruption can only leave the pre-adoption "no metadata"
* state (retryable), never a metadata-without-bytes blob.
*
* @returns `cowBlobs` (distinct blob hashes found in `_cow/`), `adopted`
* (newly copied into `_cas/`), `alreadyPresent` (already in `_cas/`),
* `incomplete` (a `_cow/` blob missing its bytes or its metadata skipped
* and reported rather than half-adopted).
*/
public async adoptLegacyCowBlobs(): Promise<{
cowBlobs: number
adopted: number
alreadyPresent: number
incomplete: number
}> {
let cowPaths: string[]
try {
cowPaths = await this.listObjectsUnderPath('_cow/')
} catch {
// No `_cow/` area (fresh/native-8.0 brain, or a non-listing backend).
return { cowBlobs: 0, adopted: 0, alreadyPresent: 0, incomplete: 0 }
}
// Distinct content-blob hashes from `_cow/blob:<hash>` keys. Non-blob `_cow/`
// entries (7.x branch COW state) are ignored — only VFS content is adopted.
const hashes = new Set<string>()
for (const p of cowPaths) {
const key = p.replace(/^_cow\//, '')
const m = /^blob:(.+)$/.exec(key)
if (m) hashes.add(m[1])
}
let adopted = 0
let alreadyPresent = 0
let incomplete = 0
for (const hash of hashes) {
// A blob counts as present only when BOTH its bytes and its metadata
// already live in `_cas/`. A half-adopted blob (bytes without meta — the
// exact "Blob metadata not found" state) is re-adopted.
const casBlob = await this.readObjectFromPath(`_cas/blob:${hash}`)
const casMeta = await this.readObjectFromPath(`_cas/blob-meta:${hash}`)
if (casBlob !== null && casMeta !== null) {
alreadyPresent++
continue
}
const cowBlob = await this.readObjectFromPath(`_cow/blob:${hash}`)
const cowMeta = await this.readObjectFromPath(`_cow/blob-meta:${hash}`)
if (cowBlob === null || cowMeta === null) {
// Can't register a blob the store can't fully describe — report it so an
// operator investigates rather than silently half-adopting.
incomplete++
continue
}
// Bytes first, then metadata: a VFS read checks metadata before bytes, so
// metadata's presence must imply the bytes are already there.
await this.writeObjectToPath(`_cas/blob:${hash}`, cowBlob)
await this.writeObjectToPath(`_cas/blob-meta:${hash}`, cowMeta)
adopted++
}
return { cowBlobs: hashes.size, adopted, alreadyPresent, incomplete }
}
/**
* @description Write a canonical object (write-cache coherent). The object
* lands in the write-through cache before the asynchronous write starts, so