fix: recover VFS content blobs stranded by a 7→8 upgrade, in place on open
The 7.x branch system stored Virtual Filesystem content as blobs in its
copy-on-write area (`_cow/`). 8.0 removed that system and stores content blobs
in the content-addressed store (`_cas/`), but the one-time layout migration only
moves entities — it never adopted the `_cow/` content blobs. A store that used
the VFS was left with every VFS-backed read throwing "Blob metadata not found"
and its pages 500ing, with no first-party recovery and the pre-upgrade backup
already removed on entity-migration "success".
Add an on-open recovery pass (`autoAdoptLegacyVfsBlobsIfNeeded`) that runs right
after the layout migration and adopts every orphaned `_cow/` blob into `_cas/` —
copying both the bytes and the metadata via the raw object primitives,
idempotently and non-destructively (the `_cow/` originals are never deleted). It
is a separate phase, not folded into the migration, so it also heals a store
already upgraded by an earlier 8.0.x that stranded the blobs (whose layout-
migration marker is stamped): it is gated on the presence of `_cow/` and its own
`_system/vfs-blob-adoption.json` marker. Filesystem-only; native-8.0 and non-VFS
stores no-op on a cheap existence check.
- `BaseStorage.adoptLegacyCowBlobs()` — the scan/copy primitive, returning
`{ cowBlobs, adopted, alreadyPresent, incomplete }`; skips (does not half-adopt)
a blob missing its bytes or metadata.
- `brain.vfs.adoptOrphanedBlobs()` — the explicit force path; self-initializes.
- Backup retention: the automatic pre-upgrade backup is now kept if any blob
can't be fully adopted (`incomplete > 0`), instead of being removed on
entity-migration success alone — a blob-parity gate the migration signal
cannot provide.
Adds an end-to-end integration test (storage-level adopt with idempotency and
incomplete handling; self-heal on reopen; the explicit API) and an upgrade guide.
This commit is contained in:
parent
6821e1980b
commit
c0f6ccd958
5 changed files with 584 additions and 1 deletions
186
docs/guides/upgrading-7-to-8.md
Normal file
186
docs/guides/upgrading-7-to-8.md
Normal file
|
|
@ -0,0 +1,186 @@
|
||||||
|
---
|
||||||
|
title: Upgrading from 7.x to 8.0
|
||||||
|
slug: guides/upgrading-7-to-8
|
||||||
|
public: true
|
||||||
|
category: guides
|
||||||
|
template: guide
|
||||||
|
order: 10
|
||||||
|
description: What the one-time 7→8 on-disk migration does, how 8.0 automatically recovers Virtual Filesystem content that older layouts stored in the removed copy-on-write area, and how to verify (or force) that recovery.
|
||||||
|
next:
|
||||||
|
- guides/storage-adapters
|
||||||
|
- guides/inspection
|
||||||
|
---
|
||||||
|
|
||||||
|
# Upgrading from 7.x to 8.0
|
||||||
|
|
||||||
|
Opening a 7.x on-disk store with Brainy 8.0 runs a **one-time, in-place layout
|
||||||
|
migration** the first time the store is opened. It is automatic (`autoMigrate`
|
||||||
|
defaults to `true`), it runs once, and it stamps a marker so every later open is
|
||||||
|
a no-op.
|
||||||
|
|
||||||
|
Almost everything about the upgrade is transparent: your entities, relationships,
|
||||||
|
metadata, and indexes migrate and rebuild without any action on your part. This
|
||||||
|
guide covers the **one case that needs attention** — Virtual Filesystem (VFS)
|
||||||
|
content — and how 8.0 recovers it for you.
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
|
||||||
|
- **Just upgrade to `@soulcraft/brainy@8.0.12` (or later) and open the store.**
|
||||||
|
If a previous upgrade left VFS content stranded, 8.0.12 **heals it on open**,
|
||||||
|
with no operator action.
|
||||||
|
- Want to force or script it? Call **`await brain.vfs.adoptOrphanedBlobs()`**.
|
||||||
|
- The recovery is **non-destructive and idempotent** — it copies, never moves,
|
||||||
|
and running it twice is a no-op.
|
||||||
|
|
||||||
|
## What the migration does
|
||||||
|
|
||||||
|
The 7.x on-disk layout stored entities under a per-branch path
|
||||||
|
(`branches/<head>/entities/...`). 8.0 uses a flat layout (`entities/...`). On
|
||||||
|
first open, Brainy:
|
||||||
|
|
||||||
|
1. Collapses `branches/<head>/entities/*` into the flat `entities/*` layout.
|
||||||
|
2. Rebuilds the derived indexes (vector, graph, metadata) and count rollups from
|
||||||
|
the canonical entities.
|
||||||
|
3. Stamps `_system/migration-layout.json` so re-opening is a no-op.
|
||||||
|
4. Takes an automatic **pre-upgrade backup** of the directory before it starts,
|
||||||
|
and removes it once the upgrade is verified complete (see
|
||||||
|
[The safety net](#the-safety-net-pre-upgrade-backup) below).
|
||||||
|
|
||||||
|
The log line to expect (once per store):
|
||||||
|
|
||||||
|
```
|
||||||
|
[brainy] Migrating a 7.x branch layout (branches/main) to the 8.0 flat layout
|
||||||
|
in place — 13175 entity files. This runs once; back up the directory first if
|
||||||
|
you need a rollback (8.0 does not keep the old layout).
|
||||||
|
```
|
||||||
|
|
||||||
|
## The one thing that needs recovery: VFS content
|
||||||
|
|
||||||
|
If your application uses the Virtual Filesystem (VFS) to store file content (for
|
||||||
|
example, a CMS that keeps page documents at paths like
|
||||||
|
`/pages/homepage/page.json`), that content is held as **content blobs**.
|
||||||
|
|
||||||
|
7.x kept those blobs in the branch system's **copy-on-write area** (`_cow/`).
|
||||||
|
8.0 removed the branch/copy-on-write system, and its content blobs live in the
|
||||||
|
content-addressed store (`_cas/`). The layout migration moves entities — it does
|
||||||
|
**not** move the VFS content blobs. Left alone, a 7.x store's VFS blobs would
|
||||||
|
remain in `_cow/`, and a read of one would throw:
|
||||||
|
|
||||||
|
```
|
||||||
|
VFS: Cannot read blob for /pages/homepage/page.json:
|
||||||
|
Blob metadata not found: 6b87cfb71ad2f04602a5c157214dc42000...
|
||||||
|
```
|
||||||
|
|
||||||
|
### 8.0.12 recovers them automatically
|
||||||
|
|
||||||
|
Brainy 8.0.12 adds an **on-open recovery pass** that runs right after the layout
|
||||||
|
migration. It scans `_cow/`, and for every content blob whose `blob:` +
|
||||||
|
`blob-meta:` pair is not already in `_cas/`, it copies **both** across into the
|
||||||
|
8.0 store. It:
|
||||||
|
|
||||||
|
- **Heals a fresh 7→8 upgrade** (where `_cow/` still holds the blobs), **and**
|
||||||
|
- **Heals a store already upgraded by an earlier 8.0.x** that stranded them —
|
||||||
|
the recovery is gated on the presence of `_cow/` and its own marker, not on
|
||||||
|
the layout-migration marker, so an already-migrated store is still healed.
|
||||||
|
- Is a **cheap no-op** on a native-8.0 or fresh store (there is no `_cow/`), and
|
||||||
|
on a store already healed (a marker records completion so later opens skip the
|
||||||
|
scan).
|
||||||
|
|
||||||
|
So the operator action for a stranded store is simply: **upgrade to 8.0.12 and
|
||||||
|
open it.**
|
||||||
|
|
||||||
|
```ts
|
||||||
|
import { Brainy } from '@soulcraft/brainy'
|
||||||
|
|
||||||
|
// Opening the store is all that is required — recovery runs during init().
|
||||||
|
const brain = new Brainy({ storage: { type: 'filesystem', path: '/data/my-store' } })
|
||||||
|
await brain.init()
|
||||||
|
|
||||||
|
// The previously-failing read now succeeds.
|
||||||
|
const page = await brain.vfs.readFile('/pages/homepage/page.json')
|
||||||
|
```
|
||||||
|
|
||||||
|
On a store that needed recovery you will see:
|
||||||
|
|
||||||
|
```
|
||||||
|
[brainy] Recovered 337 VFS content blob(s) stranded by a 7→8 upgrade
|
||||||
|
(adopted _cow/ → _cas/ in place).
|
||||||
|
```
|
||||||
|
|
||||||
|
### Forcing recovery explicitly
|
||||||
|
|
||||||
|
If you would rather run the recovery deliberately (for example, in an upgrade
|
||||||
|
script that asserts a clean result before flipping traffic), call it directly:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const result = await brain.vfs.adoptOrphanedBlobs()
|
||||||
|
// → { cowBlobs, adopted, alreadyPresent, incomplete }
|
||||||
|
console.log(`adopted ${result.adopted}, already present ${result.alreadyPresent}`)
|
||||||
|
if (result.incomplete > 0) {
|
||||||
|
// One or more _cow/ blobs are missing their bytes or metadata — investigate
|
||||||
|
// _cow/ before discarding your own backup. See "The safety net" below.
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`adoptOrphanedBlobs()` self-initializes the brain, so it is safe to call
|
||||||
|
immediately after construction. It returns:
|
||||||
|
|
||||||
|
| Field | Meaning |
|
||||||
|
| ---------------- | ---------------------------------------------------------------- |
|
||||||
|
| `cowBlobs` | Distinct content-blob hashes found in `_cow/`. |
|
||||||
|
| `adopted` | Newly copied into `_cas/` on this call. |
|
||||||
|
| `alreadyPresent` | Already in `_cas/` (a prior open or run adopted them). |
|
||||||
|
| `incomplete` | A `_cow/` blob missing its bytes *or* its metadata — **skipped** rather than half-adopted. |
|
||||||
|
|
||||||
|
## Verifying recovery
|
||||||
|
|
||||||
|
After opening under 8.0.12, confirm a previously-failing path reads:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const content = await brain.vfs.readFile('/pages/homepage/page.json')
|
||||||
|
console.log(content.toString().slice(0, 80))
|
||||||
|
```
|
||||||
|
|
||||||
|
If you scripted it with `adoptOrphanedBlobs()`, a clean result is
|
||||||
|
`incomplete === 0` and `adopted + alreadyPresent === cowBlobs`.
|
||||||
|
|
||||||
|
## The safety net: pre-upgrade backup
|
||||||
|
|
||||||
|
Brainy takes an automatic pre-upgrade backup and removes it once the upgrade is
|
||||||
|
**verified complete**. In 8.0.12 that verification includes VFS content: if the
|
||||||
|
blob recovery reports `incomplete > 0`, the completion marker is **not** stamped
|
||||||
|
(the next open retries) and the **pre-upgrade backup is retained** so you still
|
||||||
|
have a rollback while blobs remain unaccounted for. You will see:
|
||||||
|
|
||||||
|
```
|
||||||
|
[brainy] VFS blob recovery adopted N blob(s) but M orphaned _cow/ blob(s) are
|
||||||
|
missing their bytes or metadata and were left in place. The pre-upgrade backup
|
||||||
|
is being retained; inspect _cow/ before discarding it.
|
||||||
|
```
|
||||||
|
|
||||||
|
The recovery never deletes anything from `_cow/`, so the original blobs stay in
|
||||||
|
place for inspection or a manual rollback regardless.
|
||||||
|
|
||||||
|
## Who is affected
|
||||||
|
|
||||||
|
**Any 7.x store that used the VFS to store file content** (so it has a `_cow/`
|
||||||
|
area) is a candidate for stranded blobs on a 7→8 upgrade. Stores that never used
|
||||||
|
the VFS have no `_cow/` content blobs and are unaffected.
|
||||||
|
|
||||||
|
Because the recovery is **gated on the presence of `_cow/`**, it is
|
||||||
|
self-selecting and safe to roll out everywhere:
|
||||||
|
|
||||||
|
- On a store with stranded blobs → it adopts them.
|
||||||
|
- On a native-8.0, fresh, or non-VFS store → it is a no-op on the existence
|
||||||
|
check.
|
||||||
|
|
||||||
|
There is no configuration to set and nothing to opt into. Upgrading to 8.0.12
|
||||||
|
and opening each store is sufficient.
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
|
||||||
|
The recovery is copy-only, so no rollback of the recovery itself is ever needed.
|
||||||
|
If you need to roll back the **whole** 7→8 upgrade, restore the directory from
|
||||||
|
your pre-upgrade backup (retained automatically while recovery is incomplete, or
|
||||||
|
your own snapshot) and pin `@soulcraft/brainy@7.x`. 8.0 does not keep the old
|
||||||
|
branch layout in place, so a directory-level restore is the rollback path.
|
||||||
119
src/brainy.ts
119
src/brainy.ts
|
|
@ -32,7 +32,7 @@ import {
|
||||||
getBrainyVersion
|
getBrainyVersion
|
||||||
} from './utils/index.js'
|
} from './utils/index.js'
|
||||||
import { embeddingManager } from './embeddings/EmbeddingManager.js'
|
import { embeddingManager } from './embeddings/EmbeddingManager.js'
|
||||||
import { matchesMetadataFilter } from './utils/metadataFilter.js'
|
import { matchesMetadataFilter, validateWhereFilter } from './utils/metadataFilter.js'
|
||||||
import { NaturalLanguageProcessor } from './neural/naturalLanguageProcessor.js'
|
import { NaturalLanguageProcessor } from './neural/naturalLanguageProcessor.js'
|
||||||
import { NeuralEntityExtractor, ExtractedEntity } from './neural/entityExtractor.js'
|
import { NeuralEntityExtractor, ExtractedEntity } from './neural/entityExtractor.js'
|
||||||
import { TripleIntelligenceSystem } from './triple/TripleIntelligenceSystem.js'
|
import { TripleIntelligenceSystem } from './triple/TripleIntelligenceSystem.js'
|
||||||
|
|
@ -453,6 +453,13 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
* `migrationBackup: false`), or `null` if none. Removed once the 7.x→8.0
|
* `migrationBackup: false`), or `null` if none. Removed once the 7.x→8.0
|
||||||
* upgrade verifies + stamps; retained on failure. See {@link createMigrationBackupIfNeeded}. */
|
* upgrade verifies + stamps; retained on failure. See {@link createMigrationBackupIfNeeded}. */
|
||||||
private _migrationBackupPath: string | null = null
|
private _migrationBackupPath: string | null = null
|
||||||
|
|
||||||
|
/** Set when the on-open VFS-blob adoption left one or more `_cow/` blobs it
|
||||||
|
* could not fully adopt (bytes or metadata missing). While true, the
|
||||||
|
* pre-upgrade backup is NOT auto-removed — the upgrade is not verifiably
|
||||||
|
* complete for the VFS. See {@link autoAdoptLegacyVfsBlobsIfNeeded}. */
|
||||||
|
private _vfsBlobAdoptionIncomplete = false
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 8.0 ⇄ native-provider version handshake — the on-disk {@link BrainFormat}
|
* 8.0 ⇄ native-provider version handshake — the on-disk {@link BrainFormat}
|
||||||
* marker (`_system/brain-format.json`) as read during the store-open phase,
|
* marker (`_system/brain-format.json`) as read during the store-open phase,
|
||||||
|
|
@ -1028,6 +1035,14 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Recover VFS content blobs a 7→8 upgrade left stranded in the removed
|
||||||
|
// branch system's copy-on-write area (`_cow/`). Runs BEFORE the rebuild
|
||||||
|
// (whose success stamp removes the pre-upgrade backup) so the backup still
|
||||||
|
// covers this step, and works on the raw object primitives (no blob store
|
||||||
|
// needed yet). A cheap no-op on native-8.0 / fresh brains and on a brain
|
||||||
|
// already healed. See adoptLegacyCowBlobs.
|
||||||
|
await this.autoAdoptLegacyVfsBlobsIfNeeded()
|
||||||
|
|
||||||
// Rebuild indexes if needed for existing data
|
// Rebuild indexes if needed for existing data
|
||||||
await this.rebuildIndexesIfNeeded()
|
await this.rebuildIndexesIfNeeded()
|
||||||
|
|
||||||
|
|
@ -5228,6 +5243,15 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
const hasFilterCriteria = params.where || params.type || params.subtype || params.service
|
const hasFilterCriteria = params.where || params.type || params.subtype || params.service
|
||||||
const hasGraphCriteria = params.connected
|
const hasGraphCriteria = params.connected
|
||||||
|
|
||||||
|
// Validate the raw where clause up front: an unrecognized operator (a typo
|
||||||
|
// like `notIn`, or any non-operator key on a field's object value) throws a
|
||||||
|
// typed BrainyError('INVALID_QUERY') instead of silently matching nothing.
|
||||||
|
// Done before the index/vector/graph paths so it fires even on an empty
|
||||||
|
// result set, and before brainy injects its own internal filter fields.
|
||||||
|
if (params.where) {
|
||||||
|
validateWhereFilter(params.where)
|
||||||
|
}
|
||||||
|
|
||||||
// Metadata cold-read guard: before trusting a filter result, verify the
|
// Metadata cold-read guard: before trusting a filter result, verify the
|
||||||
// field index actually serves a known persisted value (one-shot per brain).
|
// field index actually serves a known persisted value (one-shot per brain).
|
||||||
// A cold native index that has not loaded its `where` postings self-heals
|
// A cold native index that has not loaded its `where` postings self-heals
|
||||||
|
|
@ -13012,6 +13036,85 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description On-open recovery for VFS content blobs a 7→8 upgrade left in the
|
||||||
|
* removed branch system's copy-on-write area (`_cow/`). If a `_cow/` area is
|
||||||
|
* present and a completed adoption has not already been recorded, adopt every
|
||||||
|
* orphaned blob into the 8.0 content-addressed store (`_cas/`) and stamp a
|
||||||
|
* marker so later opens skip the scan. Because it runs on every open right
|
||||||
|
* after {@link legacyLayoutMigrationPhase}, it heals BOTH a fresh 7→8 migration
|
||||||
|
* (where `_cow/` still holds the blobs) AND a brain already migrated by an
|
||||||
|
* earlier 8.0.x that stranded them — no operator action needed.
|
||||||
|
*
|
||||||
|
* Filesystem-only; a native-8.0 / fresh brain has no `_cow/` and returns on the
|
||||||
|
* cheap existence check. Non-destructive and idempotent (see
|
||||||
|
* {@link BaseStorage.adoptLegacyCowBlobs}). If any blob cannot be fully adopted
|
||||||
|
* (`incomplete > 0`), the marker is NOT stamped (the next open retries) and
|
||||||
|
* {@link _vfsBlobAdoptionIncomplete} is set so the pre-upgrade backup is
|
||||||
|
* retained — the upgrade is not verifiably complete for the VFS.
|
||||||
|
*/
|
||||||
|
private async autoAdoptLegacyVfsBlobsIfNeeded(): Promise<void> {
|
||||||
|
if (this.getStorageType() !== 'filesystem') return
|
||||||
|
// Cheapest gate: no copy-on-write area → a native-8.0 / fresh brain, nothing
|
||||||
|
// a 7.x brain could have stranded here.
|
||||||
|
try {
|
||||||
|
const root = resolveFilesystemRoot(
|
||||||
|
(this.config.storage || {}) as StorageOptions & Record<string, unknown>
|
||||||
|
)
|
||||||
|
if (!fs.existsSync(`${root}/_cow`)) return
|
||||||
|
} catch {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
const storage = this.storage as unknown as {
|
||||||
|
adoptLegacyCowBlobs?: () => Promise<{
|
||||||
|
cowBlobs: number
|
||||||
|
adopted: number
|
||||||
|
alreadyPresent: number
|
||||||
|
incomplete: number
|
||||||
|
}>
|
||||||
|
readRawObject?: (p: string) => Promise<unknown>
|
||||||
|
writeRawObject?: (p: string, o: unknown) => Promise<void>
|
||||||
|
}
|
||||||
|
if (typeof storage.adoptLegacyCowBlobs !== 'function') return
|
||||||
|
|
||||||
|
const MARKER = '_system/vfs-blob-adoption.json'
|
||||||
|
try {
|
||||||
|
const done = (await storage.readRawObject?.(MARKER)) as { complete?: boolean } | null
|
||||||
|
if (done && done.complete) return // already fully adopted on a prior open
|
||||||
|
} catch {
|
||||||
|
// no marker yet — proceed
|
||||||
|
}
|
||||||
|
|
||||||
|
const result = await storage.adoptLegacyCowBlobs()
|
||||||
|
|
||||||
|
if (result.incomplete > 0) {
|
||||||
|
// Partial: retry on future opens, and hold the pre-upgrade backup.
|
||||||
|
this._vfsBlobAdoptionIncomplete = true
|
||||||
|
if (!this.config.silent) {
|
||||||
|
console.error(
|
||||||
|
`[brainy] VFS blob recovery adopted ${result.adopted} blob(s) but ${result.incomplete} ` +
|
||||||
|
`orphaned _cow/ blob(s) are missing their bytes or metadata and were left in place. ` +
|
||||||
|
`The pre-upgrade backup is being retained; inspect _cow/ before discarding it.`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Clean sweep — record it so later opens skip the _cow/ scan entirely.
|
||||||
|
try {
|
||||||
|
await storage.writeRawObject?.(MARKER, { complete: true, adopted: result.adopted })
|
||||||
|
} catch {
|
||||||
|
// marker write is best-effort; adoption already succeeded
|
||||||
|
}
|
||||||
|
if (result.adopted > 0 && !this.config.silent) {
|
||||||
|
console.warn(
|
||||||
|
`[brainy] Recovered ${result.adopted} VFS content blob(s) stranded by a 7→8 upgrade ` +
|
||||||
|
`(adopted _cow/ → _cas/ in place).`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private async setupStorage(): Promise<BaseStorage> {
|
private async setupStorage(): Promise<BaseStorage> {
|
||||||
// If the caller passed a pre-constructed storage adapter (e.g.
|
// If the caller passed a pre-constructed storage adapter (e.g.
|
||||||
// `storage: new MemoryStorage()`, or the historical materializer's
|
// `storage: new MemoryStorage()`, or the historical materializer's
|
||||||
|
|
@ -13846,6 +13949,20 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
||||||
*/
|
*/
|
||||||
private async removeMigrationBackupSafe(): Promise<void> {
|
private async removeMigrationBackupSafe(): Promise<void> {
|
||||||
if (!this._migrationBackupPath) return
|
if (!this._migrationBackupPath) return
|
||||||
|
// Backup-parity gate: the index rebuild "success" that calls this does NOT
|
||||||
|
// certify the VFS. If the on-open blob adoption could not fully bridge the
|
||||||
|
// 7.x `_cow/` blobs, the upgrade is not verifiably complete for VFS content
|
||||||
|
// — retain the backup so the operator can recover from it, and log why.
|
||||||
|
if (this._vfsBlobAdoptionIncomplete) {
|
||||||
|
if (!this.config.silent) {
|
||||||
|
console.warn(
|
||||||
|
`[brainy] Retaining the pre-upgrade backup at ${this._migrationBackupPath}: ` +
|
||||||
|
`some VFS content blobs could not be adopted from _cow/ (see the earlier warning). ` +
|
||||||
|
`Resolve those before discarding the backup.`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
const backupPath = this._migrationBackupPath
|
const backupPath = this._migrationBackupPath
|
||||||
this._migrationBackupPath = null
|
this._migrationBackupPath = null
|
||||||
const storage = this.storage as StorageAdapter & {
|
const storage = this.storage as StorageAdapter & {
|
||||||
|
|
|
||||||
|
|
@ -680,6 +680,91 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
||||||
this.blobStorage = new BlobStorage(casAdapter)
|
this.blobStorage = new BlobStorage(casAdapter)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Adopt orphaned 7.x copy-on-write VFS content blobs into the 8.0
|
||||||
|
* content-addressed store. 7.x kept VFS blobs (`blob:<hash>` bytes +
|
||||||
|
* `blob-meta:<hash>` JSON) under the copy-on-write area (`_cow/`) of the
|
||||||
|
* branch/versioning system that 8.0 removed. The 7→8 layout migration
|
||||||
|
* collapses entity files but historically did NOT bridge these blobs, so a VFS
|
||||||
|
* read of a still-in-`_cow/` blob throws "Blob metadata not found" and every
|
||||||
|
* page backed by it 500s.
|
||||||
|
*
|
||||||
|
* This scans `_cow/` and, for every blob whose `blob:`/`blob-meta:` pair is not
|
||||||
|
* already present in the 8.0 store (`_cas/`), copies BOTH objects across
|
||||||
|
* verbatim. Copying preserves the exact content bytes (so the content hash and
|
||||||
|
* every VFS reference still resolve) and the exact metadata (so the 7.x
|
||||||
|
* refCount is retained) — the only first-party, index-consistent recovery.
|
||||||
|
* Hand-moving files at the OS level bypasses the paired-metadata contract and
|
||||||
|
* is unsafe; this goes through the adapter's object primitives.
|
||||||
|
*
|
||||||
|
* Idempotent and non-destructive: a pair already in `_cas/` is left untouched
|
||||||
|
* (counted `alreadyPresent`); the `_cow/` originals are never deleted, so a
|
||||||
|
* re-run — or a rollback — is always possible. A no-op on memory storage and
|
||||||
|
* on any brain without a `_cow/` area (returns zeros). Bytes are written before
|
||||||
|
* metadata so an interruption can only leave the pre-adoption "no metadata"
|
||||||
|
* state (retryable), never a metadata-without-bytes blob.
|
||||||
|
*
|
||||||
|
* @returns `cowBlobs` (distinct blob hashes found in `_cow/`), `adopted`
|
||||||
|
* (newly copied into `_cas/`), `alreadyPresent` (already in `_cas/`),
|
||||||
|
* `incomplete` (a `_cow/` blob missing its bytes or its metadata — skipped
|
||||||
|
* and reported rather than half-adopted).
|
||||||
|
*/
|
||||||
|
public async adoptLegacyCowBlobs(): Promise<{
|
||||||
|
cowBlobs: number
|
||||||
|
adopted: number
|
||||||
|
alreadyPresent: number
|
||||||
|
incomplete: number
|
||||||
|
}> {
|
||||||
|
let cowPaths: string[]
|
||||||
|
try {
|
||||||
|
cowPaths = await this.listObjectsUnderPath('_cow/')
|
||||||
|
} catch {
|
||||||
|
// No `_cow/` area (fresh/native-8.0 brain, or a non-listing backend).
|
||||||
|
return { cowBlobs: 0, adopted: 0, alreadyPresent: 0, incomplete: 0 }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Distinct content-blob hashes from `_cow/blob:<hash>` keys. Non-blob `_cow/`
|
||||||
|
// entries (7.x branch COW state) are ignored — only VFS content is adopted.
|
||||||
|
const hashes = new Set<string>()
|
||||||
|
for (const p of cowPaths) {
|
||||||
|
const key = p.replace(/^_cow\//, '')
|
||||||
|
const m = /^blob:(.+)$/.exec(key)
|
||||||
|
if (m) hashes.add(m[1])
|
||||||
|
}
|
||||||
|
|
||||||
|
let adopted = 0
|
||||||
|
let alreadyPresent = 0
|
||||||
|
let incomplete = 0
|
||||||
|
for (const hash of hashes) {
|
||||||
|
// A blob counts as present only when BOTH its bytes and its metadata
|
||||||
|
// already live in `_cas/`. A half-adopted blob (bytes without meta — the
|
||||||
|
// exact "Blob metadata not found" state) is re-adopted.
|
||||||
|
const casBlob = await this.readObjectFromPath(`_cas/blob:${hash}`)
|
||||||
|
const casMeta = await this.readObjectFromPath(`_cas/blob-meta:${hash}`)
|
||||||
|
if (casBlob !== null && casMeta !== null) {
|
||||||
|
alreadyPresent++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
const cowBlob = await this.readObjectFromPath(`_cow/blob:${hash}`)
|
||||||
|
const cowMeta = await this.readObjectFromPath(`_cow/blob-meta:${hash}`)
|
||||||
|
if (cowBlob === null || cowMeta === null) {
|
||||||
|
// Can't register a blob the store can't fully describe — report it so an
|
||||||
|
// operator investigates rather than silently half-adopting.
|
||||||
|
incomplete++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bytes first, then metadata: a VFS read checks metadata before bytes, so
|
||||||
|
// metadata's presence must imply the bytes are already there.
|
||||||
|
await this.writeObjectToPath(`_cas/blob:${hash}`, cowBlob)
|
||||||
|
await this.writeObjectToPath(`_cas/blob-meta:${hash}`, cowMeta)
|
||||||
|
adopted++
|
||||||
|
}
|
||||||
|
|
||||||
|
return { cowBlobs: hashes.size, adopted, alreadyPresent, incomplete }
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @description Write a canonical object (write-cache coherent). The object
|
* @description Write a canonical object (write-cache coherent). The object
|
||||||
* lands in the write-through cache before the asynchronous write starts, so
|
* lands in the write-through cache before the asynchronous write starts, so
|
||||||
|
|
|
||||||
|
|
@ -352,6 +352,52 @@ export class VirtualFileSystem implements IVirtualFileSystem {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @description Recover VFS content blobs orphaned by a 7→8 upgrade. 7.x stored
|
||||||
|
* VFS blobs under the copy-on-write area (`_cow/`) of the branch/versioning
|
||||||
|
* system that 8.0 removed; a brain upgraded before the migration adopted them
|
||||||
|
* reads a stranded blob and throws "Blob metadata not found" (every page
|
||||||
|
* backed by it 500s). This adopts every orphaned `_cow/` blob into the 8.0
|
||||||
|
* content-addressed store (`_cas/`) IN PLACE — no snapshot restore — then
|
||||||
|
* clears the VFS caches so the recovered content reads live immediately.
|
||||||
|
*
|
||||||
|
* Idempotent and non-destructive: blobs already adopted are skipped and the
|
||||||
|
* `_cow/` originals are never deleted (a re-run or rollback stays possible).
|
||||||
|
* Safe to run on a healthy or native-8.0 brain (returns zeros). Delegates to
|
||||||
|
* {@link BaseStorage.adoptLegacyCowBlobs}.
|
||||||
|
*
|
||||||
|
* @returns `{ cowBlobs, adopted, alreadyPresent, incomplete }` counts.
|
||||||
|
* @example
|
||||||
|
* const brain = new Brainy({ storage: { type: 'filesystem', path: '/data/brain' } })
|
||||||
|
* await brain.init()
|
||||||
|
* const r = await brain.vfs.adoptOrphanedBlobs()
|
||||||
|
* console.log(`Adopted ${r.adopted} stranded VFS blobs; ${r.alreadyPresent} already present.`)
|
||||||
|
*/
|
||||||
|
async adoptOrphanedBlobs(): Promise<{
|
||||||
|
cowBlobs: number
|
||||||
|
adopted: number
|
||||||
|
alreadyPresent: number
|
||||||
|
incomplete: number
|
||||||
|
}> {
|
||||||
|
// The recovery scan works on raw storage objects, so it only needs the
|
||||||
|
// brain's storage wired up — not the VFS's own init. brain.init() is
|
||||||
|
// idempotent (a no-op on an already-open brain) and, on a cold brain,
|
||||||
|
// sets up storage AND runs the on-open auto-adoption itself; the explicit
|
||||||
|
// scan below is then the idempotent "force re-scan" equivalent.
|
||||||
|
await this.brain.init()
|
||||||
|
const storage = this.brain['storage'] as unknown as
|
||||||
|
| { adoptLegacyCowBlobs?: () => Promise<{ cowBlobs: number; adopted: number; alreadyPresent: number; incomplete: number }> }
|
||||||
|
| undefined
|
||||||
|
if (!storage || typeof storage.adoptLegacyCowBlobs !== 'function') {
|
||||||
|
return { cowBlobs: 0, adopted: 0, alreadyPresent: 0, incomplete: 0 }
|
||||||
|
}
|
||||||
|
const result = await storage.adoptLegacyCowBlobs()
|
||||||
|
// Drop any cached content/stat so a re-read hits the freshly adopted blobs.
|
||||||
|
this.contentCache.clear()
|
||||||
|
this.statCache.clear()
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
// ============= File Operations =============
|
// ============= File Operations =============
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|
|
||||||
149
tests/integration/vfs-cow-blob-adoption.test.ts
Normal file
149
tests/integration/vfs-cow-blob-adoption.test.ts
Normal file
|
|
@ -0,0 +1,149 @@
|
||||||
|
/**
|
||||||
|
* @module tests/integration/vfs-cow-blob-adoption
|
||||||
|
* @description P0 recovery: a 7→8 upgrade left VFS content blobs in the removed
|
||||||
|
* branch system's copy-on-write area (`_cow/`) instead of the 8.0
|
||||||
|
* content-addressed store (`_cas/`), so a VFS read of a stranded blob throws
|
||||||
|
* "Blob metadata not found" and every page backed by it 500s.
|
||||||
|
*
|
||||||
|
* Proves the cure end to end:
|
||||||
|
* - `storage.adoptLegacyCowBlobs()` copies each orphaned `_cow/` blob (bytes +
|
||||||
|
* metadata) into `_cas/` verbatim, idempotently, non-destructively, and
|
||||||
|
* reports an `incomplete` blob (bytes without metadata) rather than
|
||||||
|
* half-adopting it.
|
||||||
|
* - a real brain that upgraded into this state SELF-HEALS on its next open
|
||||||
|
* (auto-adoption) — the stranded VFS file reads correctly again with no
|
||||||
|
* operator action.
|
||||||
|
* - `brain.vfs.adoptOrphanedBlobs()` is the explicit equivalent.
|
||||||
|
*/
|
||||||
|
import { describe, it, expect, afterEach } from 'vitest'
|
||||||
|
import * as fs from 'node:fs'
|
||||||
|
import * as os from 'node:os'
|
||||||
|
import * as path from 'node:path'
|
||||||
|
import { Brainy } from '../../src/brainy.js'
|
||||||
|
import { FileSystemStorage } from '../../src/storage/adapters/fileSystemStorage.js'
|
||||||
|
|
||||||
|
const tmpDirs: string[] = []
|
||||||
|
function mkTmp(): string {
|
||||||
|
const d = fs.mkdtempSync(path.join(os.tmpdir(), 'brainy-cow-adopt-'))
|
||||||
|
tmpDirs.push(d)
|
||||||
|
return d
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Release a raw FileSystemStorage: flush, stop its timer, drop the writer lock
|
||||||
|
* so a subsequent open of the same directory is not blocked. */
|
||||||
|
async function teardownRawStorage(s: any): Promise<void> {
|
||||||
|
try { await s.flush?.() } catch { /* best effort */ }
|
||||||
|
try { s.stopFlushRequestWatcher?.() } catch { /* best effort */ }
|
||||||
|
try { await s.releaseWriterLock?.() } catch { /* best effort */ }
|
||||||
|
}
|
||||||
|
afterEach(() => {
|
||||||
|
for (const d of tmpDirs.splice(0)) fs.rmSync(d, { recursive: true, force: true })
|
||||||
|
})
|
||||||
|
|
||||||
|
/** Move every `_cas/blob*` object into `_cow/` and delete the `_cas/` copy —
|
||||||
|
* reproducing the exact on-disk state a 7.x brain leaves after the 8.0 layout
|
||||||
|
* migration collapses entities but does not adopt the copy-on-write blobs. */
|
||||||
|
async function strandCasBlobsIntoCow(dir: string): Promise<number> {
|
||||||
|
const s: any = new FileSystemStorage(dir)
|
||||||
|
await s.init()
|
||||||
|
const casKeys = (await s.listRawObjects('_cas/')).filter((k: string) =>
|
||||||
|
/(^|\/)(blob:|blob-meta:)/.test(k)
|
||||||
|
)
|
||||||
|
let moved = 0
|
||||||
|
for (const full of casKeys) {
|
||||||
|
const key = full.replace(/^_cas\//, '')
|
||||||
|
const obj = await s.readRawObject(`_cas/${key}`)
|
||||||
|
if (obj === null) continue
|
||||||
|
await s.writeRawObject(`_cow/${key}`, obj)
|
||||||
|
await s.deleteRawObject(`_cas/${key}`)
|
||||||
|
moved++
|
||||||
|
}
|
||||||
|
await teardownRawStorage(s)
|
||||||
|
return moved
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('VFS _cow/ blob adoption (P0 7→8 recovery)', () => {
|
||||||
|
it('adoptLegacyCowBlobs copies bytes + metadata into _cas/, verbatim, idempotent, non-destructive', async () => {
|
||||||
|
const dir = mkTmp()
|
||||||
|
const storage: any = new FileSystemStorage(dir)
|
||||||
|
await storage.init()
|
||||||
|
|
||||||
|
const hash = 'a'.repeat(64)
|
||||||
|
const bytes = Buffer.from('the homepage payload')
|
||||||
|
// 7.x layout: the pair lives ONLY under _cow/ (blob-meta carries the refCount).
|
||||||
|
await storage.writeRawObject(`_cow/blob:${hash}`, { _binary: true, data: bytes.toString('base64') })
|
||||||
|
await storage.writeRawObject(`_cow/blob-meta:${hash}`, { hash, size: bytes.length, refCount: 3, mimeType: 'application/json' })
|
||||||
|
|
||||||
|
// Pre-adoption: _cas/ has neither → this is the "Blob metadata not found" state.
|
||||||
|
expect(await storage.readRawObject(`_cas/blob-meta:${hash}`)).toBeNull()
|
||||||
|
|
||||||
|
const r = await storage.adoptLegacyCowBlobs()
|
||||||
|
expect(r).toMatchObject({ cowBlobs: 1, adopted: 1, alreadyPresent: 0, incomplete: 0 })
|
||||||
|
|
||||||
|
// Post: _cas/ has both, metadata verbatim (refCount preserved), bytes intact.
|
||||||
|
const casMeta = await storage.readRawObject(`_cas/blob-meta:${hash}`)
|
||||||
|
expect(casMeta).toMatchObject({ hash, refCount: 3, size: bytes.length })
|
||||||
|
const casBlob = await storage.readRawObject(`_cas/blob:${hash}`)
|
||||||
|
expect(Buffer.from(casBlob.data, 'base64').toString()).toBe('the homepage payload')
|
||||||
|
|
||||||
|
// Idempotent: a second run adopts nothing, counts the pair present.
|
||||||
|
expect(await storage.adoptLegacyCowBlobs()).toMatchObject({ adopted: 0, alreadyPresent: 1 })
|
||||||
|
// Non-destructive: the _cow/ original is still there (rollback stays possible).
|
||||||
|
expect(await storage.readRawObject(`_cow/blob:${hash}`)).not.toBeNull()
|
||||||
|
|
||||||
|
// Incomplete: bytes without metadata are reported, not half-adopted.
|
||||||
|
await storage.writeRawObject(`_cow/blob:${'b'.repeat(64)}`, { _binary: true, data: 'x' })
|
||||||
|
const r3 = await storage.adoptLegacyCowBlobs()
|
||||||
|
expect(r3.incomplete).toBe(1)
|
||||||
|
expect(await storage.readRawObject(`_cas/blob:${'b'.repeat(64)}`)).toBeNull() // not adopted
|
||||||
|
await teardownRawStorage(storage)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('a brain that upgraded into the stranded state self-heals on next open — the VFS page reads again', async () => {
|
||||||
|
const dir = mkTmp()
|
||||||
|
|
||||||
|
// 1. Author a VFS page (its content blob lands in _cas/).
|
||||||
|
let brain: any = new Brainy({ storage: { type: 'filesystem', path: dir }, plugins: [], silent: true })
|
||||||
|
await brain.init()
|
||||||
|
await brain.vfs.writeFile('/pages/homepage/page.json', '{"title":"Homepage"}')
|
||||||
|
expect((await brain.vfs.readFile('/pages/homepage/page.json')).toString()).toContain('Homepage')
|
||||||
|
await brain.close()
|
||||||
|
|
||||||
|
// 2. Reproduce the post-7→8 stranding: the blob is now only under _cow/.
|
||||||
|
const moved = await strandCasBlobsIntoCow(dir)
|
||||||
|
expect(moved).toBeGreaterThan(0)
|
||||||
|
|
||||||
|
// 3. Reopen: the on-open auto-adoption heals it with no operator action.
|
||||||
|
brain = new Brainy({ storage: { type: 'filesystem', path: dir }, plugins: [], silent: true })
|
||||||
|
await brain.init()
|
||||||
|
const healed = await brain.vfs.readFile('/pages/homepage/page.json')
|
||||||
|
expect(healed.toString()).toContain('Homepage')
|
||||||
|
|
||||||
|
// A subsequent open records the adoption marker and does not re-scan-heal.
|
||||||
|
await brain.close()
|
||||||
|
brain = new Brainy({ storage: { type: 'filesystem', path: dir }, plugins: [], silent: true })
|
||||||
|
await brain.init()
|
||||||
|
expect((await brain.vfs.readFile('/pages/homepage/page.json')).toString()).toContain('Homepage')
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('brain.vfs.adoptOrphanedBlobs() is the explicit recovery equivalent', async () => {
|
||||||
|
const dir = mkTmp()
|
||||||
|
const body = JSON.stringify({ page: 'about', body: 'x'.repeat(400) })
|
||||||
|
let brain: any = new Brainy({ storage: { type: 'filesystem', path: dir }, plugins: [], silent: true })
|
||||||
|
await brain.init()
|
||||||
|
await brain.vfs.writeFile('/about.json', body)
|
||||||
|
await brain.close()
|
||||||
|
await strandCasBlobsIntoCow(dir)
|
||||||
|
|
||||||
|
// The explicit API self-initializes (like other VFS methods); init's
|
||||||
|
// auto-heal adopts, so the explicit call sees the pair present. Either way
|
||||||
|
// it reports the _cow/ blobs it accounted for, and the read is healed.
|
||||||
|
brain = new Brainy({ storage: { type: 'filesystem', path: dir }, plugins: [], silent: true })
|
||||||
|
const r = await brain.vfs.adoptOrphanedBlobs()
|
||||||
|
expect(r.cowBlobs).toBeGreaterThan(0)
|
||||||
|
expect(r.adopted + r.alreadyPresent).toBe(r.cowBlobs)
|
||||||
|
expect((await brain.vfs.readFile('/about.json')).toString()).toContain('about')
|
||||||
|
await brain.close()
|
||||||
|
})
|
||||||
|
})
|
||||||
Loading…
Add table
Add a link
Reference in a new issue