feat(8.0): auto pre-upgrade backup — hard-link snapshot before the 7.x→8.0 migration

David's ask: back up the brain before the one-time upgrade, drop it after. On
open, when the on-disk format is stale (a 7.x→8.0 rebuild will run) and the store
holds data, brainy hard-link-snapshots the brain dir to a sibling
`<brainDir>.migration-backup` BEFORE any provider rebuilds — near-zero cost/space
(shared inodes; the store is immutable-by-rename), instant even at scale. Removed
automatically once the upgrade verifies + stamps the marker; retained on failure
for rollback. Default-on; opt out with `migrationBackup: false`.

- Reuses the existing snapshotToDirectory() hard-link farm via new optional
  adapter methods createMigrationBackup()/removeMigrationBackup() (filesystem
  only — feature-detected; memory + non-fs are a graceful no-op).
- Taken before the native provider inits, so it captures true pre-migration
  state; a prior failed upgrade's backup is reused, not overwritten.
- Best-effort: a backup failure never blocks the upgrade (the migration is itself
  safe — reads canonical, reconstructable, self-heals). Catastrophe-insurance
  against a migration bug, not a data-loss guard or an off-device backup.

4 lifecycle tests (adapter hard-link / reuse / remove-without-touching-live /
empty→null; stale-epoch reopen create+remove; opt-out; memory no-op). Gates:
typecheck 0, build 0, test:unit 1753/1753, layout-migration suite 5/5.
This commit is contained in:
David Snelling 2026-07-01 15:04:01 -07:00
parent ed178e2ce9
commit 1aad1f67de
5 changed files with 319 additions and 0 deletions

View file

@ -432,6 +432,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
* not migrating the fallback `elapsedMs` a provider that omits `migrationStatus()`
* timing still gets in `getIndexStatus().migration`. See {@link migrationSnapshot}. */
private _migrationObservedAt: number | null = null
/** Path of the pre-upgrade backup taken this open (default-on, opt-out via
* `migrationBackup: false`), or `null` if none. Removed once the 7.x8.0
* upgrade verifies + stamps; retained on failure. See {@link createMigrationBackupIfNeeded}. */
private _migrationBackupPath: string | null = null
/**
* 8.0 native-provider version handshake the on-disk {@link BrainFormat}
* marker (`_system/brain-format.json`) as read during the store-open phase,
@ -856,6 +860,16 @@ export class Brainy<T = any> implements BrainyInterface<T> {
this._indexEpochStale =
this._brainFormat === null || this._brainFormat.indexEpoch !== EXPECTED_INDEX_EPOCH
// Pre-upgrade backup (default-on, opt-out via `migrationBackup: false`): the
// on-disk format is stale, so a one-time 7.x → 8.0 rebuild will run below.
// Snapshot the brain dir NOW — before any provider (native or JS) rebuilds a
// derived index — so a migration bug can be rolled back. Removed once the
// upgrade verifies + stamps; retained on failure. No-op for a reader, for
// non-filesystem storage, or for a brain with no persisted data.
if (this._indexEpochStale && this.config.migrationBackup && !this.isReadOnly) {
await this.createMigrationBackupIfNeeded()
}
// Provider: embeddings (reassign embedder if plugin provides one)
const embeddingProvider = this.pluginRegistry.getProvider<EmbeddingFunction>('embeddings')
if (embeddingProvider) {
@ -6489,6 +6503,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
await writeBrainFormat(this.storage)
this._brainFormat = { dataFormat: CURRENT_DATA_FORMAT, indexEpoch: EXPECTED_INDEX_EPOCH }
this._indexEpochStale = false
// The upgrade has verified + stamped — drop the pre-upgrade backup (no-op if
// none was taken, e.g. a non-migrating stamp of a fresh brain's initial marker).
await this.removeMigrationBackupSafe()
}
/**
@ -12991,6 +13008,8 @@ export class Brainy<T = any> implements BrainyInterface<T> {
// Migration LOCK wait budget — left undefined when omitted so
// awaitMigrationLock() applies its 30 s default at the read site.
migrationWaitTimeoutMs: config?.migrationWaitTimeoutMs ?? undefined,
// Pre-upgrade backup — default-on; opt out with `migrationBackup: false`.
migrationBackup: config?.migrationBackup ?? true,
// Vector index configuration (8.0) — algorithm-neutral surface with
// `recall` preset + `persistMode` (folded in from 7.x's hnswPersistMode).
// See BRAINY-8.0-RENAME-COORDINATION § A.2 + § G.2.
@ -13400,6 +13419,8 @@ export class Brainy<T = any> implements BrainyInterface<T> {
await writeBrainFormat(this.storage)
this._brainFormat = { dataFormat: CURRENT_DATA_FORMAT, indexEpoch: EXPECTED_INDEX_EPOCH }
this._indexEpochStale = false
// The JS inline rebuild verified + stamped — drop the pre-upgrade backup.
await this.removeMigrationBackupSafe()
}
/**
@ -13560,6 +13581,61 @@ export class Brainy<T = any> implements BrainyInterface<T> {
}
}
/**
* @description Take the pre-upgrade backup (default-on) before a native provider
* rebuilds anything for a 7.x 8.0 migration. Called at open, once the on-disk
* format is known stale, BEFORE the providers init. Feature-detected + best-effort:
* a backend without `createMigrationBackup` (memory) or a store with no data is a
* no-op, and a backup failure NEVER blocks the upgrade (the migration is itself
* safe it only reads canonical, and self-heals on re-open). The snapshot is
* removed once the upgrade verifies ({@link removeMigrationBackupSafe}) and
* retained on failure for rollback.
*/
private async createMigrationBackupIfNeeded(): Promise<void> {
const storage = this.storage as StorageAdapter & {
createMigrationBackup?: () => Promise<string | null>
}
if (typeof storage.createMigrationBackup !== 'function') return
try {
const backupPath = await storage.createMigrationBackup()
if (backupPath) {
this._migrationBackupPath = backupPath
prodLog.info(
`[Brainy] Pre-upgrade backup created at ${backupPath} (hard-link snapshot; ~zero ` +
`cost/space). Removed automatically once the 7.x→8.0 upgrade verifies; retained ` +
`for rollback if it fails. Opt out with { migrationBackup: false }.`
)
}
} catch (error) {
prodLog.warn(
`[Brainy] Pre-upgrade backup could not be created (continuing — the upgrade is safe ` +
`and self-heals): ${error instanceof Error ? error.message : String(error)}`
)
}
}
/**
* @description Remove the pre-upgrade backup once the migration has verified and
* stamped the format marker. Called from both stamp paths (the JS inline rebuild
* and the native provider's `stampBrainFormat()`), gated on a backup having been
* taken this open. Best-effort: a removal failure leaves a harmless backup dir.
*/
private async removeMigrationBackupSafe(): Promise<void> {
if (!this._migrationBackupPath) return
const backupPath = this._migrationBackupPath
this._migrationBackupPath = null
const storage = this.storage as StorageAdapter & {
removeMigrationBackup?: (location: string) => Promise<void>
}
if (typeof storage.removeMigrationBackup !== 'function') return
try {
await storage.removeMigrationBackup(backupPath)
prodLog.info(`[Brainy] Upgrade verified — pre-upgrade backup at ${backupPath} removed.`)
} catch {
// best-effort — a leftover backup dir is harmless
}
}
/**
* Check health of metadata indexes
*