feat(8.0): #18 coordinated migration LOCK — block-and-queue the 7.x→8.0 auto-upgrade
Replaces rc.8's no-freeze deference with an automatic, observable, coordinated
migration LOCK (David's reversal: "unknown/halfway states are more dangerous
than blocking"). While a native provider runs its one-time 7.x→8.0
rebuild-from-canonical (`isMigrating()===true`), brainy blocks/queues data-plane
reads AND writes so no operation ever touches a half-built index — closing the
three seam-map gaps (lost mid-rebuild writes, degraded reads, read-time rebuild).
Mechanism (one choke point):
- awaitMigrationLock() at ensureInitialized() covers all ~40 data-plane methods;
a non-migrating brain pays one boolean check. Polls isMigrating() at 250ms;
after migrationWaitTimeoutMs (default 30s) throws a retryable, exported
MigrationInProgressError. The timeout bounds the CALLER'S WAIT, not the
migration — the rebuild is unbounded and never interrupted.
- init() awaits the lock before VFS bootstrap + before serving ("not ready until
upgraded"); a rebuild past the budget surfaces MigrationInProgressError at init
(raise the budget, or run the offline migrator).
Observability (readiness-probe correct — never gated):
- getIndexStatus() gains `migrating` + `migration` (MigrationProgress); a probe
maps migrating→HTTP 503+Retry-After, not 500. health()/checkHealth() lock-exempt
(health() reports a `warn` migration check). stampBrainFormat()/close() are
ungated so cor can clear the lock — no deadlock.
- Optional provider migrationStatus() is relayed verbatim for a live %.
verifyGraphAdjacencyLive() honors the lock (no self-rebuild mid-migration); the
stamp is still withheld while any provider migrates (cor stamps after verify).
Adversarially verified: fixed a critical init/VFS-bootstrap deadlock, a mixed-
provider busy-spin (dropped the event-driven signal → pure poll), a not-yet-
initialized getIndexStatus crash, and once-per-window log/clock resets. 10 lock
tests (incl. the init-during-migration regression). Gates: typecheck 0, build 0,
test:unit 1753/1753.
This commit is contained in:
parent
ca9129a924
commit
67bbf69a5c
5 changed files with 534 additions and 31 deletions
|
|
@ -1575,6 +1575,31 @@ export interface BrainyStats {
|
|||
/**
|
||||
* Brainy configuration
|
||||
*/
|
||||
/**
|
||||
* Structured progress of the one-time, automatic 7.x → 8.0 migration (the
|
||||
* coordinated LOCK). Relayed verbatim from the native provider's optional
|
||||
* `migrationStatus()` and surfaced on `getIndexStatus().migration` while the
|
||||
* brain is upgrading. Every field is optional — a provider may report only a
|
||||
* phase, or nothing (in which case `getIndexStatus()` shows `migrating: true`
|
||||
* plus `elapsedMs` alone).
|
||||
*/
|
||||
export interface MigrationProgress {
|
||||
/** Coarse stage, e.g. `'rebuilding'` | `'verifying'`. */
|
||||
phase?: string
|
||||
/** Which derived index is currently rebuilding. */
|
||||
index?: 'metadata' | 'vector' | 'graph'
|
||||
/** Overall progress, 0–100. */
|
||||
percent?: number
|
||||
/** Canonical entities processed so far. */
|
||||
entitiesDone?: number
|
||||
/** Total canonical entities to process. */
|
||||
entitiesTotal?: number
|
||||
/** When the migration was first observed (epoch ms). */
|
||||
startedAt?: number
|
||||
/** Milliseconds elapsed since the migration was first observed. */
|
||||
elapsedMs?: number
|
||||
}
|
||||
|
||||
export interface BrainyConfig {
|
||||
/**
|
||||
* Storage configuration: either a config object resolved through the
|
||||
|
|
@ -1626,6 +1651,28 @@ export interface BrainyConfig {
|
|||
*/
|
||||
disableAutoRebuild?: boolean
|
||||
|
||||
/**
|
||||
* How long (ms) an operation **waits** on the coordinated 7.x → 8.0 migration
|
||||
* before throwing a retryable `MigrationInProgressError`.
|
||||
*
|
||||
* IMPORTANT: this bounds the *caller's wait*, NOT the migration. The migration
|
||||
* itself is **unbounded** — a native provider rebuilds a billion-scale brain
|
||||
* for as long as it needs, and this timeout never interrupts it. While the
|
||||
* rebuild runs, reads and writes (and `init()` before it serves) block so no
|
||||
* operation touches a half-built index:
|
||||
* - **Small/medium upgrade (< this window):** the caller waits, then resumes
|
||||
* transparently — no error.
|
||||
* - **Large upgrade (> this window):** the caller gets a retryable
|
||||
* `MigrationInProgressError` (the rebuild continues in the background);
|
||||
* `getIndexStatus().migrating` — never gated — is the readiness-probe signal
|
||||
* that maps to HTTP 503 + Retry-After so an orchestrator waits rather than
|
||||
* routing traffic in.
|
||||
*
|
||||
* Raise it for a latency-tolerant batch job (wait longer / effectively wait
|
||||
* through); lower it to fail fast on a request path. Default: `30000` (30 s).
|
||||
*/
|
||||
migrationWaitTimeoutMs?: number
|
||||
|
||||
/**
|
||||
* Vector index configuration (Brainy 8.0).
|
||||
*
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue