fix: transaction timeouts are a typed no-hot-retry contract; engine-side non-retry pinned; dead transaction path removed
A production incident: a native-provider op ground 38-40s inside a transaction, blew the apply-phase budget, rolled back, and a downstream pipeline hot-retried the identical operation into a 6-minute CPU storm. Brainy itself never auto-retried the timeout; the gap was that TransactionTimeoutError only said "retryable" in prose, with nothing machine-readable for a caller to branch on. - TransactionTimeoutError gains two typed, always-true fields: retryable (a later attempt may succeed once the slowness resolves or the budget is raised) and hotRetryUnsafe (an immediate identical retry re-pays the full cost that just timed out and can cascade into a CPU storm -- callers must latch and back off, never loop). context's existing telemetry fields (timeoutMs, operationIndex, elapsedMs, totalOperations, operationName) are now documented as the caller's backoff inputs. - Updated the "retryable" doc-prose sites (transact()'s timeoutMs option, transactionBudgetFloorMs, Transaction.execute()'s contract) to point at the new fields instead of bare prose. - Regression pin (tests/unit/transaction/timeout-never-internally-retried.test.ts): an execution counter proves the engine never re-drives a timed-out operation, through both the single-op engine TransactionManager/Transaction drives for every single-record write, and add()'s upsert-race retry loop (which must exit on the first TransactionTimeoutError, never treat it like the lost-insert-race signal it retries on). - Removed TransactionManager.executeTransactionWithResult -- zero callers anywhere in the codebase.
This commit is contained in:
parent
22702b81c0
commit
003e2a74ea
7 changed files with 199 additions and 76 deletions
|
|
@ -121,9 +121,13 @@ export interface TransactOptions {
|
|||
* with the batch: `max(30 000, opCount × 2 000)` — production imports on
|
||||
* network-attached disks measure ~2 s per operation, so a flat 30 s budget
|
||||
* silently capped honest bulk work at ~15 operations. A tripped budget
|
||||
* rolls the whole batch back and throws a retryable
|
||||
* `TransactionTimeoutError` naming the operation it stopped at, the batch
|
||||
* size, and the elapsed/budget times.
|
||||
* rolls the whole batch back and throws a `TransactionTimeoutError` naming
|
||||
* the operation it stopped at, the batch size, and the elapsed/budget
|
||||
* times. That error is retryable-with-latch, never hot-retry: its
|
||||
* `retryable` field says a later attempt may succeed, its
|
||||
* `hotRetryUnsafe` field says an immediate identical retry re-pays the
|
||||
* full cost that just timed out — callers must latch and back off, never
|
||||
* loop.
|
||||
*/
|
||||
timeoutMs?: number
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue