A production incident: a native-provider op ground 38-40s inside a transaction,
blew the apply-phase budget, rolled back, and a downstream pipeline hot-retried
the identical operation into a 6-minute CPU storm. Brainy itself never
auto-retried the timeout; the gap was that TransactionTimeoutError only said
"retryable" in prose, with nothing machine-readable for a caller to branch on.
- TransactionTimeoutError gains two typed, always-true fields: retryable
(a later attempt may succeed once the slowness resolves or the budget is
raised) and hotRetryUnsafe (an immediate identical retry re-pays the full
cost that just timed out and can cascade into a CPU storm -- callers must
latch and back off, never loop). context's existing telemetry fields
(timeoutMs, operationIndex, elapsedMs, totalOperations, operationName) are
now documented as the caller's backoff inputs.
- Updated the "retryable" doc-prose sites (transact()'s timeoutMs option,
transactionBudgetFloorMs, Transaction.execute()'s contract) to point at
the new fields instead of bare prose.
- Regression pin (tests/unit/transaction/timeout-never-internally-retried.test.ts):
an execution counter proves the engine never re-drives a timed-out
operation, through both the single-op engine TransactionManager/Transaction
drives for every single-record write, and add()'s upsert-race retry loop
(which must exit on the first TransactionTimeoutError, never treat it like
the lost-insert-race signal it retries on).
- Removed TransactionManager.executeTransactionWithResult -- zero callers
anywhere in the codebase.
David Snelling
003e2a74ea