feat: scaled transact budgets + labeled timeout diagnostics + envelope docs
- The transact apply budget scales with the batch — max(30s, opCount x 2s) — or is exactly the caller's new TransactOptions.timeoutMs. A flat 30s cap silently limited honest bulk work to ~15 operations on network-attached storage (~2s/op measured in a production import) while looking generous for small batches. Internal batch paths (removeMany chunks) get the same scaling via the shared transactTimeoutBudget helper. - TransactionTimeoutError now reports the operation it stopped at as i/N with the operation's name, elapsed vs budgeted time, and states the batch rolled back atomically and is retryable; context carries the fields programmatically. - The transact operational envelope is documented (optimistic-concurrency guide): budget math, chunking with ifAbsent idempotency, when to reach for addMany vs transact, and the precompute pattern (embedBatch + per-op vector) that keeps inference out of the commit path. - Embedding claims made honest per an end-to-end probe of the built dist: batch and single embedding paths produce bit-identical vectors and a vector-supplied add is fully searchable, but batch throughput on the default WASM engine measures comparable to sequential (~160ms/text) — the 5-10x speedup claim in the addMany JSDoc is replaced with the measured reality and the actual win (inference outside the budgeted write path).
This commit is contained in:
parent
e0f6e7722f
commit
6ef9fcb7a2
7 changed files with 175 additions and 15 deletions
|
|
@ -37,6 +37,24 @@ const DEFAULT_OPTIONS: Required<TransactionOptions> = {
|
|||
maxRollbackRetries: 3
|
||||
}
|
||||
|
||||
/**
|
||||
* The apply-phase budget for a batch of `opCount` operations.
|
||||
*
|
||||
* An explicit override wins untouched. Otherwise the budget SCALES with the
|
||||
* batch: `max(30 000 ms, opCount × 2 000 ms)`. The per-op term is calibrated
|
||||
* from field data — bulk imports on network-attached disks measure ~2 s per
|
||||
* operation (each op pays canonical writes + fsync + index maintenance) — so
|
||||
* a flat 30 s budget silently capped honest work at ~15 operations while
|
||||
* looking generous for small batches. Scaling keeps small transacts
|
||||
* fast-failing and gives bulk ones a budget proportional to the work they
|
||||
* actually asked for; a trip still rolls back atomically and throws a
|
||||
* retryable, fully-labeled TransactionTimeoutError.
|
||||
*/
|
||||
export function transactTimeoutBudget(opCount: number, override?: number): number {
|
||||
if (override !== undefined) return override
|
||||
return Math.max(30_000, opCount * 2_000)
|
||||
}
|
||||
|
||||
/**
|
||||
* Transaction class
|
||||
*/
|
||||
|
|
@ -114,7 +132,11 @@ export class Transaction implements TransactionContext {
|
|||
// into the catch below and rolls back like any other failure — it must
|
||||
// never bypass rollback.
|
||||
if (Date.now() - this.startTime > this.options.timeout) {
|
||||
throw new TransactionTimeoutError(this.options.timeout, i)
|
||||
throw new TransactionTimeoutError(this.options.timeout, i, {
|
||||
elapsedMs: Date.now() - this.startTime,
|
||||
totalOperations: this.operations.length,
|
||||
operationName: this.operations[i]?.name
|
||||
})
|
||||
}
|
||||
|
||||
const operation = this.operations[i]
|
||||
|
|
|
|||
|
|
@ -77,11 +77,24 @@ export class InvalidTransactionStateError extends TransactionError {
|
|||
export class TransactionTimeoutError extends TransactionError {
|
||||
constructor(
|
||||
timeoutMs: number,
|
||||
operationIndex: number
|
||||
operationIndex: number,
|
||||
telemetry?: {
|
||||
elapsedMs?: number
|
||||
totalOperations?: number
|
||||
operationName?: string
|
||||
}
|
||||
) {
|
||||
const progress =
|
||||
telemetry?.totalOperations !== undefined
|
||||
? `${operationIndex}/${telemetry.totalOperations}`
|
||||
: String(operationIndex)
|
||||
const name = telemetry?.operationName ? ` ('${telemetry.operationName}')` : ''
|
||||
const elapsed =
|
||||
telemetry?.elapsedMs !== undefined ? `${telemetry.elapsedMs}ms elapsed, ` : ''
|
||||
super(
|
||||
`Transaction timed out after ${timeoutMs}ms at operation ${operationIndex}`,
|
||||
{ timeoutMs, operationIndex }
|
||||
`Transaction timed out at operation ${progress}${name} — ${elapsed}budget ${timeoutMs}ms. ` +
|
||||
`The batch rolled back atomically; retry with a higher timeoutMs or a smaller batch.`,
|
||||
{ timeoutMs, operationIndex, ...telemetry }
|
||||
)
|
||||
this.name = 'TransactionTimeoutError'
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue