embedBatch() with large inputs (e.g. 500 chunks from highlight()) runs a single synchronous WASM forward pass that blocks the event loop for 200-500ms. Split large batches into micro-batches of 20 with setTimeout(0) yields between each, keeping max blocking per batch to ~10-30ms. Also change Cargo.toml opt-level from "z" (size) to 3 (speed) for ~15-20% faster WASM inference. Requires WASM rebuild to take effect. |
||
|---|---|---|
| .. | ||
| .cargo | ||
| src | ||
| Cargo.toml | ||