fix: prevent WASM embedding from blocking event loop during highlight()

embedBatch() with large inputs (e.g. 500 chunks from highlight()) runs
a single synchronous WASM forward pass that blocks the event loop for
200-500ms. Split large batches into micro-batches of 20 with setTimeout(0)
yields between each, keeping max blocking per batch to ~10-30ms.

Also change Cargo.toml opt-level from "z" (size) to 3 (speed) for
~15-20% faster WASM inference. Requires WASM rebuild to take effect.
This commit is contained in:
David Snelling 2026-01-27 16:49:26 -08:00
parent 1ffdfa70ae
commit bf71317d21
2 changed files with 30 additions and 4 deletions

View file

@ -45,7 +45,7 @@ getrandom = { version = "0.3", features = ["wasm_js"] }
wasm-bindgen-test = "0.3"
[profile.release]
opt-level = "z" # Optimize for size
opt-level = 3 # Optimize for speed (~15-20% faster inference, ~1MB larger binary)
lto = true # Link-time optimization
codegen-units = 1 # Single codegen unit for better optimization
panic = "abort" # Abort on panic (smaller binary)