refactor(worker-threads): pass functions as strings to resolve Promise cloning issues

- Updated worker execution logic to pass functions as stringified versions instead of reference types to avoid cloning issues in worker threads.
- Refactored utility functions in `workerUtils` to accept serialized function strings.
- Adjusted embedding and augmentation pipeline workflows to align with the updated worker handling mechanism.
- Enhanced parallel distance calculator to support structured arguments with stringified distance functions for better compatibility.
This commit is contained in:
David Snelling 2025-06-24 10:38:05 -07:00
parent 887a08b689
commit d18a5a6c22
5 changed files with 92 additions and 73 deletions

View file

@ -418,7 +418,7 @@ export class Pipeline implements IPipeline {
return augFn.apply(augmentation, workerArgs)
}
methodPromise = executeInThread<AugmentationResponse<T>>(workerFn, ...args)
methodPromise = executeInThread<AugmentationResponse<T>>(workerFn.toString(), args)
} catch (threadError) {
console.warn(`Failed to execute in thread, falling back to main thread: ${threadError}`)
// Fall back to executing in the main thread