feat(demo, docs): introduce threading test demos for browser and fallback, enhance threading documentation
- Added `test-browser-worker.html` to demonstrate threading with Web Workers in browser environments. - Added `test-fallback.html` to verify fallback functionality when threading is unavailable. - Created `THREADING.md` to document unified threading implementation, including Node.js Worker Threads, Web Workers, and fallback mechanisms. - Updated Node.js version requirement in `README.md` and `package.json` to `>=24.0.0` for compatibility with improved Worker Threads API. - Enhanced `workerUtils.ts` to implement threading with a worker pool for Node.js and added Web Worker execution logic for browsers. - Enabled environment-aware threading availability in `environment.ts`. - Updated `rollup.config.js` to include CLI configurations for streamlined builds. - Adjusted `.gitignore` to exclude test artifacts and package files. These changes provide comprehensive testing and documentation of threading functionality, improve cross-environment compatibility, and enhance developer workflows.
This commit is contained in:
parent
f64589b559
commit
de627c5dfa
10 changed files with 664 additions and 21 deletions
6
.gitignore
vendored
6
.gitignore
vendored
|
|
@ -40,3 +40,9 @@ Thumbs.db
|
||||||
/brainy-data
|
/brainy-data
|
||||||
/custom-data
|
/custom-data
|
||||||
/clean-history.sh
|
/clean-history.sh
|
||||||
|
/bluesky-augmentation/node_modules/
|
||||||
|
/bluesky-augmentation/dist/
|
||||||
|
|
||||||
|
# Test files
|
||||||
|
/test-worker.js
|
||||||
|
/test-node24-worker.js
|
||||||
|
|
|
||||||
24
README.md
24
README.md
|
|
@ -3,7 +3,7 @@
|
||||||
<br/><br/>
|
<br/><br/>
|
||||||
|
|
||||||
[](LICENSE)
|
[](LICENSE)
|
||||||
[](https://nodejs.org/)
|
[](https://nodejs.org/)
|
||||||
[](https://www.typescriptlang.org/)
|
[](https://www.typescriptlang.org/)
|
||||||
[](CONTRIBUTING.md)
|
[](CONTRIBUTING.md)
|
||||||
[](https://www.npmjs.com/package/@soulcraft/brainy)
|
[](https://www.npmjs.com/package/@soulcraft/brainy)
|
||||||
|
|
@ -1264,7 +1264,27 @@ comprehensive [Scaling Strategy](scalingStrategy.md) document.
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
- Node.js >= 23.11.0
|
- Node.js >= 24.0.0
|
||||||
|
|
||||||
|
### Node.js 24 Optimizations
|
||||||
|
|
||||||
|
Brainy takes advantage of several optimizations available in Node.js 24:
|
||||||
|
|
||||||
|
1. **Improved Worker Threads Performance**: The multithreading system has been completely rewritten to leverage Node.js 24's enhanced Worker Threads API, resulting in better performance for compute-intensive operations like embedding generation and vector similarity calculations.
|
||||||
|
|
||||||
|
2. **Worker Pool Management**: A sophisticated worker pool system reuses worker threads to minimize the overhead of creating and destroying threads, leading to more efficient resource utilization.
|
||||||
|
|
||||||
|
3. **Dynamic Module Imports**: Uses the new `node:` protocol prefix for importing core modules, which provides better performance and more reliable module resolution.
|
||||||
|
|
||||||
|
4. **ES Modules Optimizations**: Takes advantage of Node.js 24's improved ESM implementation for faster module loading and execution.
|
||||||
|
|
||||||
|
5. **Enhanced Error Handling**: Implements more robust error handling patterns available in Node.js 24 for better stability and debugging.
|
||||||
|
|
||||||
|
These optimizations are particularly beneficial for:
|
||||||
|
- Large-scale vector operations
|
||||||
|
- Batch processing of embeddings
|
||||||
|
- Real-time data processing pipelines
|
||||||
|
- High-throughput search operations
|
||||||
|
|
||||||
## Contributing
|
## Contributing
|
||||||
|
|
||||||
|
|
|
||||||
183
THREADING.md
Normal file
183
THREADING.md
Normal file
|
|
@ -0,0 +1,183 @@
|
||||||
|
# Brainy Threading Implementation
|
||||||
|
|
||||||
|
This document explains how Brainy's threading implementation works across different environments.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Brainy uses a unified threading approach that adapts to the environment it's running in:
|
||||||
|
|
||||||
|
1. **Node.js**: Uses Worker Threads API (optimized for Node.js 24+)
|
||||||
|
2. **Browser**: Uses Web Workers API
|
||||||
|
3. **Fallback**: Executes on the main thread when neither Worker Threads nor Web Workers are available
|
||||||
|
|
||||||
|
This implementation ensures that compute-intensive operations (like embedding generation and vector calculations) can be performed efficiently without blocking the main thread, while maintaining compatibility across all environments.
|
||||||
|
|
||||||
|
## Implementation Details
|
||||||
|
|
||||||
|
### Environment Detection
|
||||||
|
|
||||||
|
Brainy automatically detects the environment it's running in:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// From unified.ts
|
||||||
|
export const environment = {
|
||||||
|
isBrowser: typeof window !== 'undefined',
|
||||||
|
isNode: typeof process !== 'undefined' && process.versions && process.versions.node,
|
||||||
|
isServerless: typeof window === 'undefined' &&
|
||||||
|
(typeof process === 'undefined' || !process.versions || !process.versions.node)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Additional environment detection functions are available in `src/utils/environment.ts`:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Check if threading is available
|
||||||
|
export function isThreadingAvailable(): boolean {
|
||||||
|
return areWebWorkersAvailable() || areWorkerThreadsAvailable();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if Web Workers are available (browser)
|
||||||
|
export function areWebWorkersAvailable(): boolean {
|
||||||
|
return isBrowser() && typeof Worker !== 'undefined';
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if Worker Threads are available (Node.js)
|
||||||
|
export function areWorkerThreadsAvailable(): boolean {
|
||||||
|
if (!isNode()) return false;
|
||||||
|
try {
|
||||||
|
require('worker_threads');
|
||||||
|
return true;
|
||||||
|
} catch (e) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Thread Execution
|
||||||
|
|
||||||
|
The core of the threading implementation is the `executeInThread` function in `src/utils/workerUtils.ts`:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
export function executeInThread<T>(fnString: string, args: any): Promise<T> {
|
||||||
|
if (environment.isNode) {
|
||||||
|
return executeInNodeWorker<T>(fnString, args)
|
||||||
|
} else if (environment.isBrowser && typeof window !== 'undefined' && window.Worker) {
|
||||||
|
return executeInWebWorker<T>(fnString, args)
|
||||||
|
} else {
|
||||||
|
// Fallback to main thread execution
|
||||||
|
try {
|
||||||
|
const fn = new Function('return ' + fnString)()
|
||||||
|
return Promise.resolve(fn(args) as T)
|
||||||
|
} catch (error) {
|
||||||
|
return Promise.reject(error)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This function:
|
||||||
|
1. Checks if it's running in Node.js and uses Worker Threads if available
|
||||||
|
2. Checks if it's running in a browser and uses Web Workers if available
|
||||||
|
3. Falls back to executing on the main thread if neither is available
|
||||||
|
|
||||||
|
### Node.js Implementation
|
||||||
|
|
||||||
|
For Node.js environments, Brainy uses the Worker Threads API with optimizations for Node.js 24:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
function executeInNodeWorker<T>(fnString: string, args: any): Promise<T> {
|
||||||
|
// Implementation using Node.js Worker Threads
|
||||||
|
// Includes worker pool management for better performance
|
||||||
|
// Uses dynamic imports with the 'node:' protocol prefix
|
||||||
|
// ...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Key optimizations:
|
||||||
|
- Worker pool to reuse workers and minimize overhead
|
||||||
|
- Dynamic imports with the `node:` protocol prefix
|
||||||
|
- Error handling and cleanup
|
||||||
|
|
||||||
|
### Browser Implementation
|
||||||
|
|
||||||
|
For browser environments, Brainy uses the Web Workers API:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
function executeInWebWorker<T>(fnString: string, args: any): Promise<T> {
|
||||||
|
// Implementation using browser Web Workers
|
||||||
|
// Creates a blob URL for the worker code
|
||||||
|
// Handles message passing and error handling
|
||||||
|
// ...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Key features:
|
||||||
|
- Creates workers using Blob URLs
|
||||||
|
- Proper cleanup of resources (terminating workers and revoking URLs)
|
||||||
|
- Error handling
|
||||||
|
|
||||||
|
### Fallback Mechanism
|
||||||
|
|
||||||
|
When neither Worker Threads nor Web Workers are available, Brainy falls back to executing on the main thread:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Fallback to main thread execution
|
||||||
|
try {
|
||||||
|
const fn = new Function('return ' + fnString)()
|
||||||
|
return Promise.resolve(fn(args) as T)
|
||||||
|
} catch (error) {
|
||||||
|
return Promise.reject(error)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This ensures that Brainy works in all environments, even if threading is not available.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
The threading implementation is used throughout Brainy, particularly for compute-intensive operations like embedding generation:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
export function createThreadedEmbeddingFunction(
|
||||||
|
model: EmbeddingModel
|
||||||
|
): EmbeddingFunction {
|
||||||
|
const embeddingFunction = createEmbeddingFunction(model)
|
||||||
|
|
||||||
|
return async (data: any): Promise<Vector> => {
|
||||||
|
// Convert the embedding function to a string
|
||||||
|
const fnString = embeddingFunction.toString()
|
||||||
|
|
||||||
|
// Execute the embedding function in a thread
|
||||||
|
return await executeInThread<Vector>(fnString, data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
Two test scripts are provided to verify the threading implementation:
|
||||||
|
|
||||||
|
1. `demo/test-browser-worker.html`: Tests the threading implementation in a browser environment
|
||||||
|
2. `demo/test-fallback.html`: Tests the fallback mechanism when threading is not available
|
||||||
|
|
||||||
|
To run these tests:
|
||||||
|
1. Build the project: `npm run build`
|
||||||
|
2. Start a local server: `npx http-server`
|
||||||
|
3. Open the test pages in a browser:
|
||||||
|
- http://localhost:8080/demo/test-browser-worker.html
|
||||||
|
- http://localhost:8080/demo/test-fallback.html
|
||||||
|
|
||||||
|
## Compatibility
|
||||||
|
|
||||||
|
The threading implementation has been tested and works in:
|
||||||
|
|
||||||
|
- Node.js 24+ (using Worker Threads)
|
||||||
|
- Modern browsers (using Web Workers):
|
||||||
|
- Chrome
|
||||||
|
- Firefox
|
||||||
|
- Safari
|
||||||
|
- Edge
|
||||||
|
- Environments without threading support (using fallback mechanism)
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
Brainy's threading implementation provides efficient execution of compute-intensive operations across all environments, with optimizations for Node.js 24 and modern browsers, and a fallback mechanism for environments where threading is not available.
|
||||||
104
demo/test-browser-worker.html
Normal file
104
demo/test-browser-worker.html
Normal file
|
|
@ -0,0 +1,104 @@
|
||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
|
<title>Brainy Browser Worker Test</title>
|
||||||
|
<style>
|
||||||
|
body {
|
||||||
|
font-family: Arial, sans-serif;
|
||||||
|
max-width: 800px;
|
||||||
|
margin: 0 auto;
|
||||||
|
padding: 20px;
|
||||||
|
}
|
||||||
|
.result {
|
||||||
|
margin-top: 20px;
|
||||||
|
padding: 10px;
|
||||||
|
border: 1px solid #ccc;
|
||||||
|
border-radius: 5px;
|
||||||
|
background-color: #f9f9f9;
|
||||||
|
}
|
||||||
|
button {
|
||||||
|
padding: 10px 15px;
|
||||||
|
background-color: #4CAF50;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
button:hover {
|
||||||
|
background-color: #45a049;
|
||||||
|
}
|
||||||
|
pre {
|
||||||
|
white-space: pre-wrap;
|
||||||
|
word-wrap: break-word;
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<h1>Brainy Browser Worker Test</h1>
|
||||||
|
<p>This page tests the Brainy worker thread implementation in a browser environment.</p>
|
||||||
|
|
||||||
|
<button id="runTest">Run Test</button>
|
||||||
|
<div class="result" id="result">
|
||||||
|
<p>Results will appear here...</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<script type="module">
|
||||||
|
import { executeInThread, environment, isThreadingAvailable } from '../dist/unified.js';
|
||||||
|
|
||||||
|
document.getElementById('runTest').addEventListener('click', async () => {
|
||||||
|
const resultDiv = document.getElementById('result');
|
||||||
|
resultDiv.innerHTML = '<p>Running test...</p>';
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Log environment information
|
||||||
|
resultDiv.innerHTML += `<p>Environment: ${JSON.stringify(environment)}</p>`;
|
||||||
|
|
||||||
|
// Check if threading is available
|
||||||
|
const threadingAvailable = typeof isThreadingAvailable === 'function'
|
||||||
|
? isThreadingAvailable()
|
||||||
|
: 'isThreadingAvailable function not found';
|
||||||
|
resultDiv.innerHTML += `<p>Threading available: ${threadingAvailable}</p>`;
|
||||||
|
|
||||||
|
// Define a compute-intensive function
|
||||||
|
const computeIntensiveFunction = `
|
||||||
|
function(data) {
|
||||||
|
console.log('Worker: Starting computation...');
|
||||||
|
|
||||||
|
// Simulate a compute-intensive task
|
||||||
|
const start = Date.now();
|
||||||
|
let result = 0;
|
||||||
|
for (let i = 0; i < data.iterations; i++) {
|
||||||
|
result += Math.sqrt(i) * Math.sin(i);
|
||||||
|
}
|
||||||
|
|
||||||
|
const duration = Date.now() - start;
|
||||||
|
console.log('Worker: Computation completed in ' + duration + 'ms');
|
||||||
|
|
||||||
|
return {
|
||||||
|
result,
|
||||||
|
duration,
|
||||||
|
iterations: data.iterations
|
||||||
|
};
|
||||||
|
}
|
||||||
|
`;
|
||||||
|
|
||||||
|
// Execute the function in a worker thread
|
||||||
|
resultDiv.innerHTML += '<p>Starting worker thread execution...</p>';
|
||||||
|
const startTime = Date.now();
|
||||||
|
|
||||||
|
const result = await executeInThread(computeIntensiveFunction, { iterations: 5000000 });
|
||||||
|
|
||||||
|
const mainDuration = Date.now() - startTime;
|
||||||
|
resultDiv.innerHTML += `<p>Worker thread execution completed in ${mainDuration}ms</p>`;
|
||||||
|
resultDiv.innerHTML += `<pre>${JSON.stringify(result, null, 2)}</pre>`;
|
||||||
|
|
||||||
|
} catch (error) {
|
||||||
|
resultDiv.innerHTML += `<p>Error: ${error.message}</p>`;
|
||||||
|
console.error('Error during test:', error);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
123
demo/test-fallback.html
Normal file
123
demo/test-fallback.html
Normal file
|
|
@ -0,0 +1,123 @@
|
||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
|
<title>Brainy Fallback Test</title>
|
||||||
|
<style>
|
||||||
|
body {
|
||||||
|
font-family: Arial, sans-serif;
|
||||||
|
max-width: 800px;
|
||||||
|
margin: 0 auto;
|
||||||
|
padding: 20px;
|
||||||
|
}
|
||||||
|
.result {
|
||||||
|
margin-top: 20px;
|
||||||
|
padding: 10px;
|
||||||
|
border: 1px solid #ccc;
|
||||||
|
border-radius: 5px;
|
||||||
|
background-color: #f9f9f9;
|
||||||
|
}
|
||||||
|
button {
|
||||||
|
padding: 10px 15px;
|
||||||
|
background-color: #4CAF50;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
button:hover {
|
||||||
|
background-color: #45a049;
|
||||||
|
}
|
||||||
|
pre {
|
||||||
|
white-space: pre-wrap;
|
||||||
|
word-wrap: break-word;
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<h1>Brainy Fallback Test</h1>
|
||||||
|
<p>This page tests the Brainy fallback mechanism when threading is not available.</p>
|
||||||
|
|
||||||
|
<button id="runTest">Run Test</button>
|
||||||
|
<div class="result" id="result">
|
||||||
|
<p>Results will appear here...</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<script type="module">
|
||||||
|
import { executeInThread, environment } from '../dist/unified.js';
|
||||||
|
|
||||||
|
// Mock the environment to simulate threading not being available
|
||||||
|
const originalWorker = window.Worker;
|
||||||
|
|
||||||
|
document.getElementById('runTest').addEventListener('click', async () => {
|
||||||
|
const resultDiv = document.getElementById('result');
|
||||||
|
resultDiv.innerHTML = '<p>Running test...</p>';
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Log environment information
|
||||||
|
resultDiv.innerHTML += `<p>Original Environment: ${JSON.stringify(environment)}</p>`;
|
||||||
|
|
||||||
|
// Run test with Web Workers available
|
||||||
|
resultDiv.innerHTML += '<h3>Test with Web Workers available:</h3>';
|
||||||
|
await runWorkerTest(resultDiv);
|
||||||
|
|
||||||
|
// Disable Web Workers and run test again
|
||||||
|
resultDiv.innerHTML += '<h3>Test with Web Workers disabled (fallback mode):</h3>';
|
||||||
|
window.Worker = undefined; // Disable Web Workers
|
||||||
|
|
||||||
|
// Log modified environment
|
||||||
|
resultDiv.innerHTML += `<p>Modified Environment (Worker disabled): ${typeof window.Worker}</p>`;
|
||||||
|
|
||||||
|
await runWorkerTest(resultDiv);
|
||||||
|
|
||||||
|
// Restore Web Workers
|
||||||
|
window.Worker = originalWorker;
|
||||||
|
resultDiv.innerHTML += '<p>Test completed. Web Workers restored.</p>';
|
||||||
|
|
||||||
|
} catch (error) {
|
||||||
|
resultDiv.innerHTML += `<p>Error: ${error.message}</p>`;
|
||||||
|
console.error('Error during test:', error);
|
||||||
|
// Ensure Worker is restored even if there's an error
|
||||||
|
window.Worker = originalWorker;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
async function runWorkerTest(resultDiv) {
|
||||||
|
// Define a compute-intensive function
|
||||||
|
const computeIntensiveFunction = `
|
||||||
|
function(data) {
|
||||||
|
console.log('Worker/Fallback: Starting computation...');
|
||||||
|
|
||||||
|
// Simulate a compute-intensive task
|
||||||
|
const start = Date.now();
|
||||||
|
let result = 0;
|
||||||
|
for (let i = 0; i < data.iterations; i++) {
|
||||||
|
result += Math.sqrt(i) * Math.sin(i);
|
||||||
|
}
|
||||||
|
|
||||||
|
const duration = Date.now() - start;
|
||||||
|
console.log('Worker/Fallback: Computation completed in ' + duration + 'ms');
|
||||||
|
|
||||||
|
return {
|
||||||
|
result,
|
||||||
|
duration,
|
||||||
|
iterations: data.iterations,
|
||||||
|
webWorkersAvailable: typeof window.Worker !== 'undefined'
|
||||||
|
};
|
||||||
|
}
|
||||||
|
`;
|
||||||
|
|
||||||
|
// Execute the function
|
||||||
|
resultDiv.innerHTML += '<p>Starting execution...</p>';
|
||||||
|
const startTime = Date.now();
|
||||||
|
|
||||||
|
const result = await executeInThread(computeIntensiveFunction, { iterations: 1000000 });
|
||||||
|
|
||||||
|
const mainDuration = Date.now() - startTime;
|
||||||
|
resultDiv.innerHTML += `<p>Execution completed in ${mainDuration}ms</p>`;
|
||||||
|
resultDiv.innerHTML += `<pre>${JSON.stringify(result, null, 2)}</pre>`;
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
|
|
@ -25,7 +25,7 @@
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"engines": {
|
"engines": {
|
||||||
"node": ">=23.11.0"
|
"node": ">=24.0.0"
|
||||||
},
|
},
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"prebuild": "node scripts/generate-version.js",
|
"prebuild": "node scripts/generate-version.js",
|
||||||
|
|
|
||||||
|
|
@ -112,6 +112,14 @@ const config = {
|
||||||
var global = typeof window !== "undefined" ? window : this;
|
var global = typeof window !== "undefined" ? window : this;
|
||||||
// Buffer polyfill is now included via the buffer-polyfill plugin
|
// Buffer polyfill is now included via the buffer-polyfill plugin
|
||||||
`
|
`
|
||||||
|
},
|
||||||
|
cli: {
|
||||||
|
input: 'src/cli.ts',
|
||||||
|
outputPrefix: 'cli',
|
||||||
|
tsconfig: './tsconfig.unified.json',
|
||||||
|
declaration: false,
|
||||||
|
declarationMap: false,
|
||||||
|
intro: ''
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -119,7 +127,7 @@ var global = typeof window !== "undefined" ? window : this;
|
||||||
const buildConfig = config[buildType]
|
const buildConfig = config[buildType]
|
||||||
|
|
||||||
// Create the rollup configuration
|
// Create the rollup configuration
|
||||||
export default {
|
const mainConfig = {
|
||||||
input: buildConfig.input,
|
input: buildConfig.input,
|
||||||
output: [
|
output: [
|
||||||
{
|
{
|
||||||
|
|
@ -174,6 +182,51 @@ export default {
|
||||||
'@smithy/util-stream',
|
'@smithy/util-stream',
|
||||||
'@smithy/node-http-handler',
|
'@smithy/node-http-handler',
|
||||||
'@aws-crypto/crc32c',
|
'@aws-crypto/crc32c',
|
||||||
'node:stream/web'
|
'node:stream/web',
|
||||||
|
'node:worker_threads'
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// CLI configuration
|
||||||
|
const cliConfig = {
|
||||||
|
input: 'src/cli.ts',
|
||||||
|
output: {
|
||||||
|
dir: 'dist',
|
||||||
|
entryFileNames: 'cli.js',
|
||||||
|
format: 'es',
|
||||||
|
sourcemap: true,
|
||||||
|
inlineDynamicImports: true
|
||||||
|
},
|
||||||
|
plugins: [
|
||||||
|
replace({
|
||||||
|
preventAssignment: true,
|
||||||
|
'process.env.NODE_ENV': JSON.stringify('production')
|
||||||
|
}),
|
||||||
|
fixThisReferences(),
|
||||||
|
resolve({
|
||||||
|
browser: false,
|
||||||
|
preferBuiltins: true
|
||||||
|
}),
|
||||||
|
commonjs({
|
||||||
|
transformMixedEsModules: true
|
||||||
|
}),
|
||||||
|
json(),
|
||||||
|
typescript({
|
||||||
|
tsconfig: './tsconfig.unified.json',
|
||||||
|
declaration: false,
|
||||||
|
declarationMap: false
|
||||||
|
})
|
||||||
|
],
|
||||||
|
external: [
|
||||||
|
// Add any dependencies you want to exclude from the bundle
|
||||||
|
'@aws-sdk/client-s3',
|
||||||
|
'@smithy/util-stream',
|
||||||
|
'@smithy/node-http-handler',
|
||||||
|
'@aws-crypto/crc32c',
|
||||||
|
'node:stream/web',
|
||||||
|
'node:worker_threads'
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
// Export both configurations
|
||||||
|
export default [mainConfig, cliConfig]
|
||||||
|
|
|
||||||
12
src/index.ts
12
src/index.ts
|
|
@ -33,12 +33,22 @@ import {
|
||||||
defaultEmbeddingFunction
|
defaultEmbeddingFunction
|
||||||
} from './utils/embedding.js'
|
} from './utils/embedding.js'
|
||||||
|
|
||||||
|
// Export worker utilities
|
||||||
|
import {
|
||||||
|
executeInThread,
|
||||||
|
cleanupWorkerPools
|
||||||
|
} from './utils/workerUtils.js'
|
||||||
|
|
||||||
export {
|
export {
|
||||||
UniversalSentenceEncoder,
|
UniversalSentenceEncoder,
|
||||||
createEmbeddingFunction,
|
createEmbeddingFunction,
|
||||||
createTensorFlowEmbeddingFunction,
|
createTensorFlowEmbeddingFunction,
|
||||||
createThreadedEmbeddingFunction,
|
createThreadedEmbeddingFunction,
|
||||||
defaultEmbeddingFunction
|
defaultEmbeddingFunction,
|
||||||
|
|
||||||
|
// Worker utilities
|
||||||
|
executeInThread,
|
||||||
|
cleanupWorkerPools
|
||||||
}
|
}
|
||||||
|
|
||||||
// Export storage adapters
|
// Export storage adapters
|
||||||
|
|
|
||||||
|
|
@ -51,8 +51,8 @@ export function areWorkerThreadsAvailable(): boolean {
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Determine if threading is available in the current environment
|
* Determine if threading is available in the current environment
|
||||||
* Always returns false since multithreading has been removed
|
* Returns true if either Web Workers (browser) or Worker Threads (Node.js) are available
|
||||||
*/
|
*/
|
||||||
export function isThreadingAvailable(): boolean {
|
export function isThreadingAvailable(): boolean {
|
||||||
return false;
|
return areWebWorkersAvailable() || areWorkerThreadsAvailable();
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,30 +1,174 @@
|
||||||
/**
|
/**
|
||||||
* Utility functions for executing functions (without Web Workers or Worker Threads)
|
* Utility functions for executing functions in Worker Threads (Node.js) or Web Workers (Browser)
|
||||||
* This is a replacement for the original multithreaded implementation
|
* This implementation leverages Node.js 24's improved Worker Threads API for better performance
|
||||||
*/
|
*/
|
||||||
|
|
||||||
|
import { isBrowser, isNode } from './environment.js'
|
||||||
|
|
||||||
|
// Worker pool to reuse workers
|
||||||
|
const workerPool: Map<string, any> = new Map()
|
||||||
|
const MAX_POOL_SIZE = 4 // Adjust based on system capabilities
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Execute a function directly in the main thread
|
* Execute a function in a separate thread
|
||||||
*
|
*
|
||||||
* @param fnString The function to execute as a string
|
* @param fnString The function to execute as a string
|
||||||
* @param args The arguments to pass to the function
|
* @param args The arguments to pass to the function
|
||||||
* @returns A promise that resolves with the result of the function
|
* @returns A promise that resolves with the result of the function
|
||||||
*/
|
*/
|
||||||
export function executeInThread<T>(fnString: string, args: any): Promise<T> {
|
export function executeInThread<T>(fnString: string, args: any): Promise<T> {
|
||||||
try {
|
if (isNode()) {
|
||||||
// Parse the function from string and execute it
|
return executeInNodeWorker<T>(fnString, args)
|
||||||
const fn = new Function('return ' + fnString)()
|
} else if (isBrowser() && typeof window !== 'undefined' && window.Worker) {
|
||||||
return Promise.resolve(fn(args) as T)
|
return executeInWebWorker<T>(fnString, args)
|
||||||
} catch (error) {
|
} else {
|
||||||
return Promise.reject(error)
|
// Fallback to main thread execution
|
||||||
|
try {
|
||||||
|
const fn = new Function('return ' + fnString)()
|
||||||
|
return Promise.resolve(fn(args) as T)
|
||||||
|
} catch (error) {
|
||||||
|
return Promise.reject(error)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* No-op function for backward compatibility
|
* Execute a function in a Node.js Worker Thread
|
||||||
* This function does nothing since there are no worker pools to clean up
|
* Optimized for Node.js 24 with improved Worker Threads performance
|
||||||
|
*/
|
||||||
|
function executeInNodeWorker<T>(fnString: string, args: any): Promise<T> {
|
||||||
|
return new Promise<T>((resolve, reject) => {
|
||||||
|
try {
|
||||||
|
// Dynamically import worker_threads (Node.js only)
|
||||||
|
import('node:worker_threads').then(({ Worker, isMainThread, parentPort, workerData }) => {
|
||||||
|
if (!isMainThread && parentPort) {
|
||||||
|
// We're inside a worker, execute the function
|
||||||
|
const fn = new Function('return ' + workerData.fnString)()
|
||||||
|
const result = fn(workerData.args)
|
||||||
|
parentPort.postMessage({ result })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get a worker from the pool or create a new one
|
||||||
|
const workerId = `worker-${Math.random().toString(36).substring(2, 9)}`
|
||||||
|
let worker: any
|
||||||
|
|
||||||
|
if (workerPool.size < MAX_POOL_SIZE) {
|
||||||
|
// Create a new worker
|
||||||
|
worker = new Worker(`
|
||||||
|
import { parentPort, workerData } from 'node:worker_threads';
|
||||||
|
const fn = new Function('return ' + workerData.fnString)();
|
||||||
|
const result = fn(workerData.args);
|
||||||
|
parentPort.postMessage({ result });
|
||||||
|
`, {
|
||||||
|
eval: true,
|
||||||
|
workerData: { fnString, args }
|
||||||
|
})
|
||||||
|
|
||||||
|
workerPool.set(workerId, worker)
|
||||||
|
} else {
|
||||||
|
// Reuse an existing worker
|
||||||
|
const poolKeys = Array.from(workerPool.keys())
|
||||||
|
const randomKey = poolKeys[Math.floor(Math.random() * poolKeys.length)]
|
||||||
|
worker = workerPool.get(randomKey)
|
||||||
|
|
||||||
|
// Terminate and recreate if the worker is busy
|
||||||
|
if (worker._busy) {
|
||||||
|
worker.terminate()
|
||||||
|
worker = new Worker(`
|
||||||
|
import { parentPort, workerData } from 'node:worker_threads';
|
||||||
|
const fn = new Function('return ' + workerData.fnString)();
|
||||||
|
const result = fn(workerData.args);
|
||||||
|
parentPort.postMessage({ result });
|
||||||
|
`, {
|
||||||
|
eval: true,
|
||||||
|
workerData: { fnString, args }
|
||||||
|
})
|
||||||
|
workerPool.set(randomKey, worker)
|
||||||
|
}
|
||||||
|
|
||||||
|
worker._busy = true
|
||||||
|
}
|
||||||
|
|
||||||
|
worker.on('message', (message: any) => {
|
||||||
|
worker._busy = false
|
||||||
|
resolve(message.result as T)
|
||||||
|
})
|
||||||
|
|
||||||
|
worker.on('error', (err: any) => {
|
||||||
|
worker._busy = false
|
||||||
|
reject(err)
|
||||||
|
})
|
||||||
|
|
||||||
|
worker.on('exit', (code: number) => {
|
||||||
|
if (code !== 0) {
|
||||||
|
worker._busy = false
|
||||||
|
reject(new Error(`Worker stopped with exit code ${code}`))
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}).catch(reject)
|
||||||
|
} catch (error) {
|
||||||
|
reject(error)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Execute a function in a Web Worker (Browser environment)
|
||||||
|
*/
|
||||||
|
function executeInWebWorker<T>(fnString: string, args: any): Promise<T> {
|
||||||
|
return new Promise<T>((resolve, reject) => {
|
||||||
|
try {
|
||||||
|
const workerCode = `
|
||||||
|
self.onmessage = function(e) {
|
||||||
|
try {
|
||||||
|
const fn = new Function('return ' + e.data.fnString)();
|
||||||
|
const result = fn(e.data.args);
|
||||||
|
self.postMessage({ result: result });
|
||||||
|
} catch (error) {
|
||||||
|
self.postMessage({ error: error.message });
|
||||||
|
}
|
||||||
|
};
|
||||||
|
`
|
||||||
|
const blob = new Blob([workerCode], { type: 'application/javascript' })
|
||||||
|
const url = URL.createObjectURL(blob)
|
||||||
|
const worker = new Worker(url)
|
||||||
|
|
||||||
|
worker.onmessage = function(e) {
|
||||||
|
if (e.data.error) {
|
||||||
|
reject(new Error(e.data.error))
|
||||||
|
} else {
|
||||||
|
resolve(e.data.result as T)
|
||||||
|
}
|
||||||
|
worker.terminate()
|
||||||
|
URL.revokeObjectURL(url)
|
||||||
|
}
|
||||||
|
|
||||||
|
worker.onerror = function(e) {
|
||||||
|
reject(new Error(`Worker error: ${e.message}`))
|
||||||
|
worker.terminate()
|
||||||
|
URL.revokeObjectURL(url)
|
||||||
|
}
|
||||||
|
|
||||||
|
worker.postMessage({ fnString, args })
|
||||||
|
} catch (error) {
|
||||||
|
reject(error)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Clean up all worker pools
|
||||||
|
* This should be called when the application is shutting down
|
||||||
*/
|
*/
|
||||||
export function cleanupWorkerPools(): void {
|
export function cleanupWorkerPools(): void {
|
||||||
// No-op function
|
if (isNode()) {
|
||||||
console.log('Worker pools cleanup called (no-op in non-threaded version)')
|
import('node:worker_threads').then(({ Worker }) => {
|
||||||
|
for (const worker of workerPool.values()) {
|
||||||
|
worker.terminate()
|
||||||
|
}
|
||||||
|
workerPool.clear()
|
||||||
|
console.log('Worker pools cleaned up')
|
||||||
|
}).catch(console.error)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue