David Snelling
3ec183dab6
- **Vector Handling Updates**: - Added a `dimensions` property to `BrainyDataConfig` for specifying vector dimensions. - Introduced validation for vector dimensions during database creation and insertion to ensure consistency. - Enhanced error handling and logging for dimension mismatches. - **Model Loading Improvements**: - Implemented retry logic for Universal Sentence Encoder model loading to handle network instability and JSON parsing errors gracefully. - Improved logging and debugging support for failures during model initialization and embedding operations. - **Compatibility Enhancements**: - Updated polyfills to support TensorFlow.js compatibility across diverse server environments (Node.js, serverless, etc.). - Introduced and refactored global `TextEncoder`/`TextDecoder` definitions for seamless operation in non-browser environments. - Simplified TensorFlow.js backend setup with streamlined imports and logging for GPU/WebGL fallback. - **Purpose**: - These updates improve BrainyData's robustness, enforce correct vector usage, and extend compatibility with varied runtime environments. The changes enhance the usability, reliability, and cross-platform readiness of core functionalities.
46 lines
1.8 KiB
TypeScript
46 lines
1.8 KiB
TypeScript
/**
|
|
* CRITICAL: This file is imported for its side effects to patch the environment
|
|
* for TensorFlow.js before any other library code runs.
|
|
*
|
|
* It ensures that by the time TensorFlow.js is imported by any other
|
|
* module, the necessary compatibility fixes for the current Node.js
|
|
* environment are already in place.
|
|
*
|
|
* This file MUST be imported as the first import in unified.ts to prevent
|
|
* race conditions with TensorFlow.js initialization. Failure to do so will
|
|
* result in errors like "TextEncoder is not a constructor" when the package
|
|
* is used in Node.js environments.
|
|
*
|
|
* The package.json file marks this file as having side effects to prevent
|
|
* tree-shaking by bundlers, ensuring the patch is always applied.
|
|
*/
|
|
|
|
// Get the appropriate global object for the current environment
|
|
const globalObj = (() => {
|
|
if (typeof globalThis !== 'undefined') return globalThis
|
|
if (typeof global !== 'undefined') return global
|
|
if (typeof self !== 'undefined') return self
|
|
return null // No global object available
|
|
})()
|
|
|
|
// Define TextEncoder and TextDecoder globally to make sure they're available
|
|
// Now works across all environments: Node.js, serverless, and other server environments
|
|
if (globalObj) {
|
|
if (!globalObj.TextEncoder) {
|
|
globalObj.TextEncoder = TextEncoder
|
|
}
|
|
if (!globalObj.TextDecoder) {
|
|
globalObj.TextDecoder = TextDecoder
|
|
}
|
|
|
|
// Create a special global constructor that TensorFlow can use safely
|
|
;(globalObj as any).__TextEncoder__ = TextEncoder
|
|
;(globalObj as any).__TextDecoder__ = TextDecoder
|
|
}
|
|
|
|
// Also import normally for ES modules environments
|
|
import { applyTensorFlowPatch } from './utils/textEncoding.js'
|
|
|
|
// Apply the TensorFlow.js platform patch
|
|
applyTensorFlowPatch()
|
|
console.log('Applied TensorFlow.js patch via ES modules in setup.ts')
|