**feat(core): enhance vector handling, model loading, and compatibility**

- **Vector Handling Updates**:
  - Added a `dimensions` property to `BrainyDataConfig` for specifying vector dimensions.
  - Introduced validation for vector dimensions during database creation and insertion to ensure consistency.
  - Enhanced error handling and logging for dimension mismatches.

- **Model Loading Improvements**:
  - Implemented retry logic for Universal Sentence Encoder model loading to handle network instability and JSON parsing errors gracefully.
  - Improved logging and debugging support for failures during model initialization and embedding operations.

- **Compatibility Enhancements**:
  - Updated polyfills to support TensorFlow.js compatibility across diverse server environments (Node.js, serverless, etc.).
  - Introduced and refactored global `TextEncoder`/`TextDecoder` definitions for seamless operation in non-browser environments.
  - Simplified TensorFlow.js backend setup with streamlined imports and logging for GPU/WebGL fallback.

- **Purpose**:
  - These updates improve BrainyData's robustness, enforce correct vector usage, and extend compatibility with varied runtime environments. The changes enhance the usability, reliability, and cross-platform readiness of core functionalities.
This commit is contained in:
David Snelling 2025-07-16 13:51:00 -07:00
parent ad4af27385
commit 3ec183dab6
9 changed files with 828 additions and 365 deletions

View file

@ -16,16 +16,20 @@
// available to TensorFlow.js before it initializes its platform detection
import './setup.js'
// Export environment information
// Import environment detection functions
import { isBrowser, isNode } from './utils/environment.js'
// Export environment information with lazy evaluation
export const environment = {
isBrowser: typeof window !== 'undefined',
isNode:
typeof process !== 'undefined' && process.versions && process.versions.node,
isServerless:
typeof window === 'undefined' &&
(typeof process === 'undefined' ||
!process.versions ||
!process.versions.node)
get isBrowser() {
return isBrowser()
},
get isNode() {
return isNode()
},
get isServerless() {
return !isBrowser() && !isNode()
}
}
// Make environment information available globally
@ -46,3 +50,6 @@ console.log(
// Re-export everything from index.ts
export * from './index.js'
// Export the TensorFlow patch function for testing and manual use
export { applyTensorFlowPatch } from './utils/textEncoding.js'